|
 |
 |
| ||||
|
Summary |
I. Context Recognition |
II. Training, System Customization and Performance Monitoring SummaryCommunication processing includes the computer-based capabilities that support a person's ability to efficiently construct language for communication or literacy. Augmented communicators, researchers and manufacturers have broadly identified high priority needs for increased communication rate and clarity, improved literacy, ease of use and system optimization, performance standards, testing and monitoring, and language acquisition. Improved communication processing may draw upon advances in computer hardware and software, artificial intelligence, knowledge representation, performance monitoring, speech recognition and production, Internet-based innovations, and telecommunications. The T2RERC, along with our customer, industry and research partners are seeking emerging technologies that will improve communication processing. We are specifically looking for communication processing capabilities related to: [ Top of Page ] Context RecognitionNatural communication and literate composition is shaped by contextual information. AAC systems can derive contextual information from the user's own speech, interlocutor speech (i.e. through voice recognition), wireless technologies (i.e. Bluetooth, wireless LANs), and the global positioning system (time, date, and locale information). Contextual information (from various sources) could be used to modify the contents of dynamic pages, adapt page structure and the links between pages; improve word, phrase and sentence prediction and support powerful text storage / retrieval paradigms. Description of the ProblemNatural communication and literacy is shaped by contextual information, including the user's previous dialogue, communication partner's speech, location, time-of-day, activity, and environment. AAC systems augment a person's ability to produce literate written and oral communication - generally however, the communication processing capabilities of these systems do not utilize contextual information. Context-based communication processing may initially be more suitable for literate composition (e.g. coherent, slowly evolving context) than interactive communication (i.e. rapidly changing context). Contextual information (from various sources) could be used to modify the contents of dynamic pages, adapt page structure and the links between pages; improve word, phrase and sentence prediction and support powerful text storage / retrieval paradigms. Interlocutor speech is an important source of contextual information. Speech recognition technology is improving rapidly but performance depends upon environmental factors (e.g. single versus multiple speakers, quiet vs. noisy, speaker position and orientation relative to the microphone, speech quality, etc) and the speech recognition engine (hardware, software, microphone). Recent technological advances, such as, adaptive beam forming microphones have been employed to improve performance of commercial speech recognition software. Speech recognition software now identifies individual words - but does not capture the information found in how these words are said (i.e. tone, inflection). The user's speech is perhaps the richest source of contextual information (e.g. a person may be at a football game but talk about chemistry class). Some word prediction systems are being developed that take contextual information to adapt word lists corresponding to the situation (i.e. specific word list for English class vs. a word list for chemistry class). Wireless communication (e.g. Bluetooth, wireless LANs) can provide contextual information not available in natural communication - thereby enabling (in principle) a higher level of human communication. Wireless communication could be used to automatically download information to the AAC device (e.g. an individual enters a restaurant and the menu is downloaded, they approach a perfume counter and the merchandise listing is downloaded, etc) and communication processing would adapt in response to this information. In general, wireless access to personal computers and Internet resources is highly desirable. Global Positioning System technology (GPS) is principally used for navigation (location, time of day, date, and way finding). Contextual information provided by GPS is probably not sufficient to shape general communication processing - however GPS could provide information about location and way finding (i.e. going home, going to work) and support communication related to these activities. Buildings and geographical features can interfere with GPS signals but there are continuing improvements in this respect. GPS systems utilizing time of day information can provide the user with time specific "appropriate" vocabulary (i.e. good morning, good afternoon). The levels of ambient light and noise provide simple but important contextual information. Ambient noise level sensed via microphone can be used to adjust output volume (louder room/car/outdoor setting). Low-tech noise level detection systems have been incorporated into cars (i.e. when the car speeds up the radio volume increases gradually with the speed, takes into account engine and road noise). Ambient lighting levels sensed via a photocell can be used to adjust display brightness and contrast. AAC consumers may fear that communication processing which 'intelligently' anticipates words, phrases, sentence structure, etc. might 'bias their choice of vocabulary' and cause them to 'lose control of the conversation.' The user must therefore maintain control of context recognition capabilities in order to ensure their acceptance. The user should be able to select specific features to be activated/deactivated or adjusted in degree (i.e. interlocutor speech on/off, length of word lists). Manual controls should be easy to access and intuitive to use. Technology RequirementsThere is a clear need for advanced biosignal-based systems for AAC and related computer systems. The following requirements provide a guideline for technology solutions. It is not expected that any particular solution will satisfy all requirements.
[ Top of Page ] TRAINING, SYSTEM CUSTOMIZATION AND PERFORMANCE MONITORINGText and icon-based AAC systems are needed that provide a learning environment for language acquisition and literacy. Proficient communication and literacy are dependent upon system capabilities, ease of use and optimization, quality of training (generally provided by a clinical or educational professional), and the ability to integrate the system with clinical and educational objectives. Systems need to be transparently usable "out of the box" and provide more advanced capabilities in a natural intuitive manner as the user's abilities grow. Performance standards, monitoring and testing is needed to determine how well an AAC system is being employed for communication and as a clinical or educational intervention. Description of the ProblemResearch has demonstrated that children with severe motor and speech disorders learn language differently than normally developing children, wherein, their lack of speech puts them at a disadvantage for learning language. [8] Conclusions drawn from this research may also apply to individuals with mental retardation or autism, persons re-acquiring language and individuals with cognitive deficits. Language representation and acquisition with AAC systems for children is difficult and these children are often introduced into the classroom environment before they have sufficient language mastery. Children differ in their preference (or capabilities) for direct construction (i.e. spelling out words) or icon-based language acquisition. Icon-based language is especially difficult to learn (e.g. icon meaning changes with communication context and the user must memorize and understand this conceptually). Manufacturers typically train clinicians on the capabilities and use of their products. Many clinicians do not receive this training however and one manufacturer's training does not generalize to another manufacturer's products. Clinicians often find AAC systems complicated and may prescribe from amongst a limited set of AAC systems because of the time and effort required to learn new systems. Mastering the AAC system may be a clinician's primary focus rather than how to best apply this system as a clinical intervention or to achieve educational objectives. AAC systems should reflect the language and intervention models of clinicians. Clinicians could also benefit from training in device maintenance and repair strategies to reduce abandonment when systems break down. Online and hardcopy easy-to-follow 'flow charts' would allow users to complete basic repairs and maintenance (i.e. battery system dies, programming or software is lost, how to reboot) "Write-to-read programs" have been developed for mainstream education to support emerging literacy (critical thinking in literary skills, phonemic awareness, writing, composing, etc) and may provide insights for similar capabilities developed for AAC systems (see for instance: http://www.nps.k12.nj.us/camden_st/ write_to_read.htm). Grounded language learning should be considered (learn the meaning of a word or icon via auditory and visual input). Persons using AAC systems should be able to do more of the setup and optimization themselves. AAC systems should recognize problems (e.g. in set up, performance, optimization) and provide feedback suggesting how these problems can be corrected. A "wizard" should be available to help set up, optimize and correct problems (e.g. similar to the wizard that comes with many Microsoft products). Built in training (e.g. tutorials, help capabilities) is needed for clinicians and users (e.g. how to set up and optimize the AAC device in order to enhance communication, literacy, education etc.). Built in training should be user-friendly (e.g. encourage use, explore system capabilities without severe "penalties") and help prevent errors (e.g. guide the user by providing reasonable alternatives, suggest how to correct mistakes) rather than correcting errors after mistakes have already been made. Systems should operate in three different modes for optimal use: 1) training mode (individual can make changes and learn system without making permanent modifications to the device) 2) modification mode (clinician or user is actually programming the system) and 3) user mode (standard mode for user to access there system). System should allow the user to improve current skills or learn new skills. "Software programs" are needed that run on AAC devices to meet specific/targeted needs (e.g. help attain educational goals). Traditionally, speech language pathologists tracked communication performance using pen, paper and video - which can be cumbersome and expensive. Recent advances in system monitoring and remote data logging permit the clinician to collect detailed logs of user-device performance. With the adoption of a standard log file format by manufacturers, development of data analysis software and validation of clinical measures based upon this data, an objective analysis of the rate and quality of communication and language production is becoming possible. [1,2] AAC systems should track user performance (e.g. selection errors, spelling errors) and adapt or adjust features (e.g. display setup, increased word prediction list length) in response to changing user abilities (e.g. decreased physical precision, increased number of spelling errors). Technology RequirementsThere is a priority need for AAC systems with improved training and customization capabilities. The following statements provide a guideline for technology solutions. It is not expected that any particular solution will satisfy all requirements.
REFERENCES
[ Top of Page ] MESSAGE CREATION AND REFINEMENTAAC systems should support both literate composition and 'real-time' communication. Communication processing should support language construction as quickly as possible at an appropriate level of precision. Communication processing capabilities should be accessible with any selection protocol. Use of AAC systems should be transparent, automatic, and minimize the need for meta-linguistic processing. AAC system capabilities should include 'smart' semantic editing, error correction and efficient access to a dictionary and thesaurus. Description of the ProblemCommunication processing (CP) capabilities of an AAC system should be flexible - to do whatever the person needs to do (e.g. literate composition and 'real-time' communication) as quickly as possible with an appropriate level of precision. An ideal system will probably have a combination of both generative and pre-stored text capabilities. There is always a trade-off between increased communication rate and decreased communication precision. Communication is the two-way exchange of information between individuals to establish a mutual understanding. A person using an AAC system needs to follow a discussion and quickly participate - otherwise they may "lose" the topic as they compose; the discussion may pass on to other topics; or listeners may become impatient as they wait for comments. Communication rate is "correlated" to perceptions of competence (by self and others). Rate is impacted by cultural and situational conventions (e.g. slang and dialect specific words won't show up in a "standard" word prediction list) and individual preferences (e.g. some AAC users allow communication partners to correct "speech errors" while other users prefer to correct errors themselves). Communication processing should build upon rather than replace the user's communication abilities (e.g. residual speech, gestures). For example, a person may augment their speech with an AAC system when communicating with a family member but in a formal presentation use the AAC system exclusively. Literate composition is the ability to generate, read and comprehend, complete, logical, detailed and grammatically correct verbal dialogue and written text (e.g. poetry, technical writing, thesis, and dissertations, etc). Literacy is a critical skill needed in education, the workplace, preparation of public presentations and for independence (e.g. banking, medical care). Commercial word processing software employs rate enhancement tools (e.g. spell checking, grammar checking, automatic error correction, punctuation management, text string search). With more advanced word prediction, a person inputs a letter sequence and words most commonly used by this user are offered in the prediction list (rather than a list based upon word frequency in generic communication). Some AAC systems provide abbreviation expansion and word prediction tools to increase communication rate. In recent years, more advanced language-processing techniques such as "compansion" and "disambiguation" have been introduced. In 'natural communication,' the listener uses their innate language-processing capabilities to expand upon and understand a person's telegraphic expression. In compansion (COMpressed message exPANSION) a telegraphic input ("John Eat Apple") is transformed into well-formed sentences ("John has eaten the apple"). In this case however, the AAC system must expand upon and understand the telegraphic expression. Compansion may support higher communication rates without requiring a great deal of cognitive effort and has been used as a writing, therapy or learning tool for different user populations. It may be necessary to edit or correct 'expanded' input - this action takes time and decreases performance. Compansion and other telegraphic techniques have good potential for 'real-time' communication and 'off-line' literate composition [1]. A disambiguation interface is comprised of a small set of selectable items with multiple "letters" or "symbols" on each item. As items are selected the system predicts candidate words. Additional item selection narrows down the possible target words. For example selecting the item 'a-b-c-d' followed by the item 'l-m-o-p' can mean 'dog' but not 'duck'. Disambiguation increases selection rate for many input techniques by reducing the physical distance between selectable items (important when a person has limited range of motion). It allows bigger keys to be placed on smaller displays (reducing needed physical precision), and reducing input interface complexity. Disambiguation has great potential for eye gaze since "easy to select" large items can be placed on smaller displays. With scanning, selection rates could be '4 times faster' if there are four symbols per selectable item. A large disambiguation system can be used as an accessible writing tool to acquire early literacy. Disambiguation now requires that words be spelled correctly (i.e. no error correction - 'a-b-c-d' followed by 'l-m-o-p' could be 'dock' but not 'duck'). A user may sometimes be forced to choose from amongst viable alternative words (phrases) - this action takes time and decreases performance. Disambiguation interfaces are now available on some cell-phones and AAC systems [2]. Scan-based selection is very slow and requires that the user follow cursor position using visual and/or auditory cues. Scanning typically utilizes a fixed 'grid' of selectable items accessed through a variety of scanning protocols (e.g. row-column, circular, block). Higher dimensional scanning (analogous to linked dynamic-pages) increases the user's cognitive and memory burden but offers a more powerful strategy for language organization and access. In adaptive scanning, grid content, grid organization and/or scanning characteristics change in response to the prior sequence of selected items. The goal of adaptation is to make items having a higher likelihood of being selected more readily available. In principle, any source of contextual information could cue adaptation. With higher-dimensional scanning, the user employs cognitive strategies (e.g. memorization, visualization) to locate 'hidden' items. Adaptive higher-dimensional scanning may (or may not) increase cognitive burden. In general, keystroke savings are more critical for persons using slow selection techniques (e.g. scanning, Morse code, other switch-based approaches) than for persons using fast direct-selection techniques. Scanning is a particularly important selection technique for persons with ALS and individuals with severe motoric impairments. It is generally desirable that communication processing should prevent problems by shifting the burden of responsibility from the user to system (e.g. for error correction, correcting verb tense, etc) rather than assisting the user after an error is made. Error correction slows down communication rate. Communication processing should recognize spelling errors, 'take the user to the error' and provide spelling alternatives. Communication processing should also correct common spelling errors and word omissions, expand contractions, provide punctuation, and correct verb tense. In general, the user should be able accept or reject the corrections shown. Communication processing should adapt to poor or idiosyncratic spelling (e.g. letter reversal, dyslexia, idiosyncratic) and recognize individual spelling (misspelling) tendencies. In general, an AAC system that provides capabilities associated with a personal computer (e.g. word processing, spreadsheets, databases, Internet access, etc.) will enable the user to explore and develop writing skills, computer skills and access to educational / informational resources. Technology RequirementsThere is a clear need for advanced software and/or hardware for improved message creation capabilities of AAC devices and related computer systems. The following "requirements" provide guidelines for technology solutions - though it is not expected that all requirements will be satisfied in any single solution. Disambiguation
Compansion (and related techniques)
Literacy Tools
References
[ Top of Page ] Text Storage and RetrievalAAC systems with capabilities for efficiently storing, finding, retrieving, merging and modifying pre-stored text could significantly improve the rate and quality of communication and literate composition. Significant improvements are needed in system hardware and software capabilities in order to fully achieve these benefits. Description of the ProblemOptimal language production for communication and literate composition probably requires a mix of generative and pre-stored communication processing (CP) capabilities. An AAC system that efficiently stores, reduces, and organizes text and also provides mechanisms to find, retrieve, merge and modify this text mimics human capabilities for memory, cognitive processing and language production. From this perspective - finding, merging and modifying pre-stored text is an alternative path by which to generate language---augmenting rather than replacing other generative capabilities. It is likely that a person's reliance upon one generative capability or another will differ with practice and familiarity, a person's physical and cognitive abilities, and communicative circumstance. Often-used phrases are already 'permanently' stored on AAC systems. Standard phrases (e.g. "Just a moment please, while I pull my thoughts together") can hold people's attention while sentence construction is still ongoing. Off-line preparation and storage of text for formal presentations is also common. However, these presentations are generally 'flushed' from the AAC system memory rather than being reused at a later time. Factors limiting the use of pre-stored text for language production include AAC system memory constraints and insufficient 'processing power' and difficulties associated with reducing, organizing, finding, retrieving and modifying pre-stored text. Text storage might be automatic, initiated by the system user or a mixed combination. Stored text must be 'reduced' - to remove redundant or infrequently used text, simplify text organization, and optimize text retrieval strategies. Pre-stored text must be organized so that it can be efficiently and logically accessed. Powerful knowledge representation capabilities (e.g. organizing stored text by 'folder,' topic, context, or activity) are critical for language production based upon pre-stored text. Word prediction is a (simple) form of text storage and retrieval. Techniques must be developed to 'search for' and 'retrieve' appropriate stored text (e.g. graphical representation of text organization, Internet-style search engines). For example, 'hyperlinks' from the text being composed to pre-stored text might be employed to retrieve stored text and quickly elaborate upon a discussion or complete a literate composition. Hyperlinks to dictionary, encyclopedia or thesaurus entries might complement pre-stored text. Advanced communication processing techniques for error correction, abbreviation expansion, smart semantic editing, word prediction, etc are needed so that retrieved text can be efficiently modified for communication or merged into a literate composition. In particular, communication processing should locate errors, 'take' the user to the error, and offer error corrections. Communication processing should recognize linguistic intent and modify stored text accordingly (e.g. automatically change verb tense, singular/plural, possessive, etc). Context-based communication processing (e.g. adaptive word prediction) are complementary techniques. Icon-based language representation is used to access words and construct phrases and sentences. Communication processing capabilities to store, organize, reduce, find, retrieve and modify text may also have direct application to icon-based systems. Technology RequirementsThere is a clear need for advanced
software and/or hardware for improved storage and retrieval capabilities
of AAC devices and related computer systems. The following requirements
provide a guideline for reasonable technology solutions. It is not expected
that any particular solution will satisfy all requirements.
Text Storage, Reduction and Organization
Finding and Retrieving Stored Text
Text Modification and Error Correction
[ Top of Page ] |
|
||
|
| |
