Jump to Content
T2RERC  

home > publications > forum proceedings > communication enhancement > voice output and display technologies

Forum Proceedings

Stakeholder Forum on Communication Enhancement

Voice Output and Display Technologies: Problem Statement

 

Summary | I. Display Technology for Daylight or Poorly Lit Environments | II. Remote and Dual Display Technology
III. Speaker Accessory | IV. Voice Output | V. Automatic Volume Control

Summary

High-end AAC devices have voice, text or combinations of output methods. Auditory (voice) and visual (text) output provides feedback to the AAC user and conveys information to their communication partner. Existing output technologies currently limit communication effectiveness and ease of device use. Factors that affect current output technologies include environmental constraints such as lighting and background noise, and device constraints such as voice and display quality. The T2RERC, along with our customer, industry and research partners, seek emerging technologies that improve the output interface. The areas that require improvement and/or innovation are:

[ Top of Page ]

Display Technology for Daylight or Poorly Lit Environments

AAC display technology needs to reduce, polarize, or eliminate the effects of glare from daylight and improve contrast and brightness for night viewing. Displays that automatically "adapt" their contrast, backlighting, and brightness in response to changes in environmental lighting conditions are also desired. Displays should be user-maintained and not require intervention for set-up, adjustment or take-down. Other applications for improved display technologies include cell phones, computers, and personal digital assistants (PDA).

Description of the Problem

AAC users have identified a high priority need for displays that can be seen in any lighting condition. Glare from direct or indirect sunlight or bright lighting generally degrades the user's ability to view information on an electronic display. Although removing or minimizing the source of the reflection is the most effective method for reducing glare on displays, this is not always possible or desirable. A hood or sun-guard over the display may reduce reflection but may be physically cumbersome, unaesthetic and may not always function adequately.

Glare is problematic in several ways to both the end-user and his or her communication partner. The extent of the problem or degree of "hassle" experienced by the user and the communication partner varies depending on the characteristics of the device being used (e.g. type of display and whether the display employs a touch screen).

For the user, the effects of glare render them unable to see the message they have composed in the message window. Generally, this window is a thin panel electronic display that employs liquid crystal (LCD) or vacuum fluorescence (VFD) technology. (Liquid crystal displays are perhaps the most common display found on AAC devices - LightWriter™, Dynavox 3100, Hand Held Voice™, DigiCom, E-Talk). If the user is unable to see the message as they compose it, they may lose track of their wording or place. In effect, they must either erase and start over, or output a message that could potentially have errors.

An additional problem exists for users who employ touchscreens. Touchscreens depict language in an electronic format and change "pages" with user input. When the user selects a location on a dynamic AAC display, the device either speaks a message immediately or changes what appears on the screen. Dynamic displays operate using a system of pages, branches and levels that a user navigates through to create language or retrieve stored messages. If, due to glare, the user is unable to see their dynamic display screen and access the selection set, they will be unable to communicate.

For the communication partner, this problem is also apparent. At times (e.g. during private conversations, noisy environments) it is imperative that the communication partner read the augmented communicator's message directly from the display. When glare is present, the partner may be unable to read the display from the angle they are sitting and must further impose upon the user by leaning around or over them to read. In some cases, individuals must physically move from one location to another (out of the sun, into the shade) in order to communicate.

In addition to glare, the lack of backlighting compromises the ability to communicate in dark environments or at night. Those who use keyboards and touchscreens alike require backlighting on their keys. Some, but not all, AAC devices provide a light source for night viewing. Dynamo from Dynavox employs a backlit transflective display, while the Pathfinder has light emitting diodes (LEDs) positioned in the corner of each selection item. When a sentence is active, predicted items light up giving the user a choice of selections.

Finally, no AAC devices automatically adapt to ambient lighting conditions, whether too bright or too dark. The incorporation of adaptable backlighting would keep the user from having to manually control display contrast. This, in turn, allows the user to give more attention to the act of communication and improve overall use in most environments.

Technology Requirements

Optimized display characteristics for AAC devices or laptop computers would address important market needs and represent a clear business opportunity. A wide range of approaches may be taken to reduce glare (materials, coatings, adaptive, etc.) and creative, effective, low-cost solutions are encouraged. The following "requirements" provide guidelines for technology solutions - though it is not expected that all requirements will be satisfied in any single solution.

  • Must reduce glare and ideally would eliminate glare.
  • Must provide backlighting for use in dark environments or at night.
  • Must be unobtrusive to the person using the device (i.e. not interfere with the user's line-of-sight or display access).
  • Must be unobtrusive to the communication partner (i.e. not interfere with the communication partner's line-of-sight).
  • Must be compatible with display access techniques (e.g. doesn't reduce physical access to the display.)
  • Must not require assistance to use or set-up (with the possible exception of initial set-up).
  • Must be durable (e.g. to impacts, vibrations, etc).
  • Must perform well in all environments (e.g. temperature, moisture, dust).
  • Must be compatible with displays used on AAC devices (e.g. accommodate a wide range of display sizes and shapes, doesn't filter or modify colors, diminish contrast or brightness, or reduce/increase display size).
  • Should have the ability to adapt brightness and contrast automatically.
  • Should not significantly increase weight.
  • Should have user-controlled customization of brightness and contrast.
  • Should be nearly maintenance free-not require cleaning, unbreakable jointing, puncture resistant.
  • Should not significantly increase power consumption (for approaches drawing power).
  • Should have automatic power saver mode to reduce power consumption (for approaches drawing power).

[ Top of Page ]

Remote and Dual Display Technology

AAC devices and other related computer systems are needed that provide one or more of the following types of displays: dual, wireless wearable, and a wireless remote. A dual-display screen is needed that will provide text feedback of a constructed message directly to the person using the device, as well as to their communication partner (e.g. one screen with double displays). The dual screen should be able to automatically adjust to the position and/or orientation of the communication partner (i.e. able to orient direction of sound perception) while not obstructing the face-to-face interaction or alter their eye-to-screen viewing distance.

A need also exists for wireless displays, both wearable and remote. A wireless wearable display should be as unobtrusive to the user as possible, perhaps even virtual. Wireless remote displays are those which have the capability to display on both the AAC monitor as well as simultaneously on a second remote display (separate monitor, digital flat panel, etc.). The appearance of all displays (i.e. size, shape, color, layout) should accommodate user capabilities and preferences.

Description of the Problem

The visual display plays an important role in AAC-based communication. The display enables the speaker to see their message as they generate it and may allow their communication partner to read their utterance as they compose or speak it. The display also provides the means by which a user selects items, constructs expressions, and makes corrections to text before it is spoken. Dynamic displays change in response to user input while static displays consist of a fixed selection set.

A liquid crystal display (LCD) is the viewing screen found on many communication devices (LightWriter™, Dynavox 3100, Hand Held Voice™, DigiCom, E-Talk). To facilitate more natural conversation and ensure user privacy, it should not be necessary for the communication partner to read display information over the shoulder of the person using the AAC device. It should be possible, however, if the user wants to take part in a private conversation with one person. Some communication devices such as the LightWriter™ (Zygo Industries), have a dual display that presents the communication partner with text to follow along with during the conversation. This display can only be seen when the interactant is directly across from the person using the device and is enabled at all times.

A wireless dual display may be appropriate for a teacher (at the front of the room) to communicate with his or her students (at their desk). This accommodation could apply to a range of students, from kindergarten through college level, and is especially convenient for a teacher who wishes to view what the child has composed, or answer a question privately. In addition, a patient could communicate (from a hospital room) to their doctor or nurse (at a nursing station). In separate rooms, a second (remote) speaker will be required.

A need exists for virtual displays that can be worn by the user when a "standard" display may be inconvenient or intrusive (e.g. in bed, in a vehicle). Virtual displays could be a separate (flexible and low weight) wearable device or integrated into an everyday item such as eyeglasses, glove, watch, etc. Virtual displays may be accessed wirelessly via many of the same access methods as a regular AAC display such as keyboards, switches, etc.

Current problems occur in the inability of the user to easily alter their display screen (e.g. brightness, color, and contrast). In addition, some systems don't allow the user to turn off their display (or only one of their displays, in a dual system). The user should have full control over on/off capabilities both for privacy and power consumption. AAC users encompass a heterogeneous group of abilities (e.g. CP, ALS, MS). The display should adapt to the individual's visual, seating and positioning, motor, and cognitive abilities.

Technology Requirements

Optimized display capabilities would address important market needs and represent a clear business opportunity. A wide range of approaches may be taken to provide the user with enhanced communication opportunities and creative, effective, low-cost solutions are encouraged. The following "requirements" provide guidelines for technology solutions - though it is not expected that all requirements will be satisfied in any single solution.

Dual Display (Second display on AAC device)
  • Must facilitate dual mode communication by making it easier for interlocutor to view text.
  • Must not be obtrusive to the person using the device or to their communication partner.
  • Must not require communication partner to reposition themselves to read display.
  • Must not create a physical barrier between two communication partners.
  • Should be age appropriate (not be too text-based for children, incorporate pictures and text). Remote Display (Second display "across the room")
  • Must enable the speaker to communicate privately.
  • Should have a reliable wireless communication link.
  • Should have a remote speaker in conjunction with remote display.
Wearable Display
  • Must have rechargeable battery.
  • Must be comfortable and attractive.
Requirements for All
  • Should be durable (water, wind, shock resistant) with low sensitivity to vibration.
  • Should be watertight (e.g. sealed electronic components).
  • Should be unobtrusive (e.g. must not interfere with AAC device access).
  • Must not significantly increase power consumption. Should be nearly maintenance free-not require cleaning, unbreakable jointing, puncture resistant, etc. Must not require intervention to function or set up. Should not require a significant amount of instruction to set up. Operation capabilities, controls, interface capabilities should be similar to standard AAC device screens. Must be easily controlled (i.e. enable/disable, brightness, color, contrast.)by the user. Should have capability to customize display (i.e. change font size, color, and style). Should provide display that can be seen by the user and accessed by the user despite posture, orientation and motor capabilities. Should be easy to transport.

[ Top of Page ]

Speaker Accessory

Many individuals using AAC devices with voice output experience difficulty communicating in one or more environments (i.e. cars, parties, movie theaters). The ability to control both loudness and speaker direction would facilitate conversation in difficult communication environments and conversation with communication partners who are not directly in front of the AAC device. Speakers that can automatically track (orient toward the communication partner) may be beneficial. The ability to direct sound toward the communication partner will increase intelligibility and also increase privacy (directed sound can have lower volume). In addition, technology is needed that provides the user with a wireless remote speaker. The speaker could function as a wearable remote speaker, or desktop speaker located apart from the AAC user.

Description of the Problem

Privacy, difficulty communicating in loud environments and speaker clarity (i.e. no echo or distortion) are issues for speaker technologies that have not been addressed by current AAC systems. In order to address these problems, speaker technology needs to be developed that orients to the communication partner to provide a directional focus. Speaker systems providing directional capabilities benefit the AAC user in that they increase private communications, are less distracting to others not participating in the communication, and increase access to AAC devices in difficult environments (i.e. loud bars and sporting events). In addition, directional speaker systems can improve clarity by decreasing the need for high volume levels (i.e. when speaker is pointed directly at communication partner, it requires less volume output in order to be heard), thereby reducing the effect of echo and distortion. Low volume (i.e. a whisper) is required for privately speaking in quiet environments.

AAC devices with internal speakers draw the communication partner's attention to the device not the user. This, in effect, is dehumanizing, while at the same time it eliminates eye contact and nonverbal information that is shared through maintaining eye contact (i.e. turn taking cues, nodding affirmation). A need exists for displays that are wireless, remote and wearable. The speaker system should provide a wearable array so the "sound" appears to originate from the person's mouth. The system should not require tethering to the device and should be cosmetically appealing. This wireless wearable remote display is beneficial to AAC users to increase eye contact, thereby improving quality of communication (i.e. non-verbal). It may also improve the overall naturalness of social interactions.

Individual's using AAC device speaker systems have difficulty in work and educational settings. Current speaker systems limit privacy and can be very distracting to others in the area. A need exists for a remote speaker (possibly with a remote display), that is portable (not tethered to the AAC device), and has it's own power supply. A desktop speaker system for AAC devices that is remote and wireless can benefit AAC users by improving privacy, increasing inclusion, and decreasing distraction. In addition, there would be an improvement in communications at work and at home. Safety concerns would also be addressed with remote wireless desktop speaker systems by providing the user with a speaker that could be located in another room (i.e. AAC device in patients room, speaker in nurse's station).

Technology Requirements

Optimized speaker accessories for AAC would address important market needs and represent clear business opportunities. A wide range of improvements is required and creative, effective, low-cost solutions are encouraged. The following "requirements" provide guidelines for technology solutions - though it is not expected that all requirements will be satisfied in any single solution.

Rotational
  • Should be built-in.
  • Should be adaptive (e.g. track moving speaker, pick out and orient toward new speaker)
Remote
  • Must have wireless link.
  • Must eliminate tethering.
  • Should be able to be placed on multiple sites (on person, in room, on AAC device) with comparable quality output.
Wearable
  • Must have wireless link for wearable speaker or speaker array.
  • Should be lightweight.
  • Should be able to be placed on multiple sites (on person, in room, on AAC device) with comparable quality output.
  • Should be comfortable and attractive.
All Speakers
  • Must be unobtrusive and aesthetically acceptable.
  • Must not be distorted at any volume (no echoes).
  • Must not affect output pattern.
  • Must have good frequency response (speaking voice).
  • Must be powerful enough to be heard in a noisy room.
  • Must be compatible with both synthesized and digitized speech.
  • Should produce a high quality, undistorted sound from anywhere (as perceived from the AAC user and their communication partners).
  • Should be easy to connect and use.
  • Should have ability to be controlled through access method (input device).
  • Should require minimal set-up.
  • Should be compatible with other devices (i.e. cell phones, speaker phones).

[ Top of Page ]

Voice Output

AAC systems should provide "natural-sounding" voice output including the ability to express emotion. Voice output should be easily customizable by the person using the device. The user should be able to pre-define voice characteristics and also have the capability to change the personality characteristics manually in "real-time" (quickly customize voice characteristics as expression is being composed). Language processing should control voice synthesis. Language processing should recognize the context, structure and intent of the sentence to shape voice production.

Description of the Problem

DECTalk™ is currently the standard software used for speech output on most AAC devices. DECTalk™ is limited to nine pre-defined personalities (i.e. Perfect Paul). DECTalk™ voices are monotone and at times unintelligible unless voice parameters are modified through extensive programming. In group conversation, it is often hard to distinguish between AAC speakers since many users share the same voice. Few voices and limited ability to customize these voices prevents having their "own" voice or a voice that is a clear expression of their personality. Some users may want to switch personalities (for example, some users may switch to a different personality to be better understood on the telephone). This should be done easily and in real-time. Due to inadequate female voices, some female users even resort to using a male voice. Only few voice parameters (e.g. pitch, inflection) can be easily modified by the user. Some users (with acquired disabilities) may want to use their own voice, prior to their injury or disability. They should be given the option to store their voice for later use, if expected to decline.

Currently, voice output can be customized using macros (e.g. intonation, emphasis, tone and prosody.) Some systems allow the user to add words and customize the word pronunciation (e.g. stress, emphasis, emotion). However, adding or customizing words necessitates that the user have "programming skills", off-line time to do this programming or multiple keystrokes during the conversation. This customization process is not user-friendly in that it is time consuming and complex.

AAC devices need language processing to control speech output, by inserting pauses (e.g. in response to punctuation, separate phrases) and by controlling prosody (stress, emphasis). Language processing should recognize sentence structure to infer intent and to shape intonation (e.g. sentences ending in question or exclamation marks). Language processing is needed to control voice production so as to provide the proper intonation based upon discourse and pragmatics. Information is lost (or misleading) without context appropriate pronunciation (e.g. stress, volume, emphasis). The interface between language processing and the speech synthesizer needs to be refined.

People who use AAC must insert pauses to facilitate understanding of phrases and/or words, especially when giving speeches. Pausing is very important for emphasis and to slow down the speech for better intelligibility. However, several keystrokes are required in order to insert pauses (currently device adds pauses only at sentence ends).

Technology Requirements

Improvement to voice output for AAC devices is a clear need and would represent a significant business opportunity. The following "requirements" provide guidelines for technology solutions - though it is not expected that all requirements will be satisfied in any single solution.

  • Must sound natural and human, not computer-generated or synthesized (must match human range).
  • Must have ability to change gender.
  • Must have equal clarity and quality for both female and male voices.
  • Must provide regional, international, cultural identification (accents).
  • Must have ability to sing.
  • Should utilize individual's voice prior to loss. For acquired, progressive diseases, people should have the option to store their voice for later use.
  • Should provide user the ability to quickly and easily change the voice personality for different contexts (i.e. for telephone use or when not understood).
  • Must have ability to customize voice parameters (e.g. intonation and inflection).
  • Must have ability to express feelings or emotions (i.e. sarcasm, anger, elation).
  • Must have intuitive, easy to use control for instantaneous changes in amplification, speed, inflection, volume and emphasis.
  • Must be easily customizable by user.
  • Must provide ability to incorporate slang vocabulary.
  • Must provide user ability to easily add words (including slang) and customize pronunciation.
  • Language processing should recognize word context to identify pronunciation (e.g. "I was happy to read (reed) the story" versus "I read (red) the story").
  • Language processing should control voice production.
  • Language processing should recognize sentence structure, content and intent to shape voice production.

Automatic Volume Control

Automatic ("self-adapting") voice output volume control is needed for computers and AAC devices to reduce the effects of background noise and maintain an appropriate volume as the user changes settings. Automatic volume control will allow communication partners to hear voice output at appropriate volumes in loud or quiet environments. A built-in microphone would allow the AAC device to recognize loudness levels in the environment and adjust the voice output volume accordingly.

Description of the Problem

Many individuals using AAC devices experience difficulty communicating in noisy environments (i.e. in bars, sporting events, or cars). Many of these individuals facilitate communication by switching from speech output to text output. This strategy requires that the communication partner(s) have a clear view of the display (or dual display). This strategy will not work well in large group discussions (some individuals cannot view the display) or poorly lit environments (glare or dark environments reduces the ability to view the display). In many cases voice output is the only viable alternative, however, the act of manually adjusting volume on an AAC device can be frustrating and can inhibit communication (e.g. time consuming, breaks flow of discussion, people forget topic). In addition, AAC system feedback (outputs "volume up, volume up") may distract people from their conversations.

AAC devices should be capable of detecting the level of environmental noise and providing precise self-adjusting volume control appropriately in real-time. This compatibility is desired for AAC devices utilizing synthesized, digitized, or a combination of these methods of speech output. An override option should allow the user to turn the automatic volume control off if desired, in order to facilitate manual volume control.

Individuals using AAC devices have identified the need for AAC systems to automatically adjust the voice output volume to a desired level. Automatic volume control will facilitate communication in diverse environments and provide sound volume level appropriate to this environment. Automatic volume control would help maintain privacy by maintaining sound volume at a level just necessary to communicate in any environment.

Technology Requirements

An automatic volume control system for AAC devices would address important market needs and represent a clear business opportunity. The following "requirements" provide guidelines for a technology solution - though it is not expected that all requirements will be satisfied in any single solution.

  • Must have override capabilities - option to control volume both manually or automatically.
  • Must take place in real-time (fast response, no processing delay).
  • Must adjust sensitivity in response to the environment. Must have lower volume limit suitable for conversation in very quiet environments. Must have upper volume limit suitable for conversation in very loud environments - without damaging device speaker.
  • Should work in difficult listening environments (e.g. many sound sources, noise from all directions). Should sense noise from all sources and directions and adaptively change performance in response.
  • Should disregard intermittent bursts of noise (i.e. train whistle, car horn, etc.).
  • Should not significantly increase size or weight of device.
  • Should use unobtrusive microphone (or equivalent) for tracking environmental noise.
  • Should, with necessary processing hardware) be small and inexpensive.

[ Top of Page ]