To determine whether or not a product transferred by the T 2RERC is an improvement over its competing alternatives – i.e., other products present in the marketplace at the time of the product's transfer (tested in laboratory trials) or alternative strategies used by consumers for accomplishing the designated function (focus of home trials). Operationally, it is to verify and document the product's overall quality, its value and consequently, its importance to users with disabilities, as manifested by the consumer's subsequent acceptance and use (or disuse) of the product.
[ Top of Page ]
Consumers with disabilities (that are potential users of our transferred products); Product Development managers, Product Designers, Researchers/ Evaluators (including NIDRR project PI's)
[ Top of Page ]
Data Gathering and Analysis
The study design includes three broad phases. Phase I will develop the indicators and instruments/protocols that will guide data collection. Phases II and III will include hands on trials by consumers in the "laboratory" (contrived) and home (natural controls) environments respectively. A sample of consumers appropriate for the device under study in number and characteristics will each individually assess both the T 2RERC.product and its competitor product in Phase II. They will then test it against their current alternatives at home in Phase III.
Phase I - Developing indicators and instruments
Quality and value are the major variables that drive the evaluative conclusions about the product. We will draw indicators of these variables from both designer and consumer perspectives to build our instruments. Designers contribute to defining the technical requirements that build quality, whereas consumers are expert guides for the product's usefulness in practice. This approach will combine the designer's insight about helping people achieve the operational objectives for which they are responsible, with the consumer's desire to use A/T to fulfill life's roles.
In developing the final set of indicators, we will also draw from human factors literature, as well as previous work on product evaluation and universal design, obtaining any additional empirical data as needed.
Molly F. Story from the RERC on Assistive Medical Instrumentation, Dr. Arun Jain, Professor in the UB School of Management, and James Lenker, Director of UB's advanced graduate A/T Certificate Program will collaborate as experts during this phase, giving input to instrument development as well as the design of the next two phases as needed. Additional experts will be considered as called for by the device in testing.
Phase II - User trials in laboratory setting
Hands-on trials by users in a laboratory setting will provide information with reference to the indicators defined in Phase I. The set up will be for a repeated measures design, in which each consumer will sequentially test, and record observations, on both products (the T 2RERC's and the commercial) selected for testing. The individual testing sequence will be randomized across persons. Measures will include responses of the testing consumer to specific questions on a questionnaire and/or interview based on the indicators. A team of trained observers will also record their findings as the users try out the products. Video recordings will be made for later reference. The trial sessions will be conducted on the WNY Independent Living Project premises.
Phase III - Follow-up home trials
The participating consumers will be asked to take the T 2RERC product home, use it, test it, and give us feedback. The home trial will be broken up into two periods, at the end of which they will be financially compensated.
During the initial period of about two months, consumers will use the product and keep a log of its use as directed. Specific questions will be given to guide their observation and recording. We will also note and record what current device or alternative they use at home, against which they will be judging our product. During the final period of about three months, they will be told to continue to use the product but with no specific request for feedback.
Two sets of follow up calls will be made to the users, one at the end of each of the two home trial periods mentioned. The purpose of the calls is to get user feedback on specific criteria based on use indicators (frequency of use, how useful, role fulfillment, etc). We will also examine their log records and check their qualitative comments (about satisfying or frustrating experiences, the ease or difficulty of use, etc) against relevant indicators. Additional follow-up data on their purchase intent will be obtained as appropriate, by posing them a purchase opportunity.
Data analysis will include both statistical and descriptive methods as called for by the quantitative/qualitative nature of the data - such as Analysis of Variance (ANOVA) to infer from the repeated measures data of the onsite trials, and content analyses of qualitative narratives from home trials. Overall, however the purpose is to obtain an analytical description of product comparisons in terms of quality and value. Although data from both trials are anticipated to address both quality and value, the Phase II data comparisons will mostly generate quality related results whereas data from home trials will reflect product use and its value. Results on evaluation of product value will include cost considerations, based on user purchase data and/or purchase intent data, as well as include user decision to use/abandon the product. To the extent feasible and appropriate, we will consider the qualitative weight and sum method for final data integration (Scriven, 1991) and additional judgment standards (e.g., safety, side effects, and legal/moral issues).
[ Top of Page ]