CONSUMER CRITERIA FOR ASSISTIVE DEVICES: OPERATIONALIZING GENERIC CRITERIA FOR SPECIFIC ABLEDATA CATEGORIES

Joseph P. Lane, Douglas J. Usiak and John A. Moffatt RERC on Technology Evaluation and Transfer Center for Assistive Technology, University at Buffalo Buffalo, NY, U.S.A.

ABSTRACT

Prior research generated a set of generic consumer evaluation criteria. Device dissatisfaction and abandonment remains high, so the generic criteria are either not applied or require further refinement. The RERC-TET developed more detailed consumer criteria for a dozen ABLEDATA categories where consumer dissatisfaction is reported to be quite high. Results are included in a publication available from the authors.

BACKGROUND

Consumers lament the lack of user input to the design and development of many assistive devices. Their argument is that increased consumer input will improve the quality of such devices. In response, studies have elicited consumer opinions about assistive devices [1,2,3]. A landmark study established a set of seventeen generic criteria, ranked in order of the perceived importance to consumers [4]. In that article, about a dozen people with various impairments rank-ordered the categories, and the article reported the sum of mean ranks across all participants.

The RERC-TET, funded by NIDRR, is working to improve the quality of assistive devices available in the marketplace. Improving devices includes new devices to meet previously unmet needs, or modified devices to better meet consumer needs poorly met by existing devices. The RERC-TET is working from the supply side (technology push), by evaluating prototype inventions and helping to commercialize those inventions offering value to the consumer. The RERC-TET is also working from the demand side (technology pull), by involving the customers (consumers, family members and professional care providers) in the evaluation process.

The results from about fifty-five focus groups on ten devices -- about seven hundred participants -- indicated that the Batavia and Hammer consumer criteria [4] need further refinement. When a focus group applies those seventeen criteria to a specific device, their definitions and relative rankings change. Our experience indicates that the definitions and rankings will vary across device categories, if not for every device. Further, most consumers preferred to collapse several categories under one heading. We developed a subset of the original seventeen criteria, by combining criteria that consumers identified as redundant. Our working subset contains eleven criteria (one quantitative -- affordability, and ten qualitative). The revised set of eleven criteria collapsed the original seventeen by Batavia and Hammer as follows:

The RERC-TET took advantage of the opportunity to refine these criteria. To provide consumer evaluations in a timely manner, the RERC-TET had already established a network of fourteen consumer agencies around the nation. Every site has demonstrated its capacity to conduct consumer evaluations across disabilities and across age ranges. The site's distribution reflects geographic, demographic and ethnic diversity. To assist researchers, manufacturers and consumers, the RERC-TET had these sites hold focus groups, to establish more specific consumer criteria for selected device categories.

METHODS

How were the selected categories chosen? The RERC-TET's consumer team surveyed the fifty-five State Tech Act programs (n = 32; 58% response), 320 Independent Living Centers (n = 41; 13% response) and consumer participants from both the National Independent Living Council Annual Conference, and the New York State Coalition on Independent Living's Annual Meeting (n = 67). In all cases, the survey team worked to identify the organization's point of contact for assistive technology issues.

The survey instrument listed the seventeen general ABLEDATA categories. It asked respondents to select the five ABLEDATA categories most in need of additional product development, or most in need of improvements to existing products. It then asked the respondent to provide specific examples of the devices requiring substantial improvement within that ABLEDATA category. Response totals indicated the rank order of device categories, with those categories selected more frequently representing more need of development or improvement. The following list shows the ABLEDATA categories identified most frequently (actual category in boldface and preceding branches in italics). These are the device categories most in need of new product development or existing product improvement.

ABLEDATA Product Categories in Greatest Need of New or Improved Products

• Architectural / Indoor / Bathrooms
• Wheeled Mobility / Wheelchair Accessories
• Architectural / Indoor / Doors
• Architectural / Indoor / Doors /Door Operator/Windows / Window Opening Aid
• Architectural / Vertical Lift / Ramps
• Communications / Signal Systems / Special Dialing Telephones / Voice-activated Telephone
• Transportation / Vehicle Accessories/ Van Accessories/ Van Lifts and Ramps
• Computers / Hardware / Input / Voice Input Interfaces
• Transportation / Vehicle Accessories / Van Accessories General / Wheelchair Restraint System
• Seating / Cushions
• Computers / Hardware / Input / Voice Output
• Sensory Disability / Blind and Low Vision / Reading/Reading General / Voice Output Reading Machine

• Vocational Management / Work Stations/ Specialized Work Stations

• Wheeled Mobility / Wheels / Quick Release Wheel Axle

FOCUS GROUP PROCESS

The RERC-TET's Consumer team then organized and conducted industry standard consumer focus groups, for the top device categories identified through the survey. These focus groups were held in the fourteen evaluation sites across the country. Each device received four focus groups held in various sites. Each focus group had 8 to 15 potential customers of the assistive device (average of 13 per group), with an average number of 52 participants per device (Industry norm n = 50). The focus group process roughly mirrors the RERC-TET's internal evaluation process for evaluating prototype assistive devices. The focus groups participated in the following five steps:

Step 1 - Initial Consumer Criteria. We introduce consumers to the concept of evaluation criteria by asking them, "When you are about to purchase an assistive device, what criteria do you consider?" We record all responses for each group, then collapse responses across groups. We keep the full list of discreet answers. After each we note which of the four groups gave the response and thereby have a count across all groups. For example, "ease of learning to operate (1,3)" means this criteria surfaced in two groups, group one and group three.

Step 2 - Consumer Criteria for Ideal Device. We introduce the specific device category of interest, then we move through the eleven criteria. We present the eleven criteria in different orders for each group. We say, "We are going to discuss [device category] devices. With the ideal product in mind, how would you judge [each of eleven criteria]? For example, for the criterion "reliability," we ask, "How would you judge whether a device provides repeatable, predictable performance, and levels of accuracy under reasonable use?" As in Step 1, we record all responses in each group, then collapse the responses across groups. We repeat the process for all ten consumer criteria.

Step 3 - Ranking Consumer Criteria. Having established the consumer's familiarity with all eleven categories, we ask each group to rank order the eleven categories using a modified Delphi process. In this process, each category is compared to all other categories one at a time, until the relative importance is established between all eleven categories. The rank orders are averaged across the four focus groups to derive a final rank order.

Step 4. Consumer Satisfaction with Existing Devices. We next determine how existing products known to our participants rate across the eleven criteria. We say, "Earlier we identified some examples of [assistive device] on the market. Keeping those in mind, how well do those examples satisfy the consumer criteria?" The answers are essentially a report card on the current array of devices in the marketplace. Again, all discreet comments are recorded with multiple comments noted by number and group.

Step 5. Suggestions for Improving Existing Products. The final step is an open-ended opportunity to suggest any and all improvements needed by the device category under review. We say, "Earlier you identified some examples of ideal [device]. At this time, let's list all the ideas and suggestions that come to mind for making [device] more accessible. Regardless of how impractical they may seem or how expensive they may be." These responses are arranged according to frequency across groups.

RESULTS

The RERC-TET is compiling the results of this work in a publication available from the authors.

ACKNOWLEDGMENT

This work is supported by a grant from the National Institute on Disability and Rehabilitation Research, U.S. Department of Education.

REFERENCES

[1] Brienza, D, Angelo, J & Henry, K. (1995). "Consumer Participation in Identifying Research and Development Priorities for Power Wheelchair Input Devices and Controllers." Assistive Technology, (7), 55-62.

[2] Sandberg, K., Benktzon, M. & Landen-Lorentzen, L. (1993). "Tools for Living - User involvement in the design process." Proceedings of ECART 2. Stockholm, Sweden. 30

.3. [3] Ward, C. (1992). "Listening to the Experts: Consumer Needs Assessment project year four." Project Report for the Rehabilitation Engineering Center on Technology Transfer, Washington, DC: Electronic Industries Foundation.

[4] Batavia, AI & Hammer, GS, (1990). "Toward the development of consumer-based criteria for the evaluation of assistive devices." Journal of Rehabilitation Research and Development, 27 (4), 425-436.

Joseph P. Lane RERC on Technology Evaluation and Transfer Center for Assistive Technology University at Buffalo 515 Kimball Tower Buffalo, NY 14214-3079