What You Don't Know Can Hurt You: Measuring Outcomes in Assistive Technology

by Sarah Blackstone and Harvey Pressman
(S. Blackstone, ACI 1 Surf Way, #237, Monterey, CA 93940)
(H. Pressman, BUSD, 1835 Allston Way, Berkeley, CA 94706)

Web Posted on: December 12, 1997

Assistive technology (AT) practitioners need to know more about issues related to the measurement and management of outcomes in their field. One especially important issue is to understand the perspectives of the various stakeholder groups involved in the delivery of assistive technology services: consumers, family members, providers, payers, vendors, etc. To bring practitioners up to speed regarding state-of-the-practice approaches to outcomes measurement in the area of assistive technology, the authors organized two working symposia on Outcomes in Augmentative Communication and Assistive Technology.

More than one-hundred fifty stakeholders, who included consumers, providers, payers, policy makers, researchers, educators and family members were convened in Monterey, California in 1995 and 1996. Specific recommendations were developed at both conferences regarding measurement of functional status.

The group identified issues and current practices in the outcomes measurement area and built a beginning consensus regarding the direction of future AT outcomes management programs. Areas of measurement were prioritized and action plans developed. Consensus was reached on a consumer-based mission statement for outcomes-based measurement and on a recommended plan of action.Perhaps of equal importance, a new set of questions emerged from these deliberations.

Some of these questions were especially provocative. Examples include:

• Have a new set of "drivers" taken over the system, and are they steering in a different direction?
• Which stakeholders will define the "new vision" for Assistive Technology and AAC?
• Does good care actually cost less than poor care?
• Are the scales we are developing so crude that they fail to measure important signs of progress?
• While we are trying to be so consumer responsive, are we in danger of describing outcomes in terms that are irrelevant to the people who make the decisions about paying for the products and service?
• Is there a danger that we will get so specific in defining our outcomes that we will get the people who control the funding stream to rule us out by virtue of the own narrowness of our definitions?
• Does what gets measured affect what gets done?

Some of these questions plainly imply a need for new ways of looking at, and thinking about, outcomes issues in AAC and assistive technology. We think that a closer look at a few examples may suggest new ways in which we may want to frame our future thinking about these issues, and may even help move us towards the development of a new hierarchy of questions about outcomes in the areas of augmentative communication and assistive technology.

Have new drivers taken over the system and are they steering in a different direction?

This question is important because it has implications of a profound power shift. In the past, providers have often maintained practical control over service delivery practices. Policy makers, families and consumers bowed to the decrees of providers and researchers. The person with a problem and his/her family largely abdicated responsibility because they placed enormous trust in the experts.

Clinicians may not have been fully aware of the power they held until it started to recede. Those in hospital settings felt it first, but all providers and researchers now are aware of the rising tide of change. Many are off balance and scrambling to accommodate the changes. Both within AAC and other areas involved with assistive technology, there exists an increasingly vocal consumer empowerment movement for person with disabilities, in many different countries. This shift in power will, in all likelihood, result in many other stakeholder groups being challenged to rethink what they do and how they do it.

When people feel threatened they may perceive only barriers, but what is always present at the same time are opportunities for change. Because most professionals involved in delivering AAC and assistive technology services have always worked with multiple groups of stakeholders, adjusting to a more level playing field with more accountability to consumers, families and payers may not be so difficult.

Does good care actually cost less than poor care?

By raising such a question, Frank DeRuyter reminded participants that payers are 100 percent committed to lowering costs. Implied is that they are less concerned about lowering the quality of service provided. Many pointed out that it is possible that there are most cost efficient ways to produce desired outcomes. If we start out with greater clarity about what the outcomes are, for example, we are far more likely to be effective in achieving them more quickly.

Could treatment that is less costly actually be more effective? Suppose, for example, that someone whose needs could easily be addressed in their community is instead referred to a major AAC Center of Excellence. Costs increase, while the success of the intervention may actually lessen. Conversely, for someone who has complicated seating, positioning and equipment needs, referral to an Assistive Technology Center may actually less the ultimate costs, as intervention may be more efficient.

Which stakeholders will define the "new vision" for AAC and assistive technology?

This question caused considerable confusion and concern as people raised it in one form or another. The consensus of the participants was that it is desirable to seize the moment and together define AAC and other areas involved with assistive technology in terms that are measurable and that others can understand. For AAC providers, consumers, family members, manufacturers and researchers, the vision has been expressed quite well in the mission statement of ISAAC and its chapters, but a newer version of this vision might well require additional accountability words, such as "in ways that are cost efficient and demonstrate the effectiveness of goods an services and make a difference in the quality of life for persons with severe communication impairments. This may well be true in other assistive technology areas.

Does what gets measured influence what gets done?

Historically, providers, administrators, and researchers are the stakeholder groups most likely to measure effects of AAC and assistive technology goods and services. Researchers strive to define questions and use tools that give them answers to specific questions raised. Providers use assessment and follow-along data to document each contact and service provided. (Some information they use, while other date they collect for administrators). Manufacturers also collect information about service requests and device repairs, but rarely share in among colleagues and researchers.

At first sniff, it is tempting to assume that what gets measured gets done. It stands to reason that why you focus on is more likely to get done than something else. However, it is also possible that what gets measured today in AAC and other areas of assitive technology may not be relevant to what gets done. One example of this is reflected in participants' response to a preconference questionnaire. Few providers who responded were measuring outcomes functional outcomes systematically, those who were neither aggregating data or using the data to guide future decisions.

Questions such as these no doubt require further deliberation. The deliberations of these two outcomes conferences suggest clearly that we fail to discuss them at our own peril. We all need to know more about such things as measuring functional status, goal attainment scaling, portfolio assessment, or the choices we now make will gradually fall into the hands of others.

READINGS

Quality Improvement Digest. Multiple publications available from the American Speech-Language-Hearing Association: Rockville, MD.

Blackstone, S.W. (1995). Outcomes in AAC. Augmentative Communication News. 8:1.

Frattali, C.M. (1993). Perspectives on functional assessment: Its use for policy making. Disability and Rehabilitation, 15:1-9.

Pressman, H. & Blackstone, S. (Winter, 1995). Come out, Outcomes, wherever you are? Searching for a new paradigm. Communication Matters.

Pressman, H. & Blackstone, S. (February, 1995). Alliance 95. Outcomes in AAC. Asilomar Conference Center, Monterey, CA 93940.

Alliance '96 Portfolio. Outcomes measurement in Assistive Technology. Compiled by H. Pressman & S. Blackstone. Asilomar Conference Center, Monterey, CA 93940, February, 1996.