Cost-Utility Analysis of Assistive Technology - Report on the CERTAIN project
Jan Persson
Center for Medical Technology Assessment
Linkoping, Sweden
Håkan Brodin
Department of Pharmacy
Uppsala University
Uppsala, Sweden
Oivind Lorentsen
Rehab-Nor AS
Oslo, Norway
Karl-Gerhard Hem
SINTEF,
Oslo, Norway,
Renzo Andrich
Massimo Ferrario
SIVA,
Milano, Italy
Taeke van Beekum
Wija Oortwijn
TNO, Leiden
The Netherlands
1. Summary
The TIDE CERTAIN project (Cost-Effective Rehabilitation through Appropriate Indicators) was carried out during the period 1994 - 1996. Its over-all objective was to provide a user-oriented method to evaluate costs and user benefits with regard to assistive technologies (AT). The project was restricted to generic outcome measures based on user preferences through the utility analysis method. On the cost side, a broad approch accounting for societal costs, including direct and indirect costs, was to be taken.
Achievements were made in four main groups: 1) Critical factors to ensure optimal benefits of assistive technology (AT) were identified. Outcomes as experienced by end users, family members, social networks, service providers, financing agencies, central authorities and otherswere analysed. This work constitutes the basis for cost-utility studies. 2) A taxonomy for socio-economic evaluation was developed, called "the CERTAIN tool". It provides guidelines for how to provide information on costs and social benefits with a focus on the users' satisfaction and quality of life. 3) The "CERTAIN tool" was evaluated through application retrospectively to a number of real life case studies of provision of AT to disabled persons. It was found that the critical factors analysis is applicable to case reporting and that the guidelines derived for cost analysis are applicable. 4) A feasibility study was made through the application of "the CERTAIN tool" to TIDE technology development projects. The results of the validation phase confirm that a technique for carrying out cost-utility analysis is possible, although more work is needed to elaborate it.
It was concluded that, due to identified deficits of existing outcome instruments, the taxonomy for socio-economic evaluation needs more development. As to the methodology for assessing the costs not much more development seems to be necessary. It might be useful to have a more systematic description of what elements of costing are recommended depending on the subject and goal of the study.
| Top |
2. Results
Critical factors related to the use of AT and the outcomes of AT were identified and analysed. Three workshops with participation of users, user organisations and professionals were held to achieve this objective. The outcomes of AT can be categorized in three major domains: inner relations, activities of daily life, and outer relations. Outcomes must be analysed based on expectations and goals of each individual user, and his family and social network. Outcomes should also be analysed reflecting expectations of service providers/carers and the society. However, in order to assess these outcomes it is necessary to identify a daily activity profile of the individual user, according to the priorities of the end user, and to assess the quality of the service delivery process (see Chapter 6).
The CERTAIN study has provided a taxonomy, called "the CERTAIN tool", for cost-utility analysis of AT. The tool consists of
1. a checklist for identification of resource use
2. guidelines for the quantification of resources
3. guidelines for valuation of resources
4. guidelines for choice of utility instrument
5. a checklist of available utility instruments
The tool is intended to be instructive enough to be used without a special background in economics or behavioural sciences. The tool is modular in the sense that it may result in different solutions due to type of disability, focus of study, kind of actors requesting an analysis, the technology's life cycle phase, the social environment, utilization (social) arenas and ranking of outcomes. The tool provides a means for supporting decision making for professionals as well as at policy and program level. The main aim is to support decisions on group level, taking user benefits into consideration. It is proposed that the tool be used in all phases of technologies´ life cycles, from research and development to accepted practice. The tool should contribute, by its diffusion within the TIDE program and outside it, to improve user benefits at given resources.
Out of seven utility instruments, three were proposed for further testing:
- The Index of Health-related Quality of Life (IHQL)
- EuroQol
- The McMaster Health Classification System (MMHCS)
It has been found that the available instruments have weaknesses and need further elaboration (see below andChapter 6).
The validation of the evaluation tool dealt with the process of application in technology development projects in TIDE.
The validation process was concentrated on the usability of the "CERTAIN tool" in different intended target groups. Therefore, expert-validation was chosen as the methodology for validation. It was practically impossible to determine whether the "CERTAIN tool" really resulted in the proper choice of instrument. This would have required a comparative study of the different instruments. The validation of the tool was therefore kept rather simple.
The results of the validation phase confirm that the CERTAIN tool provides a means for carrying out cost-utility analysis, although more work is needed to elaborate the methods. Some obvious problems are likely to be with the utility instruments, as they are rather crude instruments for assessing the benefits of an intervention. They are all based on health aspects (and not disability/handicaps) and may be insensitive to changes in state. However, it can be argued, since the aim is to provide information for planning and policy making, the sensitivity should not be at such a high level that statistical fluctuations in the populations studied will yield a response. There is a need for a new utility instrument, based on the findings of the CERTAIN project. Its items should be based on individual goals expectations that are commonly felt among disabled persons, and the scores may be weighted with a population that includes disabled people. It is, however, a matter of scientific debate whose values should be the basis for socio-economic evaluation. Whether general societal populations or populations of disabled persons themselves are used to score their preferences for the described states, there are sources of error which are very difficult to overcome.
The results show that there is need for a clear step-by-step guidance when performing socio-economic analysis. A more detailed instruction than the one provided by the CERTAIN study is therefore needed. It should also be stated clearly which information is needed. It is necessary to know which questions need to be addressed at different levels of decision-making related to the stage of the life-cycle of an AT. An attempt was made to classify issues at different levels of decision making and in different stages of the life-cycle of the AT.
| Top |
3. Summary and recommendations
It was found that:
- Utility analysis is the outcome measure which is genuinely based on user preferences of his or her state.
- Utility analysis provides a global measure usable for comparisons of quite dissimilar ATs and related services.
- Utility analysis in its present form may have low sensitivity to changes. It might, however, be that changes that are not detectable are unimportant.
- Existing instruments are health-related, which means that the dimensions and/or items in them, in several respects are inadequate for handicapped people and AT. The project provides suggestions for refinement of the three most appropriate, as to our opinion, instruments to meet this problem.
- There is still a further need to work out a complete taxonomy for effectiveness and utility analysis of AT and related services. Such a taxonomy should include also outcome measures which are disability-related (e.g. mobility ability, communication ability, etc). This should provide a sensitive method for assessment of ATs for groups of people with similar disabilities and handicaps. However, comparisons between dissimilar disabilities and handicaps are probably not possible with this method.
Development of a new utility instrument for the AT field is therfore strongly recommended. Its items should be based on individual goals expectations that are commonly felt among disabled persons. It is an issue for further research whether the weighing system of the scores should be developed with a population that includes disabled people or with populations reflecting other social values. For this development the body of knowledge that has been built up in the CERTAIN project will prove to be of prime importance. CERTAIN's successor, the EATS (Efficiency of Assistive Technology and Services) project (1997 - 1999), concentrates on this task.
As to the methodology for assessing the costs not much more development is necessary. It might be useful to have a more systematic description of what elements of costing are recommended depending on the subject and goal of the study.
| Top |
4. Literature
Oivind Lorentsen, Karl-Gerhard Hem: Critical factors and general outcomes of assistive technology. EU TIDE study Cost-effective rehabilitation technology through appropriate indicators (CERTAIN). Commission of the European Communities, Directorate-General DG XIII, November 1996.
Jan Persson, Håkan Brodin: Prototype tool for assistive technology cost and utility evaluation. EU TIDE study Cost-effective rehabilitation technology through appropriate indicators (CERTAIN). Commission of the European Communities, Directorate-General DG XIII, October 1995.
Renzo Andrich, Massimo Ferrario: Cost outcome analysis for assistive technology: Case studies. EU TIDE study Cost-effective rehabilitation technology through appropriate indicators (CERTAIN). Commission of the European Communities, Directorate-General DG XIII, August 1996.
Wija Oortwijn, Taeke van Beekum: Validation of the evaluation method - Report on course and agreements with other projects. EU TIDE study Cost-effective rehabilitation technology through appropriate indicators (CERTAIN). Commission of the European Communities, Directorate-General DG XIII, January 1996.
Wija Oortwijn, Taeke van Beekum: Experiences from validation. EU TIDE study Cost-effective rehabilitation technology through appropriate indicators (CERTAIN). Commission of the European Communities, Directorate-General DG XIII, October 1996.
Jan Persson, Håkan Brodin: The TIDE study HEART (Horizontal European Activities in Rehabilitation Technology). In Bengt Richt (Ed.): Livskvalitet: teori, mätning och realitet. Studies on Health and Society 21, 1994, Linköpings Universitet, Tema Hälsa och samhälle, Appendix pp. 239 -240. Linköping 1994. (In Swedish).
Jan Persson, Håkan Brodin: Livskvalitet och rehabilitering vid funktionsnedsättningar. Om mätmetodik. In Bengt Richt (Ed.): Livskvalitet: teori, mätning och realitet. Studies on Health and Society 21, 1994, Linköpings Universitet, Tema Hälsa och samhälle, pp. 167 -187. Linköping 1994. (In Swedish).
Jan Persson, Håkan Brodin: The TIDE study HEART (Horizontal European Activities in Rehabilitation Technology). In Bengt Richt (Ed.): Livskvalitet: teori, mätning och realitet. Studies on Health and Society 21, 1994, Linköpings Universitet, Tema Hälsa och samhälle, Appendix pp. 239 -240. Linköping 1994. (In Swedish).
H. Brodin, J. Persson: Cost-utility analysis of assistive technologies in the European Commission's TIDE program. International Journal of Technology Assessment in Health Care, Vol. 11, No. 2, 1995, 276 - 283.
John Hutton, Jan Persson: Introduction: Technology and disability: Assessment needs and potential. International Journal of Technology Assessment in Health Care, Vol. 11, No. 2, 1995, 135 - 143.
Håkan Brodin, Cliff Goodman and Jan Persson: Technology Assessment in Rehabilitation: Basic Concepts and Methods. Course material for Regional Workshop of European Countries in Transition "Health Care Technology Assessment in Rehabilitation Engineering". Ministry of Health of Czech Republic, IEEE/EMBS, IFMBE, Czech Society for Biomedical Engineering and Medical Informatics, WHO, and the Swedish Institute, 1995.
Jan Persson, Håkan Brodin: Effectiveness and cost-effectiveness of assistive technologies. In I. Placencia Porrero and R. Puig de la Bellacasa (Eds.), The European Context for Assistive Technology. IOS Press, Amsterdam 1995.
O. Lorentsen, K.G. Hem, J. Persson, H. Brodin, R. Andrich, M. Ferrario: Elementi di analisi costi-benefici negli ausili tecici per le persone disabili. European Commission, TIDE program, DG XIII, Fondazione Pro Juventute Don Carlo Gnocchi I.R.C.C.S. S.I.V.A., Milano 1996.
Ursula Hass, Håkan Brodin, Agneta Andersson, Jan Persson: Assistive technology assessment: A study of user participation in the selection process at rheumatoid arthritis. IEEE Transactions on Rehabilitation Engineering, 5, 1997, 263-275.
| Top | |TIDE 98 Papers |