A Report on a Research Study to Develop Guidelines for CBR Evaluation
Dr. Tizun Zhao and Dr. Joseph Kwok+
Abstract
As CBR Practice takes on a more significant role from grass roots to national and regional levels, the need of a set of comprehensive guidelines for CBR evaluation has also become stronger. A research team was therefore sponsored by RI and RNN in 1993 to deal with this challenge. A summary of the final draft of the research report is presented, which covers all major aspects concerning CBR practice and evaluation. The study will continue to be field tested, and updating is expected on a regular basis.
Introduction
People with disabilities are estimated to form seven to ten percent of the population in any country, and around 2 percent would need some forms of rehabilitation services. Yet only 0.1 percent to 0.2 percent in developing countries actually get such services (WHO, 1981). In trying to close the rehabilitation gap, community based rehabilitation (CBR) is considered one of the most practical and efficient rehabilitation approaches.
In 1976, Helander, et al. (1980) developed CBR programmes which were designed to integrate with programmes for primary health care. Since then CBR has been implemented in countries with vast differences in race, culture, language, socio-economic-political development and religious belief. A collection of diversified and rich experiences has been documented. Along with the expansion of CBR field programmes, a range of monitoring and evaluation approaches and methods have been explored. Policy makers and field staff at various levels have revealed a need to have a set of comprehensive guidelines for evaluation of CBR which should be scientific, practical, easy to use, and cover various aspects and stages of CBR practice (Helander, 1993).
In December 1993 a research team was formed to develop "A comprehensive guidelines on the evaluation of CBR", which consisted of experienced members from Malaysia, Korea, Singapore, Japan, Indonesia, China and HongKong1. In order to invite more suggestion and feedback, the research plan was sent to key people who have been involved in CBR or rehabilitation services from inside or outside the region. There have also been intensive efforts to collections and relevant reviews, including over 30 major and updated statements, articels and research papers.
+ Dr. Tizun Zhao is the Director of CBR Department, China Rehabilitation Research Center. Dr. Joseph Kwok, Associate Professor, Department of Applied Social Studies, City University of Hong Kong.
1 In 1993, during "the Seoul Conference on Rehabilitation Manpower Development and Networking in the Asian and Pacific Decade of Disabled Persons 1993-2002", the RANAP Executive Committee submitted a proposal entitled "The Research Project-Guidelines for the Evaluation of CBR". It was later funded by the RI Regional Secretariat for Asia and the Pacific (Hong Kong Office), and the Regional NGO Network for the Asian and Pacific Decade of Disabled Person 1993-2002.
A brief summary of the key contents of "The Guidelines for Evaluation of CBR" is presented below.
Part One: A General Framework for CBR Evaluation
A. The "What" and "Why" of CBR Evaluation
A working definition of evaluation of CBR refers to a standard to make objective judgements on the activities and outcomes of CBR programmes and the rehabilitaion efficacy of people with disabilities in line with the goals, strategies, action plans, implementations of CBR programmes and the rehabilitation training schemes of the consumer. Relevance, effectiveness, efficiency, sustainability and impact are the core criteria that should be considered in the CBR evaluation (UNDP, 1993). The evaluation is a process of learning experience, and revising and improving ongoing programmes (Marie-Therese, 1986). It is a course of moving forward the evaluation management to a systematic approach, as well as a good chance of achieving re-mobilization and re-guidance.
B. Who to evaluate a CBR programme
There are at least four ways to classify CBR evaluation in terms of the role of the evaluator, namely:
1. Self-Evaluation,
2. Mutual-Evaluation,
3. Higher authorities-evaluation, and
4. External-Evaluation.
Each type of evaluation has its own strengths and limitations. However, they should all be guided by the following principles: a. be based on facts; b. be comprehensive; c. to include both quantitative and qualitative aspects; and d. to include archival data and first hand field findings.
C. When should the evaluation be done
Both evaluation for CBR programmes and individual consumers of CBR services should be an ongoing and regular programme built-in activity.
D. The Evaluation process
The process could be conceptualized to have the following phases:
1. making a detailed evaluation plan,
2. collecting materials and conducting investigations,
3. analysing results,
4. making evaluation reports, giving suggestions,
5. feedback of results and putting them into practices (Helander, 1984).
E. What kind of information should be kept
1. Community profile: a. geographical feature of a community; b. population characteristics; c. physical environment; d. institution and network; e. attitude towards people with disabilities; and f. map of a community and lists of households.
2. Survey of disabled people in the community: a. general information of disabled persons; b. the history of disability; c. rehabilitation needs; and d. social aspects.
3. CBR action plan: To be a comprehensive, developmental plan, it should include the background of a community, situation analysis, expected goals, activities and actions measure, process and organization of implementation, financial management, as well as monitoring and evaluation. As an important part of a community developmental plan, CBR action plan is essential to the evaluation of CBR.
4. Implementation situation: a. measure and process; b. executive organization and personnel; c. financial management; d. monitoring and evaluation.
5. Rehabilitation training record of disabled people: a. general information of disabled people; b. rehabilitation programme and suggestion; c. implementation of rehabilitation programme; d. comprehensive assessment in phase; and e. the new rehabilitation programme and suggestion.
6. Statistical forms to collect information should indicate: a. quantitative changes in services of institutions or units concerned; b. quantitative changes in function, service, length, payment of personnel in CBR programme; c. situations on enacting and implementing legislation; d. training of staff; e. changes in barrier-free facilities of physical environment; f. number of disabled persons in CBR programme; g. changes in degree of disabilities; h. the improvement rate of functional training for disabled persons; i. changes in schooling and educational levels, such as rate of enrolment; j. changes in employment and income generation; k. changes in the role of family and participation in family life; l. changes in participation in social or community life.
7. Keeping evaluation plans, records and reports: The evaluation report should have a list of contents, acknowledgements, summary on the evaluation activities, general introduction to the CBR programme, key discovery, analysis of results, conclusions and suggestions, appendices, references and footnotes (Jonsson, 1994). A good evaluation report should be complete, precise, reliable, and comparable.
Part Two The Key Elements of CBR Evaluation
A. Evaluation of management
The management of CBR includes policy-making, planning, training of personnel, implementation at all levels, provision of resources, and monitoring and evaluation. It involves the government and its departments, NGOs, communities and disabled persons' families (ILO, et al., 1994).
1. Government's commitment: Government's leading role in CBR should include the following: a. to integrate CBR programme with the government working goals and local social development strategies; b. to set up CBR leadership group, headed by a community leader, and consisted of all departments concerned, with a special office in charge of the daily work; c. to make CBR plan; d. to use feasible networks in the community; e. to overall coordinate the various sectors involved in CBR; f. to make and implement rules, regulations and the staff duties; g. to establish and develop CBR resource center; h. to allocate personnel and set up professional consultancy groups; i. to provide financial support; j. to conduct evaluation at regular intervals.
2. Due to the special circumstance of disabled persons, the diversity of their needs, the broad extent of their participation in social life and the challenges of achieving their rehabilitation goals, it is essential to make all community sectors concerned to take due responsibilities of CBR services and work collaboratively in rendering timely services. These sectors are discussed below.
a. Public Health Sector to set up and improve primary health care network, train CBR professionals, provide training facilities, guide and offer referral services in medical rehabilitation, improve public health, and prevent disabilities.
b. Education Sector to initiate legislations and regulations to protect disabled people's rights in education, conduct integrated education, create educational conditions which are based on families and communities, launch education project for people with visual impairment and carry out anti-illiteracy education for disabled persons.
c. Labour (or Employment) Sector to initiate legislations and regulations to protect disabled people's rights of work, set up welfare enterprise or shelter workshops to promote employment of disabled people, through multiple channels at various levels and in a variety of forms to assist disabled persons to obtain employment, create opportunities and conditions to provide technical training for disabled so as to upgrade their skills and techniques, and conduct vocational training projects for job arrangement.
d. Social Affairs (or Civil Affairs) Sector to initiate legislations and regulations and preferential policies to improve the quality of life of disabled persons, provide social welfare services and placement.
3. Support of non-governmental organizations: NGO's evaluation support should emphasise social mobilization and awareness, participation in CBR planning, manpower training, co-operation with other sectors or organizations concerned, providing technical support and referral services, organizing volunteers, launching CBR demonstration projects, conducting CBR researches, provide welfare activities, and funding assistance.
4. Communities involvement: The force and motivation of development should come from the community itself. Community's involvement includes: integrating CBR programme with community developmental plans; funding support, the rehabilitation needs of disabled people; utilizing community's resources; mobilizing community members, disabled people and their family members to participate in CBR programme; community education; creating a sound environment and atmosphere for disabled people; and extending appropriate rehabilitation skills.
B. Evaluation of CBR implementation
1. Evaluation of CBR delivery system: Most countries in Asia and the Pacific region depend on the public health care network, social security network, women organizations, child health care and person's organizations. These networks form the backbone of CBR delivery systems and referral systems, with which disabled persons can get the rehabilitation service at family, community through national levels. An effective CBR network should be provided with four functions, namely, organization management, professional technique, monitoring and evaluation, and information and statistics.
2. Evaluation of the goals of comprehensive rehabilitation services (UN, 1993):
a. Services in medical rehabiliation
(1) the rate of disabled surveyed;
(2) the rate of making file and record for disabled;
(3) the rate of making rehabilitation plan for disabled;
(4) the coverage rate of medical rehabilitation services for disabled;
(5) the improvement rate of functional training of disabled;
(6) the change of disability prevalence.
b. Services in educational rehabilitation
(1) the rate of health and education to the community masses;
(2) the enrolment rate of disabled children;
(3) the anti-illiteracy rate of disabled people;
(4) training rate on Braille for people with visual impairment;
c. Services in vocational rehabilitation
(1) vocational training rate of disabled persons;
(2) the employment rate of disabled persons;
(3) the income of disabled persons;
d. Services in social rehabilitation
(1) the change of participation in social life of disabled persons;
(2) the change of barrier-free facilities for disabled persons in roads, buildings and houses;
(3) the change of facilities and accommodations for disabled persons at places of cultural, sports, recreation, etc.;
(4) the change of activities for disabled persons.
C. Evaluation of social beneficial results from CBR
1. feedback of community leaders,
2. feedback of community members to the CBR programme,
3. feedback of disabled persons.
Part Three Evaluation of Training Effects for Disabled Individuals
The "ability" of disabled persons, i.e. daily living self-care ability, mobility ability, work ability, learning ability, and social communication ability, is the basis for the evaluation of training effectiveness. These abilities are assessed together with the general income, employment, quality of life, schooling, education level, participation in social life and self-attitude of disabled persons.
Pro forma and data capture forms:
A total of 18 pro forma or forms have been designed. They are as follows:
1. Survey form - Community Profile,
2. Survey form - Household Visit,
3. Survey form - Disabled person,
4. Record form - Visit to the Home,
5. Monthly report form,
6. Evaluation form - Rehabilitation Effect of Disabled Person,
7. Changes of Daily Living Ability of Disabled,
8. Changes of Moving Ability of Disabled,
9. Changes of Social Communication Ability of Disabled,
10. Changes of Work Ability of Disabled,
11. Changes of General Income of Disabled,
12. Changes of Quality of Life of Disabled,
13. Changes of Employment Status of Disabled,
14. Changes of Schooling Status of Disabled,
15. Changes of Education Level of Disabled,
16. Changes of Participation in Social Life of Disabled,
17. Changes of Self - attitude of Disabled,
18. The Collective form - Evaluation of Rehabilitation Effect of Disabled.
Conclusion
It is inevitable that the Guidelines presented above would contain limitations. The researchers would appreciate critical comments from specialists, reseachers, CBR workers, colleagues and people with disabilities.
References
Helander, E. (1984). A Guide to the Management of Community-Based Rehabilitation 1. Policymaking and Planning. RHB/84. 1 Provisional Version. Geneva: WHO.
Helander, E. (1993). Prejudice and Dignity, An Introduction to Community-Based Rehabilitation. UNDP. New York: United Nations.
Helander, E., Mendis, P., Nelson, G. (1980). Training the Disabled in the Community, Version 2. Genera: WHO.
ILO, UNESCO, WHO. (1994). The Joint Position Paper-Community-Based Rehabilitation, CBR for and with People with Disabilities.
Jonsson, Ture. (1994). A Guide on Operations Monitoring and Analysis of Results. OMAR in Rehabilitation.
Marie-Therese, Feueerstein. (1986). Partners in Evaluation. Macmillan Publishers.
UN (1993). The Standard Rules on the Equalization of Opportunities for Persons with Disabilities. New York: United Nations.
UNDP, (1993). Guide on Evaluation of Rehabilitation Programmes for Disabled People. First Draft.
WHO, (1981). Expert Committee on Disability Prevention and Rehabilitation. WHO Technical Report Series 1981, 688, 7-37.
ASIA & PACIFIC JOURNAL ON DISABILITY
Vol.1, No. 1, September 1997
Published by the Asia and Pacific Regional Committee of Rehabilitation International (RI) and the Regional NGO Network (RNN)