Part One: The Outlines of Evaluation of CBR

1. What is evaluation of CBR

Evaluation means assessment. A working definition of evaluation of CBR refers to a standard to make objective judgements on the activities and outcomes of CBR programme and the rehabilitation efficacy of people with disabilities in line with the goals, strategy, action plan, implementation of CBR programme and the rehabilitation training scheme of the consumer. Generally speaking, there are three types of evaluation: goal evaluation, process evaluation and outcome evaluation. Relevance, effectiveness, efficiency, sustainability and impact are the core factors that should be considered in the evaluation of CBR.

(a) Relevance
Does the programme meet the needs and offer comprehensive rehabilitation services for people with disabilities and their families, and does it relevant to the programme goals?

(b) Effectiveness
Does the programme achieve its objectives both in terms of quality and quantity, and does it make satisfactory result in service coverage and the rehabilitation efficacy of people with disabilities?

(c) Efficiency
Does the programme make the maximum outcome with the minimum income, and does it use the resource in the most efficient way?

(d) Sustainability
Can the programme continue once the external assistance is withdrawn?

(e) Impact
What effect has the programme had on its social settings, economic development, rehabilitation technique and institution, etc.

2. Why evaluate a CBR programme

(a) Assess the pertinence, implementation process, effectiveness and efficiency in accordance with the action plan of CBR programme.

(b) Find out the changes happened in community development, attitude and sense to disability and rehabilitation, etc.

(c) Judge whether the CBR service provided is effective, feasible, economical and practical..

(d) Assess the cooperation among sectors of community, the implementation of CBR programme, the approval, acceptance and participation of the community leaders, professionals, disabled people and their family members.

(e) Identify whether the disabled people have achieved the rehabilitation goals.

(f) Examine and explore the contents, methods, process of evaluation in CBR programme.

(g) Systematic collect, analyses and interpret information about the activities and outcomes of CBR programme in order to provide scientific basis for community leaders and person concerned.

In other words, the evaluation is a process of learning experience, drawing a lesson, revising plan, and improving ongoing programme, a course of changing the evaluation management from traditional and experimental way to modern and scientific way, as well as a good chance of re-mobilization and re-guidance.

3. How to evaluate a CBR programme

For achieving the evaluation purpose mentioned above, it is necessary to follow the principles of evaluation and take appropriate evaluation measures.

(a) Self-Evaluation: The evaluators are said to be those who are inside the programme or know the programme very well. For examples, programme administrators, executors, target people, service consumers, etc. They are familiar with the programme and can make contribution to the evaluation activities.

(b) Mutual-Evaluation: The evaluation is made between (or among) two (or more) CBR programmes. The evaluators come from outside their own community. They can exchange experiences, draw on each other's advantages and avoid disadvantages.

(c) Higher authorities-evaluation: The evaluation is made by superior level departments that are in charge of the programme. Therefore the programme can get the understanding, support and suggestion from their higher authorities directly.

(d) External-Evaluation: The evaluation is carried out by organizations and persons from outside the country or the community. Outsiders can take a fresh look at the programme and will not be personally involved, so the evaluation may be more objective.

Each type of evaluation has its own strengths and limitations. However, they should all be guided by the following principles: (a) be based on facts, (b) be comprehensive, (c) to include both quantitative and qualitative aspects, and (d) to include archival data and first hand field findings.

4. When should the evaluation be done?

The development of CBR is a on-going process. For reaching the goals of a CBR programme, there is a need for implementation to be in phases. Therefore, each phase's evaluation not only has to identify CBR programme for the particular phase, but also improve the programme's work for the next phase, and draw on the experiences and lessons for achieving the final goals of the CBR programme.

There are many factors relating to the realization of rehabilitation goals for disabled people. The key factors are not the types and degrees of disabilities but the attitude towards disabled people, when the rehabilitation activities start, what kind of rehabilitation measures are used, and social factors, etc. For example, a person with visual impairment can reach a higher level rehabilitation goal, but a person with better a better eyesight may only reach a lower level rehabilitation goal. Similarly, there is a need for achieving the best rehabilitation goal in each phase. Both evaluation for CBR programme and individual consumer of CBR service should be an ongoing activity. Generally evaluation should be built into the CBR programme at regular intervals. For instance:

(a) Monthly Evaluation
(b) Evaluate in stage: 3 month, half a year, one year according to the programmes development.
(c) Mid-Time Evaluation: It is essential to evaluate CBR programme during the mid- time period. It can find out the experience and problem in order to decide what and how we should do.
(d) Eventual evaluation: Evaluation should be made at the end of the programme it should be the most important, detailed and overall of the all evaluation activities.
(e) Follow-up Evaluation: some outcome or findings such as disability rate attitude change, impact and substainability can only be shown after a longer time. So it is necessary to conduct evaluation at a specified future date.

The diagram below shows the relationship between the implementation process and evaluation activities:

make plan ------- implementation ------- programme end ------------- after a period (baseline survey) (periodical evaluation) (eventual evaluation) (follow up evaluation)

5. Evaluation process

(a) Making a detailed evaluation plan.
(b) Collecting materials and conducting investigation.
(c) Analysis results.
(d) Making evaluation report, giving suggestion.
(e) Feedback of results and putting them into practice.

6. What kind of information should be kept?

The following is a list of the kind of information needed for evaluation:

(a) community profile (see table 1)
Survey on a community profile can provide an outline or a brief description, and get basic information of a community, such as general description, ethnic, culture, religious, situation on socio-economical development, environment, quality of life of disabled people, health and social activities, etc. People may think that they are familiar with their own community, and know it very well, but they may lack information which is essential for a CBR programme. So while paying attention to the survey of disabilities, we should not neglect the survey on the community profile.

A community profile includes the following contents:

(i) Geographical features of a community: This tells about the location, size, area, resources, climates, etc.

(ii) Population: This tells about the population size, nation, ethnic, cultural, religious, prevalence of diseases and disabilities, attitudes to disabled people, etc.

(iii) Physical and social environment: This tells about the situation of economic, traffic, communication, barrier-free facilities, the number (the location, scale, equipment of hospitals, rehabilitation centers, schools, shelter workshops, welfare units, vocational training stations and recreation places, etc.

(iv) Institution and network: This tells what you will need for launching CBR project, such as, the administrative set up, government departments, non-governmental organizations and implementation networks, for instance, primary health care network, social welfare network, disabled person's federation, women's organization, etc.

(v) Attitude towards people with disabilities: Along with the socio-economical development and social progress, the attitudes to people with disabilities have changed greatly. The word "attitude" has two meanings. One meaning is "understanding, respect, care, help disabled people; another meaning is prejudice, discrimination, neglect, ignorance, refuse disabled people. Wrong attitudes will impede reaching the rehabilitation goals of disabled people and the progress of CBR programme.

(vi) Map of a community with the lists of households: The map should show roads, paths, fields, water sources markets, important rehabilitation places and all households in the programme area and number in each household.

(b) Survey of disabled people in the community (see table 2-3)

The survey of disabled people should include: the following contents:

(i) General information of disabled person: Name, sex, age, nation, address, the name of household head, his(her) work unit.
(ii) The history of disability: The type and degree of disability, the causes of disability, rehabilitation or medical measures accepted and the assessment of abilities for living, learning working and social activity of disabled person.
(iii) Rehabilitation needs: It includes needs that the disabled people may have in medical care, rehabilitation service, education, employment, participate family's life and social activities of disabled person. It is necessary to judge the priority among all the rehabilitation needs.
(iv) Social aspects: It includes marriage status, family composition, source of income, employment status, attitude from family members and disabled person himself or herself, etc.

According to the individual information from the survey, we can analyse the following aspects:

(i) The percentage of the number of households with disabled people in the total number of households surveyed.

(ii) The percentage of people with disability in all people surveyed.

(iii) Age distribution and age distribution according to their disability.

(iv) Sex distribution and sex distribution according to their disability.

(v) Statistics of disability types.

(vi) Statistics of disability degree.

(vii) Status analysis of disabled children.

(viii) Income and employment status of disabled persons.

(ix) Education level of disabled persons.

(x) Marriage and family status of disabled persons.

(xi) Rehabilitation needs according to their disability.

(xii) The causes of disability according to disability type.

(c) CBR action plan
CBR action plan is made according to the laws and regulations of the state and adapted to local conditions. Being a comprehensive developmental plan for CBR programme, it includes the background of a community, situation analysis, expecting goals, activities, action measures, process, organization of implementation, financial management, monitoring, evaluation, and experience spread, etc. As an important part of a community developmental plan, CBR action plan is helpful to the overall evaluation of CBR.

(d) Implementation situation

(i) Measure and process: It includes the action time, (year, month, date), place, measures, contents, goals, etc. For example, when was the CBR leading group established? Where is the office? Who are the members of leading group? What are the duty and task? etc.

(ii) Executive organization and personnel: Who participate in the programme? Who are the persons in charge of the programme? etc.

(iii) Financial Management: The budget, the source of fund, funding distribution and expenditure, etc.

(iv) Monitoring and evaluation

(e) Rehabilitation training record of disabled people

A training file for disabled people should include:

(i) general information of disabled people

(ii) rehabilitation programme and suggestion

(iii) implementation of rehabilitation programme (see table 4-5 or see the form recommend in TCPD "to find out if person need training and to assess progress")

(iv) comprehensive assessment in phase: According to the results of assessment at the first time, mid-time and eventual time, to give the mark of rehabilitation effect (high level, middle level and low level).

(v) The new rehabilitation programme of suggestion: According to the eventual evaluation results, revise the rehabilitation programme and give suggestion. It marks the rehabilitation training and activities step in a new phase.

(f) Statistical forms: According to all the requirement for CBR Action Plan, report the statistics results to sectors concerned level by level, statistics should be carried out at least half a year or every year. Statistics results are expected to show:

(i) What changes have happened in number and services of institutions or units concerned CBR programme. For instance, health facilities (health post, primary health center, district hospital, province hospital, national hospital or center); schools (pre-schools, primary schools, secondary schools, vocational training centers, special education schools or classes); recreation facilities, etc.

(ii) What changes have happened in number, function, service, length, payment of personnel in CBR programme, such as administrators, professionals, local supervisors CBR workers, family guides, etc.

(iii) Situation on making and implementation of laws (including regulations and local statutes)

(iv) Situation on training staff in CBR Number, contents, teaching methods, length of course, examination, evaluation (pre-course, after-course and follow up) the effect of training course, etc.

(v) Changes happened in barrier-free facilities of road, buildings and house environment, etc.

(vi) Changes happened in number of disabled persons in CBR programme. Name, sex, age, type of disability, training activity and progress, etc.

(vii) Changes of degree of disabilities.(slight, medium, severely)

(viii) The improvement rate of functional training for disabled persons, for instance, improvement in daily living care, behavior, communication, mobility, etc.

(ix) Changes of schooling and education level; the rate of enrolment (or number) and the changes of individual in education level.
(x) Changes of employment and income generation .

(xi) Changes in the role of family and participation in family life.

(xii) Changes of participation in social (community) life.

(g) Keeping evaluation plans, records and reports

The evaluation plan should indicate the objectives, methods, contents, date and period, participants, results and suggestions.

The evaluation report should contain: list of contents, acknowledgement, summary on the evaluation activities, general introduction on the CBR programme, key discovery, analysis of results, conclusions and suggestions, appendix, references and footnotes.

A good evaluation report should following guidelines:

(i) Completeness: It means the evaluation report should be detailed and comprehensive.

(ii) Preciseness: It means the evaluation report should be accurate and no mistake.

(iii) Reliable: It means the evaluation report should be real and objective.

(iv) Comparable: It means the evaluation report can compare with the expecting goal of the CBR programme or with other CBR programme.

Go back to the CONTENTS

Evaluating Community Based Rehabilitation :
Guidelines for Accountable Practice

By Dr. Tizun Zhao, Joseph K.F. Kwok Ph.D, J.P.