Lesson 28 Evaluation of Watershed Programs

28.1 Scope of Watershed Program Evaluation

Evaluation is an important aspect of watershed programs. It is a multi-dimensional task which is generally performed at different times during the implementation of watershed programs. Until recently watershed program evaluators tended to favor either a quantitative or a qualitative evaluation.  Typically, quantitative evaluations reflect a simplistic view that reality takes a single form that can be perceived and measured objectively. On the other hand, qualitative evaluations reflect a more constructive view, implying that reality can have multiple versions.

There is a rising interest in mixing both the qualitative and quantitative methods of watershed program evaluation. This comes from the fact that purely quantitative and purely qualitative approaches to watershed program evaluation both have limitations. The strengths of each evaluation often compensates for the weaknesses of the other evaluation.

Quantitative Evaluation of Watershed Programs 

The quantitative evaluation of watershed programs attempts to attribute changes in various outcome variables to a project intervention (i.e., ‘treatment’) and determine whether such effects are statistically significant.  An experimental approach is often considered as an acceptable standard for quantitative evaluation of watershed programs. Yet, in many cases the results of such a study may not extrapolate beyond the watershed projects examined. 

There are many situations wherein an experimental approach to quantitative watershed program evaluation may not be possible. In such situations, various approaches have been used, each with their own strengths and limitations. 

The first approach is called a “before/after” study. The evaluator measures the levels of outcome indicators in a watershed area before and after a watershed treatment. This is a fairly weak but feasible approach that involves an unlikely assumption that there have been no other significant changes during the study period. 

A second approach consisting of a “with/without” study, is useful when no baseline data are available. This is often the case when an evaluation is commissioned after a watershed project has been implemented.

Cost-benefit analysis has long been the method of choice in economic appraisal of agricultural development and irrigation projects.  Cost-effectiveness analysis is similar but it estimates only the costs of alternate approaches of achieving a given objective.  Cost-benefit analysis aims to evaluate costs and benefits that occur with a project and compare them to what would happen without the project.  Even if all costs and benefits could be identified and valued, cost-benefit and cost-effectiveness analysis would give only a single assessment of overall project performance. However, watersheds consist of multiple users who are affected differently by the project. A favorable benefit-cost ratio could temporarily mask uneven distribution of benefits, yet those who do not benefit may be in a position to undermine the project.

Thus there are clearly multiple challenges associated with using quantitative evaluation methods for evaluation of watershed projects. Most challenges are introduced by the fact that watershed projects are not amenable to the same controlled conditions as in the experiments which provide the data for a simplistic analysis.   

Qualitative Evaluation of Watershed Programs    

In contrast to quantitative evaluation, qualitative evaluators typically place less emphasis on measurement and more on context and on understanding the subtle manifestations. In general, a qualitative approach tends to be flexibly structured and uses open-ended questions in an inductive fashion. The objective is not to obtain a numerical estimate of some phenomenon, but to develop an in-depth understanding of an issue by probing, clarifying, and listening to stakeholders talk about a topic in own words. The in-depth nature of the qualitative approach means that a study’s scale is usually smaller than that found in quantitative research.

As with quantitative evaluation of watershed programs, sampling issues in qualitative evaluation also raise questions about biases in data. While quantitative researchers use random sampling whenever possible, qualitative researchers use several strategies to increase the internal validity of their findings. In qualitative evaluation, data collection and analysis become inseparable; as such researchers collect much of the data themselves, rather than relegating this task to field assistants.

Mixed Evaluation of Watershed Programs

Researchers use mixed evaluation of watershed programs for various reasons. Here, qualitative and quantitative components may be used either sequentially or in parallel or in an integrated fashion.  When qualitative and quantitative components in a mixed evaluation are used in an integrated manner, the information and data collected from one activity is used for the other activities of the evaluation process also.

28.2 Indicators and Stages for Watershed Program Evaluation

Watershed program evaluation can be quantified in terms of certain indicators.  These indicators are the measures of targets or goals of the watershed project implementation, which facilitate the expected positive change in the watershed projects. They also give an insight into and quantify the process of evaluation. The various indicators generally used for watershed program evaluation are discussed in the following sections:                                             

i) Technical Indicators

Technical indicators in Watershed Program Evaluation include the extent of soil loss and runoff, amount of discharge in the stream and amount of sediments in flowing water at the outlet point, increase in the yield of wells and rise in water table, average annual water flow and flood peak, changes in soil moisture, concentration of suspended sediments, annual sediment yield, turbidity of water, biological and chemical properties of water, pH, annual reservoir sedimentation, pesticide concentration, etc.

ii) Common Property Resources (CPR) Use Indicators

Common property resources (CPR) use indicators are productivity of crop, fodder, fuel wood, pasture land, community forest land and milk. Further information to be collected are areas of managed agro-forestry, protected degraded forest land by social fencing, unprofitable cropland and grazing land, unused area with agro-forestry and areas of common property resources.

iii) Institutional Building and Community Organization Indicators

These indicators include the number of rural development institutions in the watershed and the coordination among them, financial independence of the institutions, their capacity building to solve managerial, administrative  and  financial problems, the number of trained professionals assigned to the project, the number of welfare  and development programs performed by the institutions, the number of farmers trained in soil conservation and modern agriculture techniques, the percentage of population willing to adopt appropriate technology to improve crop, livestock, water harvesting, etc., and the performance of self-help groups, user groups and watershed development committees (WDCs).                                                           ·

iv) Ecological Improvement Indicators

Ecological improvement indicators include the biodiversity and biomass indices, severely eroded, overgrazed and over-utilized lands, wastelands, lands under shifting cultivation, stabilized slopes, areas of treated ·gullies, number and depth of gullies, soil fertility and organic matter content of soil.

v) Economic and Social Indicators

These indicators quantify the change in the living standards, .household savings, household expenditure and household income, number of families living above poverty line (APL) or below poverty line (BPL), extent of migration to urban areas in search of .employment and indebtedness in cash or kind, prevailing wage rate in agriculture and non-farm sectors, changes in crop production, double cropped areas, agricultural and non-agricultural land values, number of annual man days generated, number of working women and young people per year, time spent in fetching and collecting drinking water, annual request for technical assistance and skill up gradation of rural artisans.

vi) Essential Service Indicators

These indicators include the literary rate, number of schools in operation, percentage of school attending children and their  age, number  of  primary  school dropouts, percentage of houses having electricity connection and drinking water facilities, number of dispensaries in operation per year and the families receiving medical care, annual mortality, percentage of population of age group 0-16 years receiving immunization, couples protected under family planning, annual birth rate, number of annual sterilizations, length of motorable road added per year in kilometers, level of child malnutrition below 1 year age group and availability of essential commodities.

Stages of Watershed Program Evaluation

It is a common practice to carry out the watershed program evaluation in four stages with the help of the six indicators mentioned earlier. These four stages are discussed here:

i) Baseline Evaluation

This is the evaluation in the initial planning stage. The data on the indicators are used as benchmark for evaluation. A reliable baseline data on hydro-meteorological, economical, social, physical and biological parameters are provided for this evaluation.

ii) Mid-term Evaluation

This evaluation is done in the middle of the watershed program implementation. In this stage of evaluation, initial problems in the planning are overcome and the flow of inputs to the target population is commenced and their response can be observed.  The purpose of such mid-term ·evaluation is to check on ·the effectiveness of each individual activity. This evaluation quantifies the short and mid-term benefits of the project.

iii) Terminal Evaluation

This evaluation is done at the end of the project economic life. It indicates the efficiency of project implementation, accuracy of the project estimates, etc.

iv) Post-Terminal Evaluation

This evaluation is carried out after 5 to 15 years of watershed program period. Long-term effects and impacts become visible in this post-terminal evaluation.

Impact of Evaluation on Watershed Management

The evaluation of watershed management during a normal year and a year under stress conditions is very difficult and complex. Developmental works in a year under stress decline significantly. Therefore, the aim of the watershed management should be to focus on utilization and harnessing of existing resources for the maximum production and benefits. One of the main thrusts of watershed management programs should be to minimize the differences in the benefits during a normal year and a year under stress, as far as possible.

Keywords: Watershed program evaluation, quantitative watershed program evaluation, qualitative watershed program evaluation, watershed evaluation indicators, watershed evaluation stages.

References

  • Das, M. M. and Saikia, M. D. (2013). Watershed Management, PHI Learning, New Delhi, India, pp. 290-297.

  • Kerr, J. and Chung, K. (2001). Evaluating Watershed Management Projects, Collective Action and Property Rights (CAPRi) Working Paper No. 17, International Food Policy Research Institute, Washington, DC, USA.

Suggested Reading

  • Becerra, E. H., Velez, I. (1995). Monitoring and Evaluation of Watershed Management Project Achievements, Food and Agricultural Organization (FAO) Conservation Guide 24, Rome, Italy.

  • United States Environmental Protection Agency (USEPA), (2008). Handbook for Developing Watershed Plans to Restore and Protect Our Waters, USEPA.

Last modified: Friday, 7 February 2014, 9:41 AM