Keep in mind that evaluations should be based on operational and theoretical frameworks and that they should be incorporated in a programme’s planning stages. Baseline and situation analyses are critical to monitoring and evaluation efforts, but rarely conducted. Please refer to the introductory section for additional information on developing an appropriate framework and collecting baseline data. (Bott, Guedes and Claramunt, 2004)
Define a clear programmatic goal for the intervention - A goal reflects the basic, very broad, conceptual aim of the project and the desired long-term outcome.
Examples of possible goals include:
1) To improve the quality of care that survivors of gender-based violence receive in health care settings.
2) To strengthen the ability of the health care sector to prevent gender-based violence.
Keeping this overall goal in mind, identify clear objectives and expected results.
- Remember to keep in mind the difference between proposed activities, outputs and outcomes, between what will be undertaken, what will be produced and what is expected will happen as a result. For example:
- Activities might include conducting training for health service providers or developing standardized protocols for responding to cases of sexual violence.
- Outputs might include the number or percentage of health service providers in a target area who have been trained or the number of health care facilities that have adopted standardized protocols for responding to cases of sexual violence.
- Outcomes might include strengthened capacity on the part of health service providers to respond to violence against women in a meaningful, appropriate manner or an integrated response on the part of the health care facility following standardized protocols.
Develop indicators for measuring each of the objectives
- Remember to distinguish between process and results indicators. Health programmes usually collect data on processes rather than on results or outcomes and may not focus on whether their activities were beneficial or effective. This does not mean however, that monitoring and evaluation frameworks should exclude process indicators.
- Process Indicators are used to monitor the number and types of activities carried out, such as the number and types of services provided, number of people trained, number of materials produced and disseminated or number and percentage of clients screened.
- Results Indicators are used to evaluate whether or not the activity achieved the intended objectives. Examples include indicators of providers’ or community level knowledge, attitudes and practices as measured by a survey, women’s perceptions about the quality and benefits of services provided by an organization or institution as measured by individual interviews, women’s experiences with health care, and the appropriateness or readiness of health unit capacity and infrastructure. (Bott, Guedes and Claramunt, 2004)
Examples of strategies, objectives and indicators for monitoring and evaluation of health sector initiatives
|
Strategy/ intervention |
Examples of possible objectives |
Example indicators |
|
1) Dissemination of materials/ information |
Raising health care providers’ awareness and understanding of gender-based violence, in particular: a) GBV as a critical human rights and public health issue b) barriers women living with violence or survivors of violence face when accessing services c) links between GBV and HIV and AIDS d) laws addressing GBV and providers’ responsibilities |
|
|
2) Training of service providers |
Strengthening health care providers’ ability to respond to cases of gender-based violence [in particular…] a) following appropriate routine screening protocols b) responding to cases of rape and sexual violence c) addressing GBV and HIV/AIDS links holistically d) establishing and using community-based referral networks of care providers and social services e) improving medico-legal documentation of cases f) changing stigmatizing norms and attitudes g) providing emergency and crisis care
Strengthening health care providers’ ability to prevent possible gender-based violence through: a) changing stigmatizing norms and attitudes b) strengthening capacity to screen for possible violence, provide appropriate care and referrals as necessary |
|
|
3) Development of protocols and norms for managing GBV cases |
|
|
|
4) Routine screening |
|
|
|
5) Campaigns to empower women |
|
|
Source: PATH, 2010.
Case Study: International Planned Parenthood Western Hemisphere Region Evaluation to Improve the Health Sector Response to Gender-Based Violence
The evaluation included four main components.
1. A baseline evaluation study including:
A knowledge, attitudes and practices survey of providers using face to face interviews. IPPF/WHR designed a survey questionnaire to gather information on health care providers’ knowledge, attitudes, and practices related to gender-based violence. The questionnaire contains approximately 80 questions. Although the questionnaire includes a few open-ended questions, most of the questions are closed-ended so that the results can be tabulated and analyzed more easily. The questionnaire covers a range of topics, including: whether, how often and when providers have discussed violence with clients; what providers think are the barriers to screening; what providers do when they discover that a client has experienced violence; attitudes toward women who experience violence; knowledge about the consequences of gender-based violence; and what types of training providers have received in the past. This questionnaire can also be adapted to evaluate a single training. One possibility is to use all or part of the questionnaire before the workshop begins and use only part of the questionnaire after the workshop is over. If the questionnaire is used immediately “before and after” a single training, the organization may be able to measure changes in knowledge, but changes in attitudes and practices usually take time.
A clinic observation/ interview guide The Clinic Observation/Interview Guide gathers information on the human, physical, and written resources available in a clinic. The first half of the guide consists of an interview with a small group of staff members (for example, the clinic director, a doctor, and a counselor). This section includes questions about the clinic’s human resources; written protocols related to gender-based violence screening, care, and referral systems; and other resources, such as whether or not the clinic offers emergency contraception. Whenever possible, the guide instructs the interviewer to ask to see a copy or example of the item in order to confirm that the material exists and is available at the clinic. The second part of this guide involves an observation of the physical infrastructure and operations of the clinic, including privacy in consultation areas (for example, whether clients can be seen or heard from outside), as well as the availability of informational materials on issues related to gender-based violence.
2. Service statistics on detection rates and services provided using standardized screening questions and indicators.
Sample tables for gathering screening data. To ensure that all three participating associations could collect comparable screening data, IPPF/WHR developed a series of model tables, which each association completed every six months. These tables may or may not be useful for other health programmes, as this depends on whether or not the health programme decides to implement routine screening, what kind of policy it adopts, what kind of questions it asks, and what kind of information system it has. Nevertheless, these tables illustrate the types of data that can be collected and analyzed on a routine basis.
3. A midterm, primarily qualitative, evaluation including:
Focus group discussions and in-depth interviews with providers, survivors and external stakeholders/key informants: A summary protocol for collecting qualitative data describes these methods, including in-depth interviews and group discussions; and also provides an idea of what types of providers, clients and other stakeholders were asked to participate.
Client satisfaction surveys: The Client Exit Survey Questionnaire is a standard survey instrument for gathering information about clients’ opinions of the services they have received. This survey is primarily designed for health services that have implemented a routine screening policy. It is important to note that exit surveys tend to have a significant limitation: many clients do not want to share negative views of the services, especially when the interview is conducted at the health center. IPPF/WHR was not able to interview clients offsite, but it did arrange for all the interviewers to be from outside the organization, so that they could reassure the women who participated that they were not going to breach their confidentiality. This questionnaire contains mostly closed-ended questions about the services. It asks women whether they were asked about gender-based violence and about how they felt answering those questions; however, the questionnaire does not ask women to disclose whether or not they have experienced violence themselves.
Case studies of pilot strategies to address various aspects of gender-based violence.
4. A final evaluation serving as a follow up to the baseline including:
- KAP survey of providers using face to face interviews
- A clinic observation/ interview guide
- Random records reviews and development of a protocol:
Throughout the course of the IPPF/WHR regional initiative, the participating associations gathered routine service statistics about clients, including the numbers and percentages of clients who said yes to screening questions. However, the quality of these service statistics depends on the reliability of the information systems and the willingness of health care providers to comply with clinic policies—both of which may vary from clinic to clinic. IPPF/WHR therefore designed a protocol to measure screening levels and documentation using a random record review approach. This manual contains a brief description of the protocol as well as a tabulation sheet.
Download the main publications related to this initiative:
Basta! The Health Sector Addresses Gender-Based Violence. Available in English and Spanish.
Improving the Health Sector Response to Gender-Based Violence. Available in English and Spanish.
Source: Bott, Guedes and Claramunt 2004.