Related Tools

Evaluation

Last edited: December 21, 2011

This content is available in

Options
or
Options

Effective programme evaluation can:

  • Support programme improvements so as to enhance outcomes for women.
  • Document programme knowledge for dissemination and applicability in other contexts – how can others learn from our experiences and apply our models to their local problems?
  • Support accountability – were resources expended in the manner planned and did the resources contribute to achieving programme goals? Why or why not?
  • Provide information on results for donors and other audiences.

The United Nations Development Programme Handbook on Planning, Monitoring and Evaluating for Development Results includes a detailed description of general considerations when creating an evaluation methodology. The UN Women website on gender-responsive evaluation provides guidance on integrating a women’s rights perspective into evaluation.

Specifically related to violence against women in the informal sector, evaluations overall should examine whether:

  • women and girls are safer in their homes and communities as a result of the programme implementation;
  • mediation and restorative justice practices are utilized; these should never be implemented in cases of violence against women and girls.
  • womens and girl’s human rights are recognized and supported when they experience violence;
  • women and girls are better able to access justice and support when they experience violence;
    • this might include improvements in physical access, financial access, timeliness, confidentiality, and supportive services depending on the programme goals.
  • perpetrators are subject to effective sanctions that deter future violent behaviour; and attitudes towards violence against women and girls have changed in a positive direction.

Sample Evaluation Interview Questions (UNDP, 2010)

Interview Guide for Beneficiaries 

Have you ever heard of the [community court programme]? Were you involved in this process? How?

What are your observations of the [community court programme]? What do they do? Is it important? Why/why not?

Have they helped you? How have they treated you? Was the process fair? Are you happy with the outcome? If you had another problem like this, would you go to them again?

What process do they use to come to agreements?

How do you see the effectiveness of the project? Any successes/challenges/constraints pertaining to project implementation? What contribute to the success you observe?

Are there any organizations doing similar work? Is the work of [the community court programme] the same or different?

Have you seen them involved in other situations? In what types of situations have they been involved? Appropriate or not?

Do you think the [community court programme] should be continued? Do you think you would do something differently in the future or the same? Why and how? What are the best practices from this process you observed?

Evaluation Challenges in the Informal Sector

Challenges in programme evaluation mirror many of the challenges in programme implementation in the informal sector including logistics, participation, and working with low literacy populations.

  • Many programmes in the informal sector are community-level trainings. Tracking down rural, poor participants, especially women and girls, after-the-fact may be difficult or impossible in some cases, unless detailed contact information was recorded during the training.
  • Many community members are not familiar with the purposes of evaluations or may not have participated in surveys or structured interviews before. Explaining the purpose of the evaluation can be a challenge, as can ensuring that respondents do not simply tell the evaluator what they think the programme organizers want to hear.
  • Especially in rural communities, accessing participants to conduct the evaluation can be a challenge, involving significant travel time.
  • If participants received incentives to participate in a particular training or activity, they may expect similar incentives to participate in an evaluation of that process. This expectation should be anticipated and addressed early on.
  • Participants may have low literacy levels and evaluators should plan for this by adapting their evaluation methods accordingly.

These challenges highlight the importance of including an evaluation plan from the beginning of the programme and ensuring that considerations for the logistics of an evaluation are part of the implementation of the programme, such as:

  • Conducting pre- and post-programme assessments immediately before and after the implementation of the programme.
  • Talking with programme participants about the fact that the programme will be evaluated at some point in the future and letting them know it is a part of the programme activities.
  • Talking with participants about how they think the programme could most effectively be evaluated during the implementation process.
  • Identifying control group communities in advance of programme implementation and considering early on how to garner their participation when they have little or no familiarity with the programme.
  • Ensuring that evaluators have familiarity with the communities and cultures where they will be gathering evaluation data, so that they can make an instinctual assessment of whether participants understand the questions and are giving responses that reflect real results, instead of what evaluators want to hear.
  • Keeping data collection instruments short, simple, and focused on only the most pertinent information.
  • Considering what type of sample size will be needed to meet the goals of the evaluation – for example, will feedback from 30% of participants be enough? 5%? 90%? Plan for how to access the desired sample.

(Interview with Margaret Snoeren, American Bar Association Rule of Law Initiative, January 25, 2011.)