Evaluation questions articulate the main issues that will be explored by the assessment. They are usually developed after the goals and objectives of a programme have been decided and the activities to support those objectives have been determined. Evaluation questions are useful to:
- Focus and provide structure to an evaluation;
- Guide the evaluation planning process, including data gathering and the methods to be used to obtain the information that is important to the programme implementers, the beneficiaries, donors and other stakeholders; and
- Inform how the results will be incorporated back into planning and implementation to improve the programme.
Steps to Developing Evaluation Questions
The most useful evaluation questions reflect a diversity of stakeholder perspectives, key components of the programme; the most important information needs, and resources available to answer those questions. Steps include the following:
1. Gather relevant stakeholders. Engage some or all of the stakeholders that were involved in the strategic planning phase to help develop evaluation questions or to share the questions that have already been developed to get their inputs and feedback.
2. Review supporting materials such as the strategic plan, programme monitoring and evaluation framework, the work plan, and any other available resources that are relevant to the programme.
3. Brainstorm evaluation questions about the overall programme or a specific programme activity. Focus on the goals, strategies and objectives in the strategic plan and workplan – the inputs, activities, and outputs to generate process evaluation questions. Many questions may be identified that can later be reduced, fine-tuned and prioritized.
4. Sort evaluation questions from the brainstorming session into categories or groups that are relevant to the programme and stakeholders. This process will help determine what resources exist to aid in answering the evaluation questions that are a priority and most important.
5. Decide which evaluation questions to answer. Evaluation questions should be prioritized that:
• Are important to programme staff and stakeholders
• Address important programme needs
• Reflect five-year programme goals, strategies, and objectives of your programme
• Can be answered with available resources, including funds and personnel expertise.
• Can be answered within the available timeframe.
• Provide information to make programme improvements
• Will be supported by partners of the programme
6. Verify that the questions are linked to the programme. Once the questions are determined, they can be checked/verified against the programme strategic plan, monitoring and evaluation framework and work plan to make sure they fit and will accomplish what they are setting out to do.
7. Determine who, what, and how to collect the data that will be required for answering the evaluation questions. Determine who will be responsible for collecting the information and analyzing it to answer the evaluation questions. Possible data sources may include persons (e.g. survivors of violence that are engaged with criminal proceedings), documents, or records.
Source: Adapted from Department of Health and Human Services, Centers for Disease Control and Prevention.2009.Evaluation Briefs, No. 4. Available in English.
Illustrative evaluation questions from the United Nations Trust Fund to End Violence against Women and Girls cohort on violence against women and HIV
A) For interventions aimed at strengthening capacity of service providers:
- Has a multisectoral network been built to improve access to services for women living with violence?
- Are women accessing and using quality services more effectively and efficiently?
- Did the capacity development activities strengthen understanding of the links between violence against women and HIV and build capacity among service providers for addressing those links?
B) For interventions aimed at raising awareness and transforming norms around violence against women and HIV
- Did the twin media and education strategies increase knowledge around violence against women women and HIV?
- Did the mobilization activities change the attitudes and beliefs of community members?
- Do the peer to peer networks increase women’s use of services?