A monitoring and evaluation (M&E) plan or framework defines the indicators that measure progress towards the campaign objectives. It determines what data must be collected at what moments of the campaign, the methods of data collection, and how findings will be analyzed and reported.
To make campaign M&E meaningful, data gathered needs to be analyzed in time and fed back into the campaign management process. This can be done most effectively through regular meetings that should be documented.
Three major questions should inform the design of the M&E plan:
- What information is needed to effectively implement the campaign and draw learning from it? Both internal aspects (e.g. campaign activities and outputs) and external developments (e.g. changes in the policy environment) need attention. But one cannot monitor every single element of what a campaign does and the context in which it occurs. Instead, M&E design must focus on crucial aspects. For example, in innovative behaviour-change campaigns, it is important to monitor those whose behaviour the campaign attempts to influence to change.
How to monitor for the results that matter most
What specifically should a campaign monitor? A campaign that commits itself to monitor outcomes would systematically – six-monthly or annually – and rigorously: (i) monitor the changes in the individuals, groups, organizations or institutions that the campaign influenced, (ii) assess the significance of these changes, and (iii) plausibly explain how and to which extent the campaign contributed through its activities and outputs, directly or indirectly, intentionally or unexpectedly. (Ricardo Wilson-Grau, personal communication).
2. Who is going to use the information, what for and when? An effective monitoring system serves the primary intended users, and uses of the data it generates. Thus, the campaign team, board members and donors will need information about major developments in the campaign to make informed decisions. What information they require, and when, will determine the degrees of certainty and precision and scientific rigor that are to be applied.
Example: The Canadian International Development Research Center (IDRC) has a policy that “at IDRC, if you cannot identify and articulate the primary intended users and uses of the evaluation you should not conduct the evaluation. Unused evaluation is a waste of precious human and financial resources.” (IDRC, 2004. Writing Terms of Reference (ToRs) for an Evaluation).
Identifying the primary intended user(s) and use(s) of a campaign’s M&E: From beginning to end, the monitoring and formative evaluation process (M&E) should be designed and carried out around the needs of the primary intended users. The users are those particular individuals or groups who intend to use the M&E process or findings to inform their decisions or actions. It is important to distinguish between the target audience and the primary intended users. An audience is a group, whether or not they are the client(s), who will or should see and may react to the findings. Thus, the audience may be interested in the monitoring and evaluation but has a more passive relationship with it than the primary intended users.
Users are actually involved in the M&E process itself – i.e. clarifying intended uses and identifying priority questions, preferred methods, and appropriate dissemination strategy. This typically results in increased use of the findings. If these individuals or groups are not included, M&E runs the risk of producing results that may never be used.
The primary intended uses of both M&E findings and process should also be clear. The findings will support decision-making or changed thinking and behaviour by the primary intended users.
The uses of the findings about the campaign can include:
- Being accountable for what has been done and achieved.
- Facilitating improvements.
- Generating knowledge.
Process uses are expected changes to result from the conduct itself of M&E. Some examples:
- Enhancing communication and shared understanding amongst campaign participants
- Supporting or reinforcing the campaign as participants learn from their involvement in the M&E. For example: ‘what gets measured gets done’.
- Increasing engagement, self-determination and ownership
- Nurturing an M&E culture
- Learning M&E thinking
- Building capacity to monitor and evaluate
Naturally, not all users will have the same uses for the process or findings.
Source: Ricardo Wilson-Grau, personal communication.
3. Where and how can the necessary information be obtained and analyzed most efficiently, if possible using existing campaign resources? Participatory monitoring involving members of the target audiences builds extra momentum for the campaign.