Many shelters are able to strike a balance between supporting and empowering women and encouraging them to participate in evaluation and monitoring by developing a culture of accountability and knowledge-based service delivery. This involves ongoing staff training, presentations of evaluation/monitoring information to staff and frequent discussion of what the information means for shelter operations, clients and in general.
Monitoring and evaluation readiness checklist
Is there awareness and buy-in from community and staff?
Have staff members who will be maintaining the monitoring programme been included in its development? Trained in the use of the system?
Does the initiative require an Advisory Committee or a Steering Committee? If so, have the Committee’s mandate and Terms of Reference been defined? Has membership on the committee been reviewed to ensure that specialized expertise which may be required is available (e.g. funding application expertise, project design, data analysis)?
Is the programme framework complete? Are inputs, outputs and outcomes/indicators clearly defined and measureable? Have staff questions been addressed in the framework? Has the data collection plan been reviewed to ensure that information responsive to each outcome of interest will be collected?
Have existing monitoring and measurement tools been reviewed and revised as necessary (intake and exit forms, database)?
Have any other measures been obtained and reviewed? Has staff received required training in their use?
Are adequate resources in place to enable effective monitoring and evaluation? For example, confirm the presence of:
- Staffing resources
- Equipment required for the project
- Effective safety procedures to secure data and/or to ensure safety of participants
- Methods and tools sensitive to residents’ different cultural traditions, expectations and language preferences, with adequate staff capacity to use these tools?
Have participant consent forms been developed and reviewed by the Advisory Committee and/or by shelter management to ensure they meet all ethical standards and information management requirements? If individually identifiable data is being collected, it is always necessary to ensure that women understand what their involvement in monitoring and evaluation will mean and what it will require of them. See sample consent forms are available on most national evaluation societies’ websites and on government research websites.
Canada Evaluation - www.canadaevaluation.ca or www.sshrc-crsh.gc.ca
Puttinging Women First: Ethical and Safety Recommendations for Research on Domestic Violence against Women (WHO, 2001). Available in English, French and Spanish.
Ethical and Safety Recommendations for Interviewing Trafficked Women (WHO, 2003). Available in Armenian, Bosnian, Croatian, English, Japanese, Romanian, Russian, Serbian and Spanish.
Researching Violence against Women: A Practical Guide for Researchers and Activists (Path 2005). Available in English and Spanish.
- Once the monitoring system or evaluation of a project is underway, it is essential that supervisors conduct reviews with staff to ensure that the process is operating effectively. This may be started immediately after activities are tracked to provide clarification on the process, identify any problems staff are encountering and to institute necessary revisions. This is critical to prevent data errors from accumulating, such as data entry system may be found to include overlapping variable descriptions that require rewriting to ensure data accuracy or staff questions about the intake or exit interview. Early support is a valuable investment as data problems are easier to correct in the early stages of programming.
- Clear guidelines should be provided so staff members who are responsible for either written or database entries are aware of and comply with timelines for data entry and accuracy requirements. A hard copy data book can be useful and should include specific instructions on data requirements, variable descriptions and definitions and contact information for support with the process.
- Supervisory staff should conduct routine audits of written and computer-managed data at least monthly to ensure that data entry is complete and accurate. Without such processes, it is easy to overlook missing or inaccurate data until analysis is ready to begin. It is much more difficult to correct problems at the analysis stage, particularly if there is high turnover of staff or if it has been some time since the project began. Locating missing data in paper files can be time-consuming and frustrating if files are disorganized and/or incomplete.
- Data entry occurs in three basic phases:
- When survivors enter the shelter/programme, including through intake information and any pre-tests with standardized tools
- During the survivors participation, including tracking of services provided, observations of any intervention being evaluated, referrals made, etc.
- Upon departure of the survivor, including post-test information, exit survey or interview data and data from any follow-up phase
EXAMPLE: A non-governmental shelter programme for trafficking victims was evaluated for its ability to reintegrate trafficked girls back into their Nepalese communities. The evaluation was able to show that the organization achieved a significantly high rate of reintegration success despite the stigma the Nepalese community often attached to trafficked girls.
The evaluation first compiled participant characteristics such as the girls’ ages and length of time spent in forced prostitution. It then tracked programme inputs like staff and funding. The evaluation clearly defined the intended beneficiaries and articulated the feasibility of delivering the programme, including by outlining the activities, such as providing girls with training for income-generation initiatives that were relevant to the Nepalese economy and coordinating family visits.
The outcomes or expected changes from these interventions were identified as:
a) girls are reintegrated into their family after accessing the shelter
b) girls are employed in income-generating activities
c) girls are in satisfactory health
The indicators used to determine whether the expected changes occurred included:
- percentage of girls who reported that their family members are able to accept them back into their family
- percentage of girls who reported experiencing satisfactory adjustment
- percentage of girls employed in income-generating activities
- percentage of girls who reported satisfactory health.
A standardized paper form was created for tracking and follow-up over a three month to five year period. These were used to evaluate the reintegration programme and its outcomes. The evaluation provided evidence that the re-integration programme of the non-governmental organization shelter was successful in re-integrating girls back into their families and communities.