Shelters should plan for monitoring and evaluation throughout all stages of their programming, covering the scope of activities and intended results as well as the various approaches that will be used to assess the services provided. This may include:
- Baseline assessments in the planning phase.
- Process evaluations, which monitor the implementation of services and describe the process of developing and implementing a particular initiative.
- Outcome or (less frequently) impact evaluations, which help determine whether or not the intended benefits of a programme are actually achieved by the end of or in reviewing an intervention.
Challenges to effective monitoring and evaluation of shelter services are similar to the general issues of monitoring violence against women programmes, which relate to collection of sensitive information from women and girls. Specific challenges unique to shelter initiatives include:
- Multiple demands on survivors. Since women access shelters at a time of crisis, much of the shelter’s work is focused on supporting them to address the crisis and meet their basic physical, emotional and practical needs before they leave the shelter (e.g. security, housing, employment and other family supports). Given their multiple priorities, it is difficult and at times, inapporpriate to ask women to complete specific monitoring and evaluative questionnaires or feedback forms.
- Information sharing. There are a number of challenges, barriers and professional practice issues associated with ensuring that women’s privacy is respected and that professional ethics and legal requirements for the protection of personal information are followed. Examples include:
- Women may worry about the implications and possible consequences of sharing information about themselves; may be reluctant to talk about details of the abuse they or their children have experienced; mental health concerns, or substance abuse history, since these disclosures may affect their custody rights or expose them to possible intervention from child welfare or other authorities.
- In cases of domestic violence, information disclosed to shelter staff may be summoned for use in legal proceedings and may be used against the survivor’s case.
- In some locations, there are limits on what information can be provided to shelters from other services such as police or social service agencies, which may limit the ability of shelters to access information related to women’s circumstances and well-being.
- Certain professional organizations require staff to follow strict confidentiality and privacy codes that limit information which can be requested or provided by shelters.
- Length of stay in shelters. Many women and children remain in shelter for very brief periods (often less than a week in emergency shelters), which prevents their full participation in shelter programmes and limits the feasibility of engaging them in medium and long-term evaluations assessing shelter outcomes. Many shelters provide outreach programming (staff offering programmes to women in their homes or clients attending programmes at the shelter after their departure), which can improve the likelihood of useful evaluations being completed, but often require additional resources and may present unique challenges related to the length of sustainable follow-up, staff attrition and participant attrition.
- Multiple shelter episodes. Women may use shelters and related services multiple times before making changes in their lives or achieving the goals they have set for themselves. It is not always possible, due to confidentiality and information sharing limitations, to track women’s use of services from one shelter to another, or even from one programme to another within a shelter. As a result, there may be information for single interventions, but little or no data on the cumulative impact of the overall shelter stay for the woman.
- Culturally-sensitive services and staff cultural competence. Women using shelter services come from many diverse backgrounds and cultures, but there are few assessment instruments which incorporate cultural considerations. This is linked to broader issues related to enuring services meet the diversity and accessibility needs of all women and girls.
Because the safety and empowerment of women in shelter is of paramount importance, the amount, timing and method of information collection should follow ethical guidelines and prioritize, respect and be sensitive to the needs of women and children in the shelter.
See general guidance in the module on Monitoring and Evaluation for violence against women programmes.
Engaging relevant stakeholders
Stakeholder involvement is a key component in getting started and shelters should ensure that the planning process for the development and introduction of monitoring and/or evaluation is well-organized, transparent and effective in engaging shelter staff, women using services and other programming partners, such as representatives of other shelters, funders, other services (e.g. police, other civil society groups, state agencies), among others. The monitoring and evaluation processes depend on the willingness of staff to undertake any new process in a committed and consistent way and on the willingness of the community and related agencies to affirm the project is a worthwhile use of community resources. Failure to invest in the engagement process is likely to result in weak or ineffective monitoring systems or evaluation processes.
Different approaches can be used with different groups and should be tailored to their specific role within the initiative.
Shelters should seek staff input throughout the process, beginning with the definition and development of the framework and preparation of funding applications, if required. This ensures that staff are invested in the success of a monitoring system or evaluation and understand its eventual benefits to themselves and the women they are dedicated to helping. Specifically, such commitment is critical to the completion of tasks such as data collection and data entry, which may be perceived as an additional burden to already overworked shelter staff.
The establishment of a small steering or advisory committee can be helpful to oversee project implementation, assist in the resolution of difficulties, advocate for additional programme support and resources and assist with the dissemination of results to externally promote the programme. These committees may be led by the organization’s administrator responsible for implementing the project, with members selected on the basis of expertise or influence they can contribute. For example, if a shelter does not have staff with expertise in project design or data analysis, community experts may be invited by senior project staff to participate. They may also provide specific assistance as needed, such as with ethics and information sharing issues. Committees are usually comprised of six to eight members and generally receive quarterly progress reports from staff, with meetings are held at least quarterly, either in-person, or through teleconferencing or web-based tools.
Among staff implementing the programme, it is helpful to have a small internal design and implementation committee assigned to lead the project and report to the Advisory or Steering Committee and to internal management. Such a team may be responsible for creating initial drafts of revised or new documents, drafting the initial project design framework, undertaking literature searches or research as relevant and completing other required tasks to move the project forward (including those related to staff training or additional needs identified by the Advisory or Steering Committee).