Outcome evaluations help determine whether or not the intended benefits of a programme are actually achieved (i.e. whether or not the programme is able to meet its intended purpose). A carefully designed evaluation with a sufficient number of participants may also answer more detailed questions that can assist with refinements of the programme’s content and format (e.g. Who benefits most from the programme? What characteristics of participants are associated with positive outcomes and drop-outs? What did the participants find most and least helpful?)
Data collected in an outcome evaluation may draw from:
- Observations during programme sessions (usually rating scales)
- Case studies, such as a narrative description of a typical woman seeking assistance upon her arrival, during the programme and after its completion.
- Assessment forms and pre/post measures of the outcome being sought.
Example: Dastak, the first private shelter in Pakistan, used semi-structured interviews to evaluate the utility of a human rights model in establishing their shelter’s guiding principles and programmes. Open ended questions solicited staff’s perceptions regarding the organization, the services provided and their challenges and successes. Interviews were then analysed for key themes, text and concepts using the qualitative methodology of constant comparison.
Dastak incorporated their knowledge of the political, cultural, legal and religious contexts that discriminate against and violate the rights of the women they serve as a basis for their shelter practice and programming using a human rights-based model. The evaluation was focused on determining the effectiveness of using such an approach. Staff were assigned specific roles to manage and deliver the programmes with assistance from the shelter’s legal section. In addition to using the legal framework, Dastak identified programme elements that were empowering (i.e. education, employment skills, counseling and therapy, social activism and legal advocacy) and were designed to outline women’s rights and capacities. It was believed that women who accessed the shelter and participated in its programmes would have greater awareness of their human rights and capacities. The outcomes being assessed included: women’s achievement of their goals (e.g. gaining custody of children); women’s sense of their ability to exercise their rights; and the ability for women to protect and act in the best interest of their own and their children’s. The evaluation demonstrated that the use of a human rights-based model was effective.
When possible, an outcome evaluation should be carried out by an external evaluator, or someone other than the agency or individual designing or providing the service, to reduce the potential bias in the assessment. It should also aim to meet the standards of national and international research and evaluation associations, including specific ethical considerations (e.g. competence, integrity and accountability) that are an essential component of evaluation.
For example, the desired outcome in a children’s programme within a shelter may be to decrease trauma-related stress among children. In this case, a measure of stress appropriate to the child’s developmental stage would be selected and completed by a parent or staff member prior to the child’s participation in the programme. The same instrument would then be completed again at programme discharge so the two scores can be compared to identify any significant change. If the programme is too brief or the family’s stay in the shelter is short, other approaches such as staff use of observational measures of stress in the child’s free or structured play may be used.
See a table of mental health and trauma-related instruments, particularly used for the assessment of children and mother-child interaction. If individual data can be protected from legal inquiry and programmes are aimed at altering levels of maternal depression or other mental health conditions, additional measures for women can be selected. Many other instruments are available online or through publishing companies such as Psychological Assessment Resources or the National Child Traumatic Stress Network, which includes a database of searchable measures for use with persons exposed to violence.