Community Mobilization

Brief overview of monitoring and evaluation of community mobilization initiatives

• In recent years, there has been increased recognition that community-based initiatives to prevent violence against women are key to reducing overall levels of violence in a society.  Implementing laws and providing safe haven and services for survivors of violence, while vitally important, do not address the underlying causes of violence against women and girls, which have a lot more to do with changing the way that men and women interact in their communities, and how they view issues such as balancing power between men and women.
• While a number of groups have developed innovative ways of mobilizing communities to address prevailing norms and behaviours around violence and gender, evidence around the effectiveness of strategies used by these programmes remains sparse.
• Few groups have the resources or capacity to carry out effective monitoring and evaluation of their programmes, and there are not many validated models for measuring social change.  Social change typically takes place over many years, involves many different actors, and is difficult to measure.  For the same reasons it is difficult to determine “attribution,” or the degree to which a change can be credited to a specific intervention.

Key lessons learned about community-level evaluations

The lessons below come from the experience of evaluating Stepping Stones in South Africa and are generally applicable to most community-level evaluations.

• Communities will support a research project of this nature if they feel it is addressing an issue they are concerned about and will be of value to them. Researchers must invest time in explaining the proposed study
• Building a strong Community Advisory Board with a range of stakeholders can be of great assistance.
• Community mobilization should be viewed as an ongoing process and not a once-off task.
• Resources for community mobilization should be adequate for the entire duration of the planned project, and allocated independently of fieldwork.
• Local community politics may be complicated, and there may be many different interest groups with differing concerns and priorities with respect to the research. Establishing good relations with as many groups as possible takes time and is very important for success of the project.
• Informed consent should be seen as a process to maximize freedom of choice around research participation and levels of commitment to the study.
• Instruments, which have been validated in other settings, must have their validity established in the local settings. Cross-cultural validity should not be assumed.
• Involving field workers in validating and translating instruments greatly increases their depth of understanding of the instrument and resultant data quality.
• Research projects of this nature in rural areas often employ many staff who have not worked in the formal sector before. Extra time needs to be provided for problem solving and team building with inexperienced staff.
• Training on research, monitoring and evaluation related to the project should be ongoing throughout the project including for fieldworkers and intervention facilitators. See the SASA! guidance for facilitators developed by Raising Voices.
• Rural youth may be highly mobile, especially if they have to travel for school. Arrangements for follow up should include ascertaining when youth will return home for interviews, using peers or other community members to help the team determine when young people return.

Indicators

MEASURE Evaluation, at the request of The United States Agency for International Development and in collaboration with the Inter-agency Gender Working Group, compiled a set of indicators for community mobilization. The indicators have been designed to measure programme performance and achievement at the community, regional and national levels using quantitative methods. Note, that while many of the indicators have been used in the field, they have not necessarily been tested in multiple settings. To review the indicators comprehensively, including their definitions; the tool that should be used and instructions on how to go about it, see the publication Violence Against Women and Girls: A Compendium of Monitoring and Evaluation Indicators.

The compiled indicators for community mobilization are:

• Proportion of individuals who know any of the legal rights of women

What It Measures: Knowledge of women’s constitutional and legal rights remains low particularly among women in many countries. Given the situation, human rights and women’s rights education programs are implemented in communities in a number of countries. Program managers and evaluators may be interested in knowing the extent to which men and women are aware of the constitutional and legal rights of women. This outcome indicator measures the extent to which the public are aware of such rights.

• Proportion of individuals who  know any of the legal sanctions for violence against women and girls

What It Measures: Knowledge of the legal sanctions associated with VAW/G is low, particularly among women in many countries. If women do not know of any legal recourse, they may not seek help, thinking that nothing can be done. Program managers and evaluators may be interested in knowing the extent to which men and women are aware of the legal sanctions for acts of VAW/G that exist in a country. This outcome indicator measures the extent to which the public are aware of such sanctions.

• Proportion of people who have been exposed to violence prevention messages

What It Measures: In countries (or regions within countries) where communication programs related to VAW/G prevention are implemented using IEC techniques, interpersonal communication channels and community outreach workers, program managers and evaluators may need to know the extent to which the intended audience is exposed to the communication programs. This outcome indicator measures the extent to which a population targeted by specific VAW/G programs and projects is exposed to VAW/G prevention messages through any means that they might be communicated.

• Proportion of people who say that wife beating is an acceptable way for husbands to discipline their wives

What It Measures: This outcome indicator measures the level of acceptability of wife-beating in an area (region, country, community) for any reason, at the point in time that it is measured. A high proportion would indicate that most people in the targeted population feel that wife beating is acceptable under certain conditions.

• Proportion of people who would assist a woman being beaten by her husband or partner

What It Measures: Neighbors are often aware when a woman is being beaten by her husband because they can hear the incidents. Relatives (living outside the household) and friends are often told about these incidents by women. People’s willingness to extend help to the woman may save the woman from the pain, disability and death associated with IPV. A low proportion of individuals stating that they are wiling to help may indicate a general acceptance of IPV in the community surveyed. Women living in such communities are at higher risk for the consequences associated with IPV, and they may have little recourse.

• Proportion of people who say that men cannot be held responsible for controlling their sexual behavior

What It Measures: People who feel that men cannot control their sexual behavior (i.e., that the reason they act sexually is because of a factor outside of themselves) also feel that on some level, men cannot be held responsible for what they do sexually. This places the blame for violent sexual behavior on the victim instead of on the perpetrator.

• Proportion of people who agree that a woman has a right to refuse sex

What It Measures: Cultural norms around the authority of husbands over wives may include that he has a right to have sex with his wife or partner regardless of whether or not she wants it. Alternately, there may be conditions under which it is acceptable (or not acceptable) for her to refuse. This outcome indicator measures how acceptable the idea of a woman refusing sex with her husband or partner is, and under which reasons people feel this is acceptable. In areas where there are no acceptable reasons or very low proportions of people agreeing with the reasons, women’s power of sexual negotiation may be very low and could leave them at risk for violence and exposure to sexually transmitted infections including HIV.

• Proportion of people who agree that rape can take place between a man and  woman who are married

What It Measures: Cultural norms around the authority of husbands over wives may include forced sexual intercourse. Marital rape may be more likely to occur in places where both women and men ascribe to this belief. This outcome indicator measures how unacceptable marital rape is in the targeted population. In areas where the proportion is very low, women may be at high risk for marital rape.

• Proportion of target audience who has been exposed to communication messages recommending the discontinuation of female genital cutting/mutilation (FGC/M)

What It Measures: In countries (or regions within countries) where communication programs related to the elimination of FGC/M are implemented using mass media, program managers and evaluators may need to know the extent to which the intended audience is exposed to the communication messages. This outcome indicator measures the extent to which the public (or population targeted by specific programs and projects) remembers seeing or hearing FGC/M elimination messages through various communication channels after exposure.

• Proportion of people who believe that FGC/M should be stopped

What It Measures: The practice of FGC/M is deep-rooted in culture, and is supported by beliefs that are in favor of the practice. International organizations, however, recognize the practice as a violation of international standards for girls’ and women’s rights, and that the practice has serious health consequences. Communication campaigns and community education programs are implemented in countries where the practice is prevalent to discourage support for FGC/M. This outcome indicator measures the level of public acceptance of FGC/M within a given population.

• Proportion of women who do not intend to have any of their daughters undergo FGC/M

What It Measures: The practice of FGC/M is deep-rooted in culture, and is supported by beliefs that are in favor of the practice. International organizations, however, recognize the practice as a violation of international standards for girls’ and women’s rights, and that the practice has serious health consequences. Programs that aim to eliminate the practice of FGC/M will want a measure of women’s intentions regarding their own daughters. This outcome indicator provides a measure of the effectiveness of programs and initiatives that aim to reduce the practice of FGC/M.

• Proportion of people who believe child marriage should be stopped

What It Measures: The practice of child marriage is deep-rooted in culture and is supported by beliefs and customs transmitted through the generations. International organizations, however, recognize the practice as a violation of international standards for girls’ and women’s rights, and that the practice can result in serious emotional and physical health consequences. Communication campaigns and community education programs are implemented in countries where the practice is prevalent to discourage support for child marriage. This outcome indicator measures the level of public acceptance of child marriage within a given population.

• Proportion of women who do not intend to marry their daughters before the age of 18

What It Measures: The practice of child marriage is deep-rooted in culture and is supported by beliefs and customs transmitted through the generations. International organizations, however, recognize the practice as a violation of international standards for girls’ and women’s rights, and that the practice can result in serious emotional and physical health consequences. Communication campaigns and community education programs are implemented in countries where the practice is prevalent to discourage support for child marriage. This outcome indicator provides a measure of the effectiveness of programs and initiatives that aim to reduce the practice of child marriage.

For additional indicators on working with youth and working with men and boys, see the full compendium.

Programme descriptions and indicators for select community-based initiatives:

Monitoring and evaluation methods (including indicators) will vary depending on the programme’s goal and objectives.  The descriptions and indicators presented below are examples that come from some of the most promising community-based initiatives that have demonstrated promise in reducing violence against women and girls.

The Rural AIDS Development Action Research (RADAR) Programme Intervention with Microfinance for AIDS and Gender Equity in South Africa (Pronyk et al 2006)

The programme Intervention with Microfinance for AIDS and Gender Equity (IMAGE) study in South Africa-- one of the few rigorous evaluations of community-level interventions globally related to gender-based violence-- showed that an intervention combining micro-financing to women with education and mobilization around violence reduced women’s experiences of violence by 55% over a two-year period.

The IMAGE study used a cluster randomized trial design to test the hypothesis that combining a microfinance-based poverty alleviation programme with participatory training on HIV risk and prevention, gender norms, domestic violence, and sexuality can improve economic well-being, empower women, and lead to reductions in inter-personal violence. The study was conducted between September 2001 and March 2005 in South Africa’s rural Limpopo province.

Outcome measures included past year’s experiences of intimate-partner violence drawn from the World Health Organization’s Violence against Women study instrument and nine indicators of women’s empowerment measured around three domains: 1) power within; 2) power to; and 3) power with.  The nine indicators they used to measure these three areas of empowerment were:

• Power within – self confidence, financial confidence and challenges gender norms.
• Power to –  autonomy in decision-making, perceived contribution to household, household communication and partner relationship
• Power with – social group membership and collective action

In addition to the main outcome of reduced intimate partner violence, two other outcome indicators were measured because of their correlation to experiences of violence:

• Past year experience of controlling behavior.
• Progressive attitudes towards intimate partner violence.

Qualitative data about changes occurring within intimate-relationships and the community were also collected through facilitated focus group discussions, contributing significantly to understanding the nuances behind the numbers. For example, discussions revealed that there was no equivalent for the term “empowerment.” Rather, women used phrases such as “the ability to claim personal power and use it to change for the better,” noting that “you can have money and still not be empowered.” Qualitative responses also highlighted that reductions in violence resulted from a range of responses enabling women to challenge the acceptability of such violence, expect and receive better treatment from partners, leave violent relationships, give material and moral support to those experiencing abuse, mobilize new and existing community groups and raise awareness about gender-based violence and HIV. [Pronyk et al 2006]

To view the IMAGE evaluation methodology, intervention materials, questionnaires (baseline and follow-up) for young people, women and households, and research reports visit the University of Witwatersrand, Johannesburg, Intervention with Microfinance for AIDS & Gender Equity website.

Gender-equitable men (GEM) scale to assess norms and behaviours among young men in Brazil and India

Few interventions to promote gender-equitable norms and behaviours among young men have been systematically implemented or evaluated, and relatively little is known about how best to measure changes in gender norms, violence and their effect on HIV/STI protective and risk behaviours.

To address these gaps, the Horizons Programme and Instituto Promundo through their Programme H examined the effectiveness of interventions designed to improve young men’s attitudes toward gender-equitable norms, including gender-based violence, and to reduce HIV/STI risk.

GEM Evaluation Brazil: Set in Rio de Janeiro in 2001, the study compared the impact of different combinations of programme activities implemented in three different but fairly homogeneous low-income communities. One intervention was interactive group education sessions for young men led by adult male facilitators. The other was a community-wide campaign to promote condom use, using gender-equitable messages that reinforced those promoted in the group education sessions. A group of young men aged 14 to 25 was followed over time in each community.

To assess the impact of the programme, researchers from the Horizons Programme developed and validated the Gender-Equitable Men (GEM) Scale with a representative household sample. The GEM Scale, which measures attitudes toward gender norms, includes items in five keys areas: (1) violence, (2) sexuality and sexual relationships, (3) reproductive health and disease prevention, (4) domestic chores and childcare, and (5) homophobia and relationships with other men. Twenty-four (24) items were selected, 17 of which comprise the “inequitable” gender norms subscale, addressing norms that have been considered more “traditional,” and 7 which comprise the “equitable” subscale. Responses were categorized and scored as “least equitable,” “moderately equitable,” and “most equitable,” and collated for an overall score.

Changes in attitudes over time were also analyzed in order to correlate them with other outcomes. A variable was created to reflect changes in respondents’ GEM scores. If a respondent’s score increased after the intervention, it was reflected as a positive change, if it stayed the same or decreased, it was reflected as no or negative change. (Pulerwitz, Barker, Segundo and Nascimento. 2006.)

See the evaluation summary, available in English.

See the GEM scale in English, Spanish and Portuguese.  The scale used in Ethiopia is also available in English.

GEM Evaluation India: In India, the Horizons Programme, CORO for Literacy, MAMTA, and Instituto Promundo developed and piloted a behaviour-change intervention, Yaari-Dosti, based on Programme H implemented in Brazil.  The team conducted research to examine the effectiveness of the interventions to improve young men’s attitudes toward gender roles and sexual relationships, and to reduce HIV risk behaviours and partner violence. Set in urban areas of Mumbai, the study tested the impact of different combinations of intervention activities on young men’s support for inequitable gender norms, HIV/STI risk behaviours, and partner violence.

Attitudes toward gender norms of the young men were assessed using a version of the Gender Equitable Men (GEM) Scale adapted to the Indian context with the addition of specific items, for example, “A man is happily married only if his wife brings a big dowry,” and, “A woman should not need to ask her husband for permission to visit her parents/family.” (Verma et al 2008)

See the evaluation summary, available in English.

International Men and Gender Equality Survey (IMAGES)

Instituto Promundo and the International Center on Research and Women developed the International Men and Gender Equality Survey (IMAGES) to strengthen understanding of men’s behaviours and attitudes, and changes in those attitudes and behaviours. The ultimate goal of the survey is to inform, drive and monitor policy development to promote gender equality by engaging men and women in such policies.

IMAGES is one of the most comprehensive surveys developed on this topic and has sought to incorporate the most recent instruments on gender, quality of life, childhood antecedents of violence, health indicators, gender-based violence, family gender dynamics, and fatherhood. In some countries, IMAGES is serving as a baseline instrument to assess men’s and women’s attitudes and behaviours on these key issues before the implementation of major policy and programme initiatives.

The specific objectives of the survey are to:

• Assess the current behaviours and attitudes of men on a wide range of issues as they relate to gender equality, including fatherhood and caregiving (time spent in providing care for children, and others in the home), use of violence in intimate and sexual relationships (sexual, physical and psychological), work-life balance, use of health services, negotiation/ communication with partner about family size and sexual relations, among others;
• Compare these results with women’s attitudes and behaviours on the same issues;
• Assess men’s knowledge of and attitudes toward about policies that have sought to promote gender equality (e.g. employment and political participation quotas for women, women’s economic empowerment, paternity establishment, gender-based violence, among others), and explore to the extent possible men’s ideas about what policies and changes in existing policies they think would help them become even more involved in gender equality; and,
• Explore factors that may explain variation in men’s behaviours in their family lives and intimate and sexual relationships, including childhood experiences of violence, gender norms in family of origin, stress, migration, and unemployment, and others.

The IMAGES Survey for men is available in English and Portuguese.

The IMAGES Survey for women is available in English and Portuguese.

Additional Resources for Evaluation:

Measures for the Assessment of Dimensions of Violence against Women: A Compendium (Flood, 2008). Available in English.

Somos Diferentes, Somos Iguales, Sexto Sentido (Puntos de Encuentro). The evaluation report is available in English and Spanish.

Soul City 4 Evaluation Media Campaign Monitoring and Evaluation Materials. Available in English.

SASA! Tips Booklet - Strengthening Your Activism: Skills for Preventing Violence against Women and HIV Infection (Raising Voices, 2009). Available in English.

Good Schools Toolkit (Raising Voices, 2009). Available in English.

Incorporating Evaluation into Media Campaign Design (Institute for Health and Development Communication, South Africa).  Available in English.

Lessons in Evaluating Communications Campaigns: Five Case Studies (Communications Consortium Media Center/ Harvard Family Research Project, 2003). Available in English.