Formal quantitative surveys, taken at regular intervals, help track progress against the baseline. Surveys are most useful when there is a well-defined target audience, and the campaign strategy includes a theory of change and precise indicators that can be measured. Knowledge and behaviour can be assessed in fairly straightforward ways, while attitudes and feelings can only be observed by proxy (stakeholders’ own statements, observed behaviour etc).
Designing and administering surveys can be challenging, especially if they are not only meant to collect data on what has happened but also why. Open questions can be used to gather evidence on the causes of change and the likely contribution the campaign has made. Interviewers should be trained to administer the questionnaire in well-defined ways so as to obtain comparable data. To obtain a fuller picture of change and campaign contributions to change, direct questioning should be combined with other tools, including observation, role play or direct response tracking for example.
Practical tips for survey design
- Keep it short. Responding the questionnaire should not take more than 30 minutes.
- Keep it simple. Formulate questions in a way that is easily understandable, leaving as little room as possible for interpretation. Avoid terms that require specific knowledge.
- Keep it open. Avoid language that suggests what your opinion is or what would be the correct answer (e.g. “Don’t you think that….”).
- One concept per question. Never try to measure two different things through a single question, and make sure that possible responses match the question (e.g. Please rate the design and content of poster XY on a scale from 1 to 6, where 1 means “like it very much” and 6 means “don’t like it at all”.) Otherwise results will be unreliable.
- Start with something interesting. The first questions should raise the respondent’s interest in participating. Avoid starting with sensitive questions or questions that are difficult to answer (e.g. normative questions of the type “do you think violence against women can be justified?”).
- Pre-test the questionnaire with members of the target audience and adapt it as necessary.
The evaluation reports are available online:
Colvin, Christopher J., Report on the Impact of Sonke Gender Justice Network’s “One Man Can” Campaign in the Limpopo, Eastern Cape and Kwa-Zulu Natal Provinces, South Africa, Sonke Gender Justice Network, 2009.
Hubert, Carol, Violence against Women: It’s against all the rules, Evaluation of the NSW Statewide Campaign to Reduce Violence against Women, Violence against Women Specialist Unit, NSW Attorney General’s Department, 2002.