When does advocacy work? How do we know? Despite advocacy and campaigning being the go-to methods for influencing and spreading information and affecting behaviour change, understanding whether it actually works, has received little attention.
To address these questions and other emerging trends in the world of impact evaluations, the Water Supply and Sanitation Collaborative Council (WSSCC) in partnership with 3ie convened a meeting on Measuring the Impact of Advocacy at the Palais des Nations, Geneva on 5 April 2016. The event was extremely well attended, with close to 100 experts on evaluation; WSSCC Steering Committee Members; and professionals from the United Nations community.
Dr. Jyotsna Puri, deputy executive director and head of evaluation, 3ie, made a presentation on the possible approaches for measuring the impact of advocacy while Archana Patkar, programme manager, Networking and Knowledge Management, WSSCC, presented the practitioner’s point of view on the challenges of evaluating advocacy approaches.
In opening remarks, Ms. Amina J. Mohammed, the Chair of WSSCC, talked about the huge amount of advocacy that went into the Sustainable Development Goals (SDGs), and stressed the value of evaluations to ensure accountability. “The role of evidence is still very clear, but we’re still very weak in producing the data,” said Mohammed. “We need data to inform what we do and reinforce what we do at the country level.”
Dr. Puri highlighted the complexity of evaluating advocacy campaigns and discussed how to illustrate evidence and standards of evidence in behaviour change advocacy. “Implementation and effectiveness is hard to measure, but let’s measure what we treasure,” Puri said. She also facilitated a discussion on the gaps in rigorous tools when it comes to evaluating policy advocacy.
In her remarks, Archana Patkar stressed that advocacy is at the root of behaviour change, and also recounted WSSCC’s long history with achieving significant advocacy outcomes in the WASH sector, saying that “we need to be able to measure and replicate what is effective because resources are limited.”
The main objectives of the meeting were to discuss how advocacy can be evaluated; what tools and methods are most appropriate or promising for assessing the impact of different advocacy approaches (and where the methodological gaps are); and how to create multi-disciplinary evaluation frameworks for measuring the impact of advocacy work.
3ie’s evidence reviews reveal that the majority of rigorously evaluated advocacy initiatives to change behaviour or attitudes are implemented through information campaigns. These are commonly with goals of changing behaviours, attitudes, capacity building, affecting budget or participatory involvement and have been executed in a wide range of sectors from health to politics and are easily evaluated. For example, in the field of water and sanitation, there are studies capable of employing experimental methods to answer which advocacy approaches are effective, for instance in improving take up of chlorine dispensers or hand washing.
On the other hand, non-experimental approached in these areas raise questions of data quality and causal linkages. When looking at the bigger picture, the eventual aim often is to affect policy. In policy advocacy, the long-term, dynamic, and often shifting path from inputs to outcomes and impacts and single observation nature is one that often requires alternative evaluation methodologies like process tracing methods.
The summary of the Measuring the Impact of Advocacy meeting can be found here.
The JMP SDG baseline findings set a clear agenda on the work to be done towards the shared vision of WASH for All
Countries are not increasing spending fast enough to meet the water and sanitation targets under the SDGs says the 2017 GLAAS report
For SWA’s HLMs 2017 Chris Williams argues that governments should be investing in disease prevention