In 2015, DMI ran a one-year radio campaign (with eight months of broadcasting) on behaviours linked to child survival in eight provinces of the Democratic Republic of Congo. The campaign, funded by UNICEF, IMA World Health (with DFID funding) and Save the Children, aimed to reduce child mortality by promoting healthy behaviours through radio adverts broadcast on 33 community radio stations. The campaign failed to achieve its main objective (changing behaviours linked to child survival), although it did have an impact on knowledge related to most of the target behaviours.
This short blog post is an attempt to analyse and understand where we went wrong. It compares the design and implementation of this project to our Saturation+ framework for maximising the impact of media campaigns on behaviour change. (Although we initially designed the campaign using the Saturation+ framework, we watered down some of the key principles due to a combination of budgetary constraints, the differing priorities of our three funders, and the many and varied challenges of running a media campaign in DRC.)
We did we fail here when we succeeded elsewhere (for example, with our RCT in Burkina Faso)?
The pitiless conclusion: we got it wrong. The silver lining: thinking about where and why we got it wrong suggests that the assumptions behind the Saturation+ framework are correct, and in particular two key elements of saturation (frequency and duration of broadcasting):
We recommend broadcasting radio spots 8-12 times per day.
We worked through an organisation that has a network of around 100 radio station partners across DRC, and that distributed our spots and managed those stations on our behalf. Our qualitative and quantitative research (including figures provided by people we paid to listen to the radio stations every day on our behalf) suggest that many of our 33 radio stations were broadcasting far fewer spots per day, perhaps 4 on average.
We are still working out how best to get around this operational problem in countries where you need to broadcast on a large number of radio stations to reach enough of your target audience, which makes it impractical to build and manage relationships with them all directly. And there are sometimes cases where the media environment is so fragmented that it makes more sense to broadcast a smaller number of spots on a larger number of radio (or TV) stations, rather than allocating most or all of your airtime budget to one or two stations; so our recommended frequency of 8-12 spots per day is not a hard and fast rule.
We recommend using mathematical modelling to estimate the impact of each message, and then allocating airtime to each message based primarily on predicted impact.
We did this in DRC, but we did not broadcast for long enough on each message to have the desired impact on the targeted behaviours. Our Burkina Faso RCT suggests that you need to broadcast for around 10 weeks (10 times per day, every day, but spread across a larger number of weeks) to change a particular behaviour by around 9%. In DRC we broadcast 18 distinct messages on 13 health topics, spread over 35 weeks, so only averaged two weeks per message (at 4-6 times per day, on average).
In future we will be much clearer with funders about the number of messages that they can expect to cover, based on the number of weeks of campaigning that they are prepared to pay for, to ensure that they understand that including too many messages in a short campaign will dramatically reduce impact on behaviours.