How do we prove impact?

DMI and LSHTM are running a five-year cluster-randomised controlled trial in Burkina Faso, funded by the Wellcome Trust and Planet Wheeler Foundation, to test the predictions of our model in a real-life setting. The trial uses radio broadcasts to change behaviours in order to improve child survival, covering a range of health issues, and is the largest, most rigorous evaluation ever conducted of a mass media intervention. It will generate a substantial body of further evidence for the impact of mass media campaigns on a range of behaviours, including those linked to demand for healthcare services, and on child mortality.

The trial involves fourteen geographical areas that were randomised and equally divided into an intervention group and a control group. Messages are being broadcast for 2.5 years in the seven geographical areas of the intervention group, but not in the control group. 

A controlled trial using radio would not normally work, due to the risk that people in 'control' areas would listen to radio stations broadcasting from 'intervention' areas. However, Burkina Faso has a very localised, radio-dominated media environment, so we can use local FM radio stations to broadcast our messages to intervention areas (blue) without 'leaking' into control areas (red).

For 30 months we are broadcasting 60-second advertisements at least 10 times per day on seven radio stations (one in each intervention zone), in six languages. In addition, we are broadcasting two hours per night, five nights per week on each station. This represents a total of 70 hours per week of live radio.

This would be logistically almost impossible to do using a soap opera format, for example, given the six languages involved. We needed to devise a format that is cheap, that can be broadcast daily, that can be produced ‘live’ (which costs a fraction of pre-produced radio), and yet can be controlled centrally. We have created a system of self-contained drama modules that are written in French in the capital city, emailed to our partner radio stations, and improvised live by actors on location in their own language within their two-hour shows. This works well in a fragmented media environment, which is becoming the norm in most developing countries.

We are not paying airtime fees to our partner radio stations in Burkina Faso; instead, we provide on-the-job training and produce the live programmes together. The stations’ incentive is that they thrive in a competitive media market.

We implemented a system of grassroots recruitment, leafleting university campuses, meeting places and bars, inside and outside the capital. We invited interested people to public meetings and challenged them to write a script. We received over 600 scripts, interviewed 80 people and hired 13 as scriptwriters – who came from a diverse range of previous employment, ranging from teachers to a security guard. Every week we conduct creative training workshops for all staff. We have an in-house research team which conducts monthly formative, pretesting and feedback research, with one scriptwriter participating in every research trip.

Example radio outputs from the trial

Malaria radio spot Breastfeeding radio spot
Breastfeeding radio spot 2 Hygiene radio drama

 

The evaluation, led by LSHTM and Centre Muraz, includes a baseline mortality survey of 50,000 children under 5 years (with a two-year recall period), and a baseline behaviour survey of 5,000 households, before radio broadcasts began in March 2012. Broadcasts will stop in January 2015, and full results, based on an endline mortality survey of 100,000 children (with a one-year recall period), will be available in mid- to late-2015.

We have recently published our midline results on behaviour change (based on a survey of 5,000 households).

In addition, an economic evaluation, led by Jo Borghi and Anne Mills at LSHTM, will assess whether the intervention represents 'value for money', measuring the cost per child death averted and per DALY averted. We will also model the likely cost-effectiveness of implementing the campaign at national scale in Burkina Faso and elsewhere in Africa.

If similar five-year campaigns are implemented in 10 African countries, the model predicts that one million lives should be saved. This process would, for the first time, place mass media campaigns in the mainstream of public health interventions, based on robust scientific evidence. We are currently designing health behaviour change campaigns in these countries and are looking for partners and funders through our Media Million Lives initiative.