Measuring impact

We have developed a particular expertise in being able to measure the health impacts of our media campaigns, in addition to the behaviour change outcomes. We have worked with the London School of Hygiene & Tropical Medicine to develop a mathematical model, based on wide-ranging and robust evidence, which predicts that we can save one million lives by running campaigns in 10 countries over several years.

Challenges in measuring the impact of media campaigns

The quality and size of the evidence base for the impact of mass media on health behaviours has been limited by the quality of evaluation designs in many cases. In particular:

  • Evaluations are often limited to indicators focusing on reported or intended, rather than observed, behaviour
  • Many evaluations collect data on trends in health behaviours, but few have looked at impact on health outcomes
  • As a result, it is often difficult to attribute behaviour change outcomes to a mass media campaign.

Nonetheless, a 2001 Cochrane review concluded that “despite the limited information about key aspects of mass media interventions and the poor quality of available primary research, there is evidence that these channels of communication may have an important role in influencing the use of health care interventions.”


Measuring impact on mortality reduction

Campaigns run by DMI staff have used progressively more robust evaluation designs to measure impact on behaviour change and health outcomes. However, there has never been any attempt to measure or even to model the impact of campaigns on mortality. Historically this has been because the impact on all-cause mortality would be very hard to identify statistically due to the many possible causes of deaths and the limited impact of any single campaign.

We have conducted ground-breaking research, in partnership with the London School of Hygiene and Tropical Medicine. This research calculates how many children's lives could be saved if campaigns were conducted in a new, far more cost-effective manner.

To estimate the potential impact of comprehensive campaigns, we have adapted and extensively updated the models used for the Lancet Child Survival Series in 2003 and 2005. These models predict how many lives could be saved if coverage of key interventions (such as breastfeeding and bed nets) was increased from current levels. Our own model uses evidence from previous behaviour change campaigns to predict the increase in coverage of these interventions that could be achieved by campaigning on all causes of death using mass media alone.

The predictions of the DMI-LSHTM mathematical model are very promising. They indicate that child mortality in most low-income countries could be reduced by 16% to 23%, depending on the profile of the country (see predictions by country). The cost per life-year saved is also lower than any currently available intervention. The research was published in The Lancet in February 2015.

The majority of our impact on child mortality is focused on the big three killers of children under five: malaria, pneumonia and diarrhoea (with malnutrition as an important secondary cause). This pie chart gives an estimated breakdown of our impact by cause of death (mouse over the segments to find out more).

We are currently running a randomised controlled trial in Burkina Faso to test these predictions on the ground.



Film: The Science

What is the science that underpins DMI's work? How many lives can we actually save? What is a DALY? Professor Jimmy Whitworth (Wellcome Trust), Dr Richard Horton (The Lancet) and Roy Head (DMI) present the key concepts.