The Responsibility to Report Results — Health and Communications

Communicating results is a key responsibility of NGOs in global health –and other fields, too.

Those who invest their time and money deserve a proper accounting.
Those who evaluate, need access to information.
Those who manage should do so from a foundation of facts.

That responsibility demands that the reporting of results be meaningful, comparable, faithful, rigorous, and transparent:

Meaningful
Too often what’s reported is activity, rather than progress. Sending a team, say, so many times to a remote village, is a measure of activity and an indicator of focus or purpose, but it does not directly chart progress.

Comparable
Edward Tufte said it best: “At the heart of quantitative reasoning is a single question: Compared to what?” [ 1 ] If you’re reporting, for example, X-number of vaccinations you also need give that figure context by comparing it to other figures (e.g., across time periods or regions, or against target goals.)

Faithful
In reporting statistics, don’t take advantage of readers’ assumptions. If you fed one person last month and two this month, it’s somewhat disingenuous only to report a 100% increase in the number of meals served because readers would naturally have assumed that you would have been helping a far greater number.

Rigorous
Reported results can’t be more precise than the figures used to calculate them, survey results need to have accompanying measures of statistical significance, plot axes need to be labeled, etc.

Transparent
Sources of data should be provided so readers can judge validity or delve deeper.

Malaria – An Example

The table below shows what ten organizations are reporting. The filter was to look at (a) only those organizations devoted exclusively to fighting malaria, and (b) only their Home pages to see what the organizations themselves felt it was important to communicate. Many had fuller accountings deeper into their sites [ 2 ], but by this measure we look at what they see as key information to convey to their viewers right up front. [ 3 ]

In this sampling, about two-thirds of the reported statistics fall into the Enumeration and Activity categories while only a third relate to true Results. Why is this? Well, Enumeration is an easy way to make the case for the work that needs to be done and consequently the donations and support needed to prosecute the fight. For instance, Malaria No More’s site highlights these figures:

Malaria is one of the top 3 killers of kids worldwide.
Every 60 seconds, a child dies from this preventable and curable disease.

That’s compelling. And these are figures that don’t need to be gathered by the organization itself; they can simply be aquired from reputable sources and then reported as facts. As for Activity, again, these figures are easily obtained; an organization, for example, would surely know exactly how many mosquito nets it has distributed.

Results, on the other hand, are far more difficult to ascertain. They require on-the-ground follow up and measurement as well as a clear methodology for deciding which results are directly attributable to their efforts and which are related to other factors, like the work of others, shifts in demographics caused by migration or conflict, or even environmental changes affecting the spread of disease.

Conclusion

Reporting results, and not just a demographic snapshot or an accounting of activities, is an obligation as well as a powerful way to demonstrate the worth of what your organization does. It may require more work to quantify those results and typically more time to gather the information needed, but the benefits in terms of demonstrating integrity and value are well worth it.

Notes

[ 1 ] See Edward Tufte, Envisioning Information, page 67. See all of Tufe’s books for thoughtful and reasoned guidelines on how to present information, including results, with clarity and integrity: http://www.edwardtufte.com/tufte/books_vdqi

[ 2 ] For example, ALMA (African Leaders Malaria Alliance) has a very extensive scorecard two clicks into their site: http://alma2030.org/scorecards-and-reports/map

[ 3 ] There are dozens of other GOs and NGOs working on both malaria and other diseases, not shown in the table, such as WHO, PATH, CDC, MSF, etc., but it seemed like an unfair standard to expect them to single out malaria results on their Home pages. Roll Back Malaria has a list of over 230 of them: http://www.rollbackmalaria.org/about/links#m