Evidence Principles for Sustainability Challenges

December 6, 2018

Photo © Mwangi Kirubi

Many in the environmental community strive for evidence-based decision making because we believe it’s the fastest way to the biggest, and most lasting impacts. Not surprisingly, the same is true in health and economic development.

Despite this, evidence is an elusive term. The health community has come up with a definition of ‘strong evidence’ through Cochrane, while the development community has their own, somewhat different definition, championed through the Campbell Collaboration. The environmental community has no agreed approach to assessing strength of evidence. This makes evidence-based decision making very challenging, especially as conservation organizations increasingly think beyond sole environmental outcomes to the reality that lasting conservation must also often deliver health, income or other benefits to people.

The Nature Conservancy’s Global Conservation Agenda puts such strategies right at the core of our work, recognising that the many linked challenges facing people and nature need to be matched by cross-disciplinary solutions. And we’re clearly not alone in this awareness; a look at the UN Sustainable Development Goals illustrates that the leaders of nearly all nations appreciate the intertwined challenges facing people and nature.

Photo © Michael Yamashita

Solving the problems at the top of today’s global agenda will require the use of evidence from many disciplines. Yet, discipline-specific approaches to evidence, driven in large part by the types of evidence historically available in different disciplines, represent a massive barrier.

Using a medical standard to assess evidence on environment outcomes, for example, would mean ignoring much available evidence (for instance, studies that are observational rather than experimental), reducing the ability to choose among interventions and likely missing key insights about whether and where an intervention will lead to the desired change.

Alternatively, assessing evidence in a manner that employs a different understanding of validity than medicine can be a barrier to collaboration across disciplines where health outcomes are concerned. Uncertainty about how to reconcile these different views of evidence has permeated professional communities, taking implementing groups (like us), funders and policy maker out of their comfort zone. We require an approach to evidence assessment that can combine evidence from different disciplines.

Our proposed solution was published recently in the journal Nature Sustainability. The Nature Conservancy brought together leading evidence thinkers from disciplines including environment, economics, health, law and philosophy. The diverse composition of this group enabled a key breakthrough: a recognition that despite many differences in the ways each discipline would assess evidence, there was a set of characteristics that we all agreed were indicative of stronger evidence, regardless of evidence type or discipline. These are four, cross-disciplinary, evidence principles.

  • The variety of evidence types that support an association between a phenomenon or intervention and observed outcomes. If different methods show the same thing, the less likely the result is an artefact of the method.
  • The consistency of the effect found in the evidence about the causal association. When many studies show the same direction and order of magnitude of effect, that constitutes stronger evidence.
  • The credibility of the evidence sources being integrated or considered.
  • The applicability of the evidence to the question of interest. Applicability is high when the places studied and the place(s) where evidence is being applied are similar in ecological, social, cultural, political and economic context.
The four principles that underpin a cross-disciplinary approach to evidence. Where all four principles are met, the body of candidate evidence would provide strong support for a hypothesized causal association. Graphic © E.T. Game & H. Tallis

These principles provide a set of characteristics of strong evidence to look for when gathering and assessing evidence. Importantly, they allow the assessment process to recognise that we need to draw on, and combine, a wide variety of evidence sources including things like community knowledge.

In the words of Nancy Cartwright, one of the world’s leading evidence philosophers and co-author on the paper, they provide a “practicable and reasonable” approach to evidence assessment. These principles intentionally move away from a strict, hierarchical view of evidence where evidence perceived as being from weaker sources is ignored in favour of those perceived to be stronger.

Since our group of co-authors identified these principles, the Bridge Collaborative was founded to take this and other ideas at the intersection of health, development and environment forward. Through six highly diverse working groups, these principles were improved and shaped into an evidence grading rubric that makes their application more straightforward in evidence synthesis and program evaluation efforts.

Photo © Jason Houston

The publicly available guidance document released last year makes these principles more actionable for researchers and practitioners alike. Efforts are underway to test and improve this rubric through the evaluation of evidence across a wide set of disciplines.

For example, a team of engineers, health experts, ecologists, and hydrologists are applying these principles to understand how grey (built infrastructure) and green (natural habitat) interventions impact sanitation outcomes and freshwater biodiversity. The evidence principles and rubric may help them compare findings from models, engineering studies, WASH impact evaluations and conservation biology efforts more consistently.

The recent IPCC Global Warming of 1.5 Degrees report is our most recent reminder of the pace and scale of change our world needs to achieve if we are to reach the Sustainable Development Goals and a future where things are better for both people and nature. Today’s siloed evidence bases and fundamentally different views of evidence across disciplines stand in the way of needed, rapid progress.

We hope these evidence principles are one tool that can grease the wheels of change, and help people in all disciplines and sectors draw on the massive wisdom the human race has generated.

The authors would like to thank the David and Lucille Packard Foundation for generously funding this research. 

Eddie Game

Eddie Game is the lead scientist for The Nature Conservancy, Asia Pacific Region. He is responsible for ensuring that the Conservancy remains a world leader in making science based conservation decisions, can robustly report on our impact, and that we get the greatest return for our conservation investments. Eddie has worked on conservation projects in over 15 countries, helping to apply innovative methods and analyses to projects as diverse as community protected areas in the Solomon Islands, grazing management in northern Kenya, and catchment restoration in Colombia. More from Eddie

Join the Discussion

Please note that all comments are moderated and may take some time to appear.

1 comment