Mobile Text Surveys: A Smarter Way to Measure Conservation’s Impacts on People?

Samburu woman with her cell phone. Image credit: Eddie Game/TNC.

A Samburu woman checks her cell phone. Image credit: David Mbiyu/Demotix.

Eddie Game is a senior scientist with The Nature Conservancy.

Conservation today has a lot riding on being able to demonstrate the positive impact of its work not just on nature, but also on human well-being, especially in developing countries. Donors, other funders, governments and potential partners are demanding that our work be sensitive to people’s welfare and even enhance it if possible. If we don’t prove we’re having that positive impact, we risk being shut out of many global conversations and dynamics.

One problem: The way we’ve been measuring the impact of our work on people — through in-person household surveys — can often be intrusive, expensive and unreliable. It can take hours to complete one of these surveys. Subjects sometimes tell surveyors what the subjects think they want to hear. And it’s impractical to survey people frequently.

Enter the mobile phone — especially the SMS (short message service) text.

Mobile phone use in developing countries has exploded at a rate unmatched by any other technology in history. A person in a developing nation is more likely to have access to a mobile phone than clean water or electricity. 77% of the world’s 6 billion active mobile phone subscriptions are in the developing world.

They also use SMS in overwhelming numbers. About 96% of Indonesia’s mobile owners use SMS regularly. In Kenya, it’s 89%, Mexico 82%, and China 80% (World Bank 2012).SMS is the lowest common denominator in phone technology — all phones are capable of it.

The availability of mobile phones in the developing world is radically changing the approach to data collection and information delivery in fields like health and agriculture. We believe mobile phones can do the same for conservation.

That’s why The Nature Conservancy is piloting the use of mobile phone text surveys to collect data for human well-being indicators in Kenya’s Northern Rangelands — one of the poorest regions of the country and one historically beset by tensions between nomadic pastoralists and wildlife.

If we’re successful, the approach could spark a revolution in how conservation tracks its impact on people.

Why Collecting Data Via Mobile Phones Can Be Better Than In-Person Surveys

This isn’t to dismiss the value of in-person household surveys. But in many developing country situations like the Northern Rangelands, collecting data via mobile phones can be a superior approach for these reasons:

Quick. Compared with in-person surveys, answering a text message is quick and relatively unobtrusive. When was the last time you were enthusiastic about spending an hour to take a survey?

Accurate. If someone asked you how many time have you been sick in that past two years? You’d take a wild guess and probably forget a bunch of times — unless you happened to be sick in the past couple of weeks, in which case you’d probably overestimate. (Psychologists know this as availability bias.) But if asked whether you have been sick in the past week, you should be able to answer it pretty accurately. Text message questions can be asked in near real-time, thus collecting more accurate answers.

Frequent. The environment changes continuously and understanding how these changes influence human well-being and behavior is difficult with surveys conducted once every few years. Mobile phones will allow for nearly continuous data collection, giving the power to relate changes in human well-being to important environmental or even social events.

Cheap. For many conservation projects, household surveys can be prohibitively expensive. A World Bank study found that getting the same information through in-person surveys cost more than 6 times as much as via text message.

How it Works: The Mobile Data Collection Pilot in Northern Kenya

Kenya is a nearly ideal place to test the mobile data-collection approach. Nairobi is fast becoming the tech hub of Africa. Mobile phone and text message use is high in Northern Kenya, even among the widely dispersed pastoral communities of Samburu, Turkana, Borana, and Somaili who live here. And the need is great: Increasing population and aridity in the region have put enormous pressure on both its people and its wildlife.

We’re working on the pilot with our local partner, the Northern Rangelands Trust, and our initial focus is the Kalama community conservancy — which, like other community conservancies in northern Kenya, has both wildlife conservation objectives (for species such as the endangered Grevy’s zebra) and a range of objectives concerned with improving the lives of the communities there.

But while Kalama has a good system in place to monitor wildlife populations and rangeland ecosystems, that system doesn’t yet have a good counterpoint for tracking human well-being. So we are setting up a monitoring system based on SMS text messages that will tell us how investments in actions such as creating grass banks, improving pastoral management and establishing alternative livelihoods are improving people’s lives.

Participants at a project site register into a database and are then periodically sent a small number of SMS questions regarding aspects of human well-being. As an incentive, a small amount of airtime credit — 10 Kenya shillings in the Kalama pilot — is offered when people answer questions. (As an indication of relative value, sending a single SMS in Kenya typically costs 1 shilling.)

We license a free number or “short code” with each of the mobile networks so that participants are not charged for the SMS messages when they register or answer questions; we also have partnered with the web mobile platform provider EchoMobile to establish a system that automatically transfers airtime credit to a respondent’s account when an answer is received. By repeatedly asking the same question through time, we can use the answers to look for changes in well-being linked to the conservation project.

What are some of the issues our questions to the Kalama participants touch on? For instance, we want to know about milk yields from their cattle, which we hope will improve as a result of better grazing management and healthier rangelands. Or incidents of banditry or other violence that the participants might perceive to diminish as a result of community cooperation and the presence of wildlife scouts.

Because there’s no chance for follow-up, SMS survey questions need to be tailored for the purpose. Ideal questions reflect real-time or very recent experiences, such as “Have you worked for cash in the past two weeks?” Response rates to questions can help us fine tune the way we ask them.

And just like the many a/b tests that Google does to improve its service, sending out large number of questions via SMS allows us allows to ask the same question in different ways and look at the response rate.

For example, many people in northern Kenya do not know their birth by a date, but relative to events such as droughts, floods, elections, etc. We tried asking participants their age, both directly and in broad age categories to see if they gave us the same demographic, and whether one would receive more responses than another. The answer? No difference.

Mind the Gap: Are Those Without Phones and the Illiterate Excluded?

Ownership

An obvious risk of our survey technique is that it might exclude those people who don’t have access to a phone — a real problem if, for instance, the poorest section of society didn’t own phones.

To find out whether there are any demographic blind spots for this approach in our pilot site, we undertook a thorough survey of mobile phone ownership and use amongst Kalama communities. The results surprised even our local partners: over 90% of adults owned a mobile phone. Importantly for the proposed monitoring approach, whether someone owned a phone or not was unrelated to age, gender or wealth.

Literacy

Another challenge — and one that I was more worried about in northern Kenya — is relatively low literacy rates. Reading and responding to an SMS survey question requires basic literacy, and literacy rates among adults in Northern Rangelands communities are well below the Kenyan national average of around 90%.

Overall in Kalama, 59% of those surveys were comfortable using SMS, and another 26% said that someone in their household helped them to read and respond to SMS. As expected based on literacy rates, however, older community members, and especially older women, were much less likely to use SMS — only 45% of elders said they were able to use SMS themselves, with a further 25% saying someone in their household could help. Although we feel these numbers will not invalidate the survey results, it will be important that we find a way to hear the voices of those people unable to participate in the mobile survey.

To ensure the literacy barrier is no bigger than necessary, we are able to send SMS questions in the most locally appropriate language. In northern Kenya, the most widely spoken language is Samburu, but few people see it written so we decided to send the messages in Swahili. However, text-book Swahili is long way from its colloquial use in the rangelands, so each message needs to be crafted to be both simply and consistent with local dialect.

‘How Will This Help Us?’

This was the first question I was asked when I met with the Kalama Conservancy board of elders in June to seek their approval to pilot mobile phone monitoring in Kalama.

I described two tangible benefits: First, knowing whether projects in Kalama were delivering the desired outcomes or not would offer a chance to improve those projects. Second, being able to firmly demonstrate the impact of projects would help make the case for more resources to expand successful activities.

In addition to these, the elders offered two other benefits I hadn’t considered: the system would provide a way for community members to communicate directly with NRT about experiences or concerns. It would also provide a mechanism for the Kalama board to send out community announcements about events or warnings.

Very reasonably, I fielded numerous questions about data security and access. Who would have access to their numbers and their responses? Was there a risk the government might use their number to monitor their activities? Despite my unsatisfying answer that governments appear keen on monitoring mobile phones irrespective of projects like these, it was a good litmus test of the security protocols we had established.

And — not to be underestimated — there was a palpable sense of novelty as a number of the elders sent a registration text message, answered some questions, and received a small airtime transfer and thank you message.

The Community Response

Once we secured the approval and support of the Kalama board, we shared the project more broadly with Kalama residents — and the response far exceeded my expectations.

In a few days, about 370 people voluntarily registered and answered a series of initial demographic questions via mobile phone. Importantly, these people included folks from all ages, both genders, and a range of occupations.

Buoyed by this response, I now want the pilot not simply to reach a representative sample, but most Kalama’s 2,000 or so adults with this approach. While I’m sure the airtime incentive helped, it was immediately clear that respect for the Northern Rangelands Trust was instrumental in encouraging participation.

It remains to be seen what sort of response rate will be maintained through time. Previous mobile phone survey work has indicated a long-run response rate of around 50%, and our early results are slightly above this. As this is not intended to be a panel survey that dedicatedly follows the same person through time, it is fine if participants miss some questions and respond to others, provided there is a critical mass of responses to each question.

What’s New Here? And What’s Next?

Of course, collecting data via mobile phones is hardly a new idea. Substantial efforts have been made — particularly in the fields of development, health and advertising — to use SMS as an avenue for data collection.

But our pilot takes a novel approach. Most SMS data collection efforts fall into one of three types: 1) an aggregator (such as a community nurse) who collects local statistics and shares them via SMS; 2) panel surveys that follows a small number of individuals through time; or 3) one-time surveys asking for opinions on a product or event.

In contrast, our pilot is based on looking at trends in response through time by asking questions directly to a large number of participants in a semi-randomized fashion.

The statistical power of our approach comes through the volume of responses and randomization rather than trying to control other variables. In genesis, this paradigm owes more to the methods developed by Harvard psychologists to study happiness than it does to other SMS data collection approaches.

And we are cautiously optimistic that the approach will prove useful, replicable and scalable. In addition to extending our pilot to other community conservancies in northern Kenya, we are also starting a pilot in Colombia later this year to track well-being impacts of the Agua Por La Vida water fund — water funds having rapidly become a flagship conservation strategy throughout Latin America, but one with little data to date on impacts on people.

With a little support, conservation’s ability to document and improve our impact on people’s lives could be transformed.

Reference

World Bank. 2012. Information and Communications for Development 2012: Maximizing Mobile. World Bank, Washington D.C.

Opinions expressed on Cool Green Science and in any corresponding comments are the personal opinions of the original authors and do not necessarily reflect the views of The Nature Conservancy. 

 

Posted In: Human Well-Being

Eddie is the Conservation Planning Specialist with The Nature Conservancy’s Conservation Methods and Tools Team. Based in Brisbane, Australia, he works across the organization, trying to improve approaches to spatial prioritization and promote good conservation decision making. Eddie received his PhD from the University of Queensland, under Professor Hugh Possingham, and has previously worked in fisheries and marine conservation. He has published on conservation planning, coral reef resilience, pelagic protected areas, dynamic decision making, evolution and mountain biking in Kyrgyzstan.



Comments: Mobile Text Surveys: A Smarter Way to Measure Conservation’s Impacts on People?

 Make a comment




Comment

Forest Dilemmas

Too many deer. Logging one tree to save another. Beavers versus old growth. Welcome to forest conservation in the Anthropocene. Beginning Monday, July 21, join us for a provocative 5-part series exploring the full complexity facing forest conservation in the eastern United States.

What is Cool Green Science?

noun 1. Blog where Nature Conservancy scientists, science writers and external experts discuss and debate how conservation can meet the challenges of a 9 billion + planet.

2. Blog with astonishing photos, videos and dispatches of Nature Conservancy science in the field.

3. Home of Weird Nature, The Cooler, Quick Study, Traveling Naturalist and other amazing features.

Cool Green Science is managed by Matt Miller, the Conservancy's deputy director for science communications, and edited by Bob Lalasz, its director of science communications. Email us your feedback.

Innovative Science

Investing in Seagrass
Marine scientists and fishers alike know that grass beds are valuable as nursery habitat. A new Conservancy-funded study puts a number to it.

Drones Aid Bird Conservation
How can California conservationists accurately count thousands of cranes? Enter a new tool in bird monitoring: the drone.

Creating a Climate-Smart Agriculture
Can farmers globally both adapt to and mitigate the impacts of climate change? A new paper answers with a definitive yes. But it won't be easy.

Latest Tweets from @nature_brains

Categories