Estimating the Value of Mobile Money

By Jeff Kaufman 🔸 @ 2016-12-21T13:58 (+8)

In the US we have many options for sending money around: there's the traditional cash, check, credit card, bank transfer, etc., and then there's PayPal, Venmo, Square, Wallet, etc. In poorer countries, however, if you want to send money to someone in another city your best option is often to send cash on a bus. In 2007 Safaricom introduced M-Pesa in Kenya, where it has been very successful. Can we get an estimate of how valuable M-Pesa has been?

In The long-run poverty and gender impacts of mobile money (Suri and Jack 2016) they try to figure out the effect of the M-Pesa rollout on per-capita consumption. They argue that from 2008 to 2010, when M-Pesa was just getting started, new agents were distributed effectively randomly. [1] Then they look at how areas that were close to M-Pesa agents in 2008-2010 compare to ones that weren't, and argue that this difference represents the effect of having access to M-Pesa.

They find that having access to M-Pesa raised log [2] per-capita consumption by 0.012 (n=1593, standard error=0.005, Sidak-Holm p=0.04). (EDIT: this is an underestimate; see update below.) The reason to care about log consumption is that the more money you have the less valuable additional money is, and this relationship is approximately logarithmic (though it might fall off even more sharply). So by averaging log consumption over all the people in their sample we're getting a rough approximation of how much better M-Pesa access made people's lives.

What does a 0.012 increase in log per-capita consumption look like? Here's a table:

per-capita consumption change
$50,000 $603
$25,000 $302
$10,000 $121
$5,000 $60
$2,500 $30
$1,000 $12
$500 $6
$250 $3
$100 $1

 

To get a comparision, another way to increase someone's consumption would be to just give them money. GiveDirectly is currently the best option for doing this, and on average GiveDirectly gives $288 per-person to recipients with an annual consumption of $238, a log-increase of 0.79. This is 66x the 0.012 log-increase for M-Pesa availability. GiveDirectly is about 80% efficient, so getting someone $288 costs about $360. So, very roughly, giving someone access to M-Pesa one year earlier is about as valuable as donating $5.45 to GiveDirectly. [3]

While M-Pesa was very successful in Kenya, its expansion to other markets has been slow. Improving M-Pesa's expansion, or starting a competitor that can expand faster, could be very valuable. For example, the population of Ethiopia is 94M, so getting mobile money available in Ethiopia a year earlier would be about as good as donating $500M to GiveDirectly. [4]

This is a very rough analysis, and as it is essentially a multiplicative chain errors anywhere along the chain are magnified. For example, two errors in the first version of this post moved the final estimate by 2.3x and 1.5x respectively. This estimate is nowhere near as robust as, for example, GiveWell's estimates for GiveDirectly or the AMF, and those are still very uncertain. Still, it suggests deploying mobile money is an extremely promising intervention, competitive with the top global poverty interventions.

Update 2017-01-01: Tavneet Suri, corresponding author of the paper I'm drawing on here, responded to my question about the interpretation of cell 1 in table 1. Above I interpret it as the effect of having an M-Pesa agent within 1km, but it's actually the effect of having one additional M-Pesa agent within 1km. Since the average number of M-Pesa agents within 1km was ~9, naively we'd expect this to mean the effect of M-Pesa is 9x what I estimated above. But since the first M-Pesa agent is probably much more valuable than the tenth, and their sample mostly didn't include this no-access-to-access transition, the true effect is probably more than 9x. On the other hand, this is kind of big, maybe implausibly so. Instead of a ~1% increase, we're talking about more like 20%. Could M-Pesa really have made Kenya that much richer?

Cross-posted from jefftk.com.


[1] On the other hand, after 2010 they believe there was a systematic effort to expand in places with higher demand. Overall, this is one of the weakest parts of this analysis: if places that got agents earlier were more actually more economically dynamic then it wouldn't be at all surprising to see greater consumption growth there.

[2] The paper doesn't say what base they're using for the log, but I'm assuming here that it's e: natural log is standard in economics. Additionally, in Table S1 of the supplement they give both "Daily per Capita Consumption" and "Log Daily Per Capita Consumption", and the particular numbers they have rule out a base as large as 10. In a draft of this post I had initially assumed log10: thanks to Rob Wiblin for catching this.

[3] My model of this is that you can either wait and get the consumption boost from mobile money eventually, or you can do something to make it happen sooner. If we take a window long enough to include the eventual deployment of mobile money I would expect to see something like:

The solid line represents the path the average log(per-capita consumption) would take, with a jump at the eventual introduction of mobile money, while the dashed line represents moving that jump forward in time. So the benefit is the area between the two curves: how much better mobile money is for people, multiplied by how much you can accelerate it.

[4] Or donating $105M to the AMF, according to GiveWell's cost-effectiveness estimates.

undefined @ 2016-12-25T19:22 (+3)

Because it is helpful to think about exactly what intervention is needed to help mobile money expand (which may differ by country), I'm throwing here a few potential barriers (mostly based on my own experience in Kenya and Myanmar):

undefined @ 2017-01-03T21:35 (+1)

The intervention I had in mind when writing this post was joining a start-up that has been working on this and has been seeing good results so far: http://www.jefftk.com/p/leaving-google-joining-wave

undefined @ 2016-12-21T16:16 (+2)

Thanks for this Jeff - a very informative post.

The study doesn't appear to control for cash transfers received through access to M-Pesa. I was thinking about how much of the 0.012 increase in ln(consumption) was due to GiveDirectly cash transfers.

Back of the envelope:

So GiveDirectly accounted for (very roughly) a fifth of the 0.012 increase in ln(consumption) due to M-Pesa.

[1] this is an overestimate as it assumes all transfers went to Kenya and none to Uganda

(Done in haste - may have got my sums / methodology wrong)

undefined @ 2017-01-04T18:13 (+1)

I spent about two hours looking at this in further depth and made an initial stab at modeling out the impact. I estimate an effectiveness of $200/hr (95% interval: $50/hr to $511/hr), not taking into account the value of donating the salary earned from working at Wave.

Some places where I notice we disagree or I am confused:

1.) I disagree with you here (footnote 1) that there is a 50% chance of failure (or success). I think the chance of failure could be significantly higher. From https://en.wikipedia.org/wiki/M-Pesa: M-Pesa expanded to Kenya (>10M subscribers), Tanzania (5M), South Africa (100K in a year, 1M in five years), India (???), Mozambique (???), and Lesotho (???).

Also, a 2016 Vodaphone press-release suggested M-Pesa seems to have 25M customers worldwide after 10 years of effort.

Based on this, I model that a Kenya-level success (>10M subscribers) thus looks like it would have a less than 1/10 chance and a South Africa-level success (1-10M subscribers) looks like it would have a ~3/10 chance. However, I think this success figure could be lower due to diminishing marginal returns since M-Pesa has already plucked low hanging fruit. It's possible better technology could increase this chance. I'd have to know more specifically about what problems M-Pesa runs into and how these are addressed.

-

2.) I think your estimate that getting M-Pesa a year earlier is only 66x worse than getting a $288 transfer from GiveDirectly is an overestimate because I expect future roll-outs will take place in countries with higher base consumption. However, as you point out, that estimate is also already an underestimate due to misunderstanding the study. I don't know how to correct for this either way, so I used the 66x number literally in my calculation.

-

3.) I disagree with you here (footnote 1a) that marginal ETG donations are at GiveDirectly levels of cost-effectiveness. I expect them to continue at AMF levels (or greater) for at least a few more years, for reasons OpenPhil mentioned and Carl mentioned. I did an AMF-adjustment in my model for this reason.

-

4.) I really don't know how many staff years it would take to either complete a roll-out or know that it's not going to happen and this is an important part of the model. I currently guess 5-10 full-time staff for 2-5 years, or 10-50 total staff years. This does not count field agents or other hired locals. I couldn't find any information on M-Pesa's total staff count anywhere. I note that Wave has at least 44 staff (from counting faces on the about page), but I don't know if they're all full-time or all focused on expanding cash transfers.

-

5.) I'm confused about why GiveDirectly is stated to be 5x more cost-effective than AMF when from GiveWell's cost-effectiveness estimate, GiveDirectly has a median of $7702 per life saved, ranging from $2200 to $16000, excluding outliers. AMF has a median of $3282 per life saved, ranging from $2200 to $4800, excluding outliers. Together, this implies a comparison centered around 2.35x but ranging from 1x to 3.3x. Maybe I misread the sheet -- I haven't invested that much time in making sure I fully understand it yet.

undefined @ 2017-01-05T21:33 (+1)

Re 1, it's worth noting that M-Pesa was administered by very different teams in different countries. In Kenya it was allowed to operate mostly as a startup with limited oversight from Safaricom (or anyone else), whereas in other countries (to varying degrees) the people running M-Pesa were constrained by stricter management from the telecom's country head. This means that there are obvious ways in which M-Pesa was executed less well in other countries. For instance, Wave integrates with M-Pesa in both Kenya and Tanzania, and despite running on exactly the same technology, the Tanzanian system's uptime is substantially worse. Similarly, the quality of their agents and their customer support staff in Tanzania is noticeably lower.

Since Wave isn't hamstrung by oversight from a stodgy and risk-averse telecom, I think you should give less weight to examples from countries other than Kenya as a base rate.

undefined @ 2017-01-04T19:23 (+1)

1) ... I think the chance of failure could be significantly higher.

Possibly, but they are already starting to operate in the country in question, and my understanding is that's been going pretty well. My impression is that they're much more competent than Safaricom. My inside view is much higher than 50%, and getting down to 50% was a discount from there.

2) ... I expect future roll-outs will take place in countries with higher base consumption

I'm confused. I was trying to talk about the counterfactual for a specific very poor country if Wave were not working there. So if future mobile money rollouts by other organizations happen first in countries with higher base consumption then that increases the counterfactual impact of Wave choosing to come into a country with very low consumption.

3) ... I expect them to continue at AMF levels (or greater) for at least a few more years

See http://www.jefftk.com/p/leaving-google-joining-wave#fb-835897806972_835943804792

4) ... I really don't know how many staff years it would take

That, combined with estimating marginal impact, makes this pretty awkward. I figure something like 40 person years?

5) ... I'm confused about why GiveDirectly is stated to be 5x more cost-effective than AMF

This comes from cell F31 of the "Results" tab. I haven't put time into understanding how that's calculated, but it looked like the relevant bottom line number.

undefined @ 2017-01-04T19:43 (+1)

Thanks Jeff!

-

1) ... I think the chance of failure could be significantly higher.

Possibly, but they are already starting to operate in the country in question, and my understanding is that's been going pretty well. My impression is that they're much more competent than Safaricom. My inside view is much higher than 50%, and getting down to 50% was a discount from there.

Sounds like you definitely have inside info that I don't have, so for now I'd have to rely on my outside view, but I can work to acquire that inside info if I look into this more.

-

2) ... I expect future roll-outs will take place in countries with higher base consumption

I'm confused. I was trying to talk about the counterfactual for a specific very poor country if Wave were not working there. So if future mobile money rollouts by other organizations happen first in countries with higher base consumption then that increases the counterfactual impact of Wave choosing to come into a country with very low consumption.

I don't know what country Wave is looking at or how they are doing what they do because you have inside info that I don't have. If it has consumption comparable to Kenya than my point is invalid. I just was concerned that it wouldn't.

-

3) ... I expect them to continue at AMF levels (or greater) for at least a few more years

See http://www.jefftk.com/p/leaving-google-joining-wave#fb-835897806972_835943804792

Cool. Sounds like this isn't a disagreement between us then.

-

4) ... I really don't know how many staff years it would take

That, combined with estimating marginal impact, makes this pretty awkward. I figure something like 40 person years?

Agreed that it is pretty awkward to estimate. I modified my model to use some of your inputs -- such as a 40% chance of 1-10M subscribers and a 10% chance of >10M subscribers and 40 person years -- and it comes out to $383/hr (95%: $145/hr to $834/hr). The new mean is still in my old 95% interval which is about the best I can hope for with this level of uncertainty.

-

5) ... I'm confused about why GiveDirectly is stated to be 5x more cost-effective than AMF

This comes from cell F31 of the "Results" tab. I haven't put time into understanding how that's calculated, but it looked like the relevant bottom line number.

Oh, I see that now. I suppose this is a question for GiveWell and not you. I'll ask them.

undefined @ 2017-01-05T18:29 (+1)

Sounds like you definitely have inside info that I don't have, so for now I'd have to rely on my outside view, but I can work to acquire that inside info if I look into this more.

If you're interested in working for Wave, or are advising other people on whether it's a good idea for them, I could imagine they'd be quite interested in talking to you!

if it has consumption comparable to Kenya than my point is invalid. I just was concerned that it wouldn't.

It's poorer than Kenya.

undefined @ 2016-12-21T14:57 (+1)

Very cool post.

Just saw that the transaction costs for m-pesa are quite high - the company makes ~20% profit... so there might be something that a Wave-like startup could do: https://en.wikipedia.org/wiki/M-Pesa#Cost.2C_transaction_charges.2C_statistics

maybe using crypocurrency - see here:

http://phys.org/news/2016-10-cryptocurrency-bottom-billion.html

undefined @ 2016-12-21T18:19 (+2)

Just saw that the transaction costs for m-pesa are quite high - the company makes ~20% profit

The transaction costs listed on the wikipedia page you cite aren't trivial, but would average well less than 20% unless most transactions are (a) very small and (b) to unregistered users. I'm missing something.

EDIT: could it just be that their profit is 20% of expenses, as opposed to 20% of the money that flows through the M-Pesa network?

maybe using crypocurrency

That article doesn't really show that cryptocurrency helps here. Mostly they're unhappy with transaction fees on international remittances, but you can have low transaction fees just by automating interactions with the money transfer organization, without going to cryptocurrency. And with cryptocurrency generally you pay someone a fee to get your money into the cryptocurrency and then your recipient pays someone else a fee to get it into their local currency.

undefined @ 2017-01-05T21:34 (+2)

I think their profit is 20% of their revenue (for a money transfer company, revenue is the total fees brought in, not total money paid into the network).