Market analytics: Figuring out how much cash million-dollar Android app makers really make
Million (and sometimes billion) dollar decisions are made based on the best analysis of incomplete data. If you'd like some inside baseball on how researchers come up with some of their insights, here's a quick view into the insanely profitable business of in-app Android game sales.
There is no doubt that the lion's share of the mobile app market comes from games. According to Appbot's analysis, 98 percent of smartphone and tablet games are monetized through in-app purchases. We know that there's a ton of money there, but just how much money has each app generated?
That's what I set out to determine. Google lists its top-grossing apps, which gives us a starting point. Unfortunately, Google doesn't provide much more information.
This is a classic market analytics problem. Many businesses don't release their numbers, because they consider that confidential and proprietary information. However, business planners, investors, and we pundits are often desperate for some inside baseball on what's happening in the companies we follow.
It's from this type of problem that the entire industry of market research was born. The idea is to somehow come up with a reasonably accurate analysis from incomplete data. While you can often use the information produced by market researchers, it's important you don't treat such numbers as the gospel truth. If you look carefully at any given market, you'll see different estimates by different forecasters. This is similar to the problem forecasters had handicapping the recent US election, and that forecasters are now struggling with for the upcoming French elections.
The process of coming up with a market analysis is equal parts art and science. Or, if you want, lies, damned lies, and statistics. Basically, we work from what numbers we have, we build computational models, and we make assumptions. It's the assumptions part that's the art, which often comes from years of experience watching a market.
For example, I'll later show you how I make an assumption about the weight of negative reviews. That comes from my experience observing that negative reviews often have more velocity in a market than positive reviews.
The top grossing games
To come up with a set of numbers on the Android game app market, I started with the 25 top-grossing games. I eliminated the very few apps (Google Drive and Pandora) that were in the top 25 list, but weren't games. That left me with 23 games, so I included the next two games in the ranking to fill out my 25.
Now that I had my list, the first thing that became apparent was that Google's Top Grossing list was probably not based on the all-time life of an app. The first game on the list, Mobile Strike, had substantially fewer total downloads than Clash of Clans, the game listed in second place.
While it's possible that more people are spending $399 (yep, that's a real number, folks) on items in the Governator's Mobile Strike game than the $99 max in Clash of Clans, it's far more likely that the Android Play top grossing list is based on monthly numbers than cumulative results.
By the way, if you're looking at the above screenshot and realizing that Mobile Strike is NOT listed in the number one slot, you're right. When I took my data dump last week, it was the top game. This reinforces my observation that the Top Grossing list isn't cumulative.
This speculation is also supported by looking at Candy Crush Saga, which has somewhere between half a billion and a billion downloads, yet was listed as number four on the list when I took my data snapshot. There's no doubt Candy Crush has crushed it when it comes to in-app income, but it's been around for a long time. Its cumulative revenue may be greater than Mobile Strike, but its recent revenue may not be.
These 25 wildly popular Android games are raking in the most cash from in-app purchases
Changing data is a big issue in market analysis. How anal do we need to be about the currency of our data? In other words, should we be freaking out that the Top Grossing list changed since I compiled my analytics data?
The answer is somewhat complex. If, for example, I was being paid by corporate clients who were making impending billion dollar decisions, it might be an issue. If that were the case, I'd probably have live feeds and constantly updating metrics.
But, for our purposes, which is to just get an understanding of the scale of the Android game app market, it's not something to worry about. What we're concerned with mostly is a matter of scale: realizing just how insanely big this market is at the top end.
There are a number of ways we can approach measuring total revenue. This is the practice of building a model that might help us come up with some information.
I started by first looking at work done by AppsFlyer, a research firm that calculated that the Google Play store revenue per user is $0.43 per app. This is a measure across all users of the apps, and incorporates both paying and non-paying customers in its measurement.
By this measure, the #4 ranking Candy Crush Saga brought in $215-$430 million compared to the $21.5-$43 million of Mobile Strike.
While taking a measurement based on total number of users is one way of guestimating overall revenue, there are better ways. A much better way takes into account (or tries to estimate) the number of users who are in-app purchasers. Remember that the number of in-app purchasers will nearly always be far smaller than the larger number of users who limit themselves to the base, free versions of most apps.
Calculating paid users
To calculate the number of paid users, there are two main measures: how many total users there are, and some way of determining the percentage of users who are paying customers. While the Android Play store does show download numbers, the number of downloads cover a huge range.
For example, going back to Candy Crush Saga, the difference between the minimum and maximum total install count is half a billion users. That's a big number. That said, while we can't calculate an exact value of the number of users, publicly listed install values are stratified. Therefore, it's possible to see that a product like Candy Crush Saga, even with it's 500 million range of accuracy, still has many more downloads than, say, Summoner's War, which ranges from 50 to 100 million downloads.
AppsFlyer provides another metric: their analysis says that 4.6 percent of Android users make one in-app purchase per month. Unfortunately, they don't connect that to individual products, so it's not possible to tell which app was used to make that one purchase.
There's still value in that measure, though, because it gives us a sense of purchaser activity. We can extend this observation to say that, for a core of about 5 percent of app users, about 12 in-app purchases are made a year.
Since it's unlikely that most of these users are making in-app purchases for as many as 12 different games, it's more likely that the in-app purchases are restricted to one or two favorite games. Based on that assumption, it's not unreasonable to expect, conservatively, that at least one in twenty downloads results in an in-app purchase.
Another approach that can be applied is to use another metric as a proxy for the number of purchases for a given app. The one that I chose to use is the number of reviews for each app. Since reviews require interaction beyond the game itself, we're likely to see the more engaged users (both satisfied and cranky) engage with the reviews system.
There are some odd results from this measure, in particular with Clash of Clans. For that game, somewhere between 7 percent and 35 percent of all game downloads resulted in a review. By contrast, the average number of reviews based on the minimum number of downloads is 9 percent, while the max is a much more reasonable 2 percent.
Using number of reviews as a proxy for purchases also helps us more closely target the actual number of users, since the number of reviews is not listed as a range, but an absolute number. Clearly those who are actually playing the game are more likely to rate each game.
Taking just the number of reviews doesn't account for dissatisfied users. Some users simply may not have enjoyed playing, while others, like King of Avalon user Jennifer Justus who said, "You will spend thousands and they lie about everything," seem to have had very bad (and costly experiences).
As Ms. Justus shows, you don't have to be happy about the game to spend money on it. That said, the more unhappy the overall audience of users, the more likely it is that revenue will be impacted.
Based on this fundamental premise, I decided to weight the buyer proxy formula such that buyers are calculated based on the number of three, four, and five star reviews, minus one and two star reviews. Because unhappiness tends to have a more magnified impact on product success, I doubled the weighting of negative reviews when calculating my purchaser volume proxy.
How much do buyers spend?
The next question is how much did those buyers spend? Users spend between $0.99 and $399 per item in Mobile Strike, but only -- only(!) -- a buck to about a hundred dollars per item in Candy Crush Saga. For this, we'll go back to AppsFlyer, who says that the average paying user spends $9.60 per app, per month.
That number seems like a lot, but it's about the same that a Netflix subscription costs. If you think about the idea that some paying game players derive many hours of entertainment from their games -- and usually in situations where that might be the only available, appropriate entertainment -- the cost doesn't seem out of balance with other entertainment products.
For example, someone stuck on a bus for an hour each way to and from work might well enjoy some strategy gaming for an hour and find the ten bucks spent monthly more than worth it as a way to stave off boredom and discomfort.
In an attempt to see how much money has been made by these top apps, let's assume a year of use. Clearly, some apps have been around much longer. Those apps are likely to have collected more reviews. Because we're using review activity as a purchaser volume proxy to identify paying buyers, the total number of reviews not only helps factor in engagement, but also provides a view into sales longevity.
Our final estimated in-app purchase income is then calculated based on the $9.60 monthly average multiplied by 12 months. That value is then multiplied by the purchaser volume proxy based on reviews. That final value gives us our estimate value for in-app purchases for any given app.
This approach actually works rather well. For example, take a look at the estimated revenue using this model for Clash of Clans and Clash Royale, the two games seem to have pulled in close to $3 billion. A quick look at Wikipedia for information on Supercell, the Finish company who makes those two games, shows that the company had revenues of $2.23 billion in 2016.
Over the course of a few years, Supercell could absolutely have brought in the $3 billion the model estimates, especially when you factor in Hay Day and Boom Beach, two additional games from the company that were not in the top 25.
What else can we learn from this data?
One of the more interesting observations that comes from having an ordered list containing overall estimated revenue, and another list containing what appears to be current gross revenue numbers, is that it becomes possible to see which apps seem to be growing in popularity.
For example, Mobile Strike is listed as the top grossing app on Google Play, but is way down on our list of overall gross revenue. Clearly, the app hasn't yet earned as much as other apps, but seems to be extremely strong as a revenue producer right now.
It's helps to conceptualize trends when we give them names. For example, it is far easier to think about revenue growth when you read names like Rocket, Rising Star, Smooth Sailing, Falling Star, and Sinking Ship than it is just looking at a chart.
So, we rated any app ranking more than 10 points higher on Google Play than in cumulative revenue as a Rocket. Any app ranking more than 5 higher was rated a Rising Star. Any app ranking more than 5 lower on Google Play than overall revenue was considered a Falling Star. Any app that's ranking more than 10 lower on Google Play than overall revenue was declared a Sinking Ship. Finally, any app that's showing steadiness in ranking was labeled as Smooth Sailing.
After all that work, is this analytics model absolutely accurate? Of course not. But is it close enough to be able to gauge the level of activity and the profit potential for this exploding part of the tech industry? Yeah, it is.
It gets the job done and shows us, bottom line, that an insane amount of money is being spent on mobile games. It also shows how powerful the freemium business model can be when applied to the right market.