Testdroid Brings Clarity to Android Fragmentation

Bitbar, the mobile devops company. Logo, large

Saying the Android market is fragmented is an understatement. Develop for Android and you are essentially developing for more than 400 different Android devices in the world today running different versions of Android with an unknown amount of carrier and manufacturer-dictated tweaks. The market is huge and varied, which is a challenge as well as a potential benefit.

Currently, most often, mobile app developers administer in-house tests one by one on the top ten devices, leaving the rest of the devices untested, but we want to make sure that you can do better because you really need to do more than those top ten devices.

Few weeks back Chris Velazco of Tech Crunch brought to public attention a great piece of work published by OpenSignalMaps where they give the very first visual representation of Android fragmentation. The data is real user data from their more than 600 000 users. They had presented the data in such a nice HTML table that with just a little bit of scripting we were able to pull the data to a database and we were able to refine the data quite a bit futher. The results are staggering!

For instance:

  • In order to ensure that your app works on 25% of most used Android devices, you need to test it on seven devices (Samsung Galaxy S2, Samsung Galaxy S, HTC Desire HD, Samsung Galaxy Ace, HTC Desire, Samsung Galaxy Note and Samsung Galaxy Mini) — five Samsungs and two HTCs.
  • To get from 25% coverage to 50% coverage, you need to add 25 more devices, with more spread on the manufacturers — nine Samsungs, eight HTCs, four Motorolas, two LGs and two Sony Ericssons.
  • To get your test coverage from 50% to 75%, you need to add 75 more devices! Each of these individual devices represents 0.6%-0.2% of the devices out there. And it just gets worse…
  • For 80% test coverage, you need to add another 147 different Android devices – that is 254 devices altoghether!
  • You have to add another 145 devices to get your test coverage from 80% of the most used devices to 90% (that’s total of 399 devices!) and the remaining 10% of the market consists of 3701 models. The long tail or fragmentation is really long and thin and most of these are single instances of a some special Android model variant.

Testdroid Cloud’s current devices cover about 70% of the devices used by Android users and we want to give our users as much information as possible about their potential coverage. So, we are releasing a feature that shows what percentage of most used devices is covered by the device combination selected by the user, to give some numbers to the fragmentation and some organization to the mess.

We are the first to provide this level of detailed market info and we are happy to do so because the more coverage we can provide our users, the more time and money they will save and the better equipped they are to thrive in today’s fragmented Android market. We thank OpenSignalMaps for providing the data in such open and processable format 🙂