“What devices should we use to test our applications” is a question we get on a daily basis from our customers. As we have over hundreds of thousands of users from all over the world and from every possible industry sector, there is no simple answer to this question.
The diversity that different application segments require for mobile app testing is enormous and for example game developers tend to use different types of devices for testing when compared to enterprise app developers. The very same applies to users, people who use that app/game on their devices. We’ve included both Android and iOS devices in our data.
Which Mobile Devices and Platforms Should You Test On?
How can a mobile app, game and website developer maximize the coverage and device compatibility across the most relevant devices available out there?
In short, there are significant differences within mobile user segments and not every type of device is used for every app. For instance, in the gaming sector, gambling and casino users use very different devices that high-end 3D gamers. Customers of an app utility company in the energy sector use an entirely different device that the users of Pokemon Go, and so on.
There are many ways to create a test device matrix for the mobile app testing: If you already are in the market with your app, you get some relevant statistics from the app store (Apple Store and Google Play). If you are just entering the market, then you do not have that data at hand. Some mobile analytics companies also release device usage data, which can be used to determine what devices to use in your development, QA and testing.
You can also gather sales data, but newer models are never in the top devices as they are just entering the market. Typically it takes three to six months to see which devices will gain the popularity and are used for certain types of apps in the market.
Furthermore, if you are using data from mobile analytics company, there can be bias in the data. If the analytics company is mainly serving gaming companies, their metrics are biased towards younger and gaming oriented device users.
The most important questions to answer to get an idea and comprehensive understanding of device roster are as follows:
- Where is your target audience? In which countries are your users? This is the first criteria to define device roster for your testing needs as even the device with all specs, and brand name, indicates it being similar with global device variants that’s not the case. There are tons of customizations and different add-ons on certain devices used in certain mobile networks (and devices branded for certain telcos).
- What are the form factors and types of devices used by end-users? Nowadays, mobile apps run across a variety of different form factors (phones, tablets, wearables, etc.) and in many cases apps are optimized for better user experience in each of these.
- What is the ‘target OS’ version and what other variants should be supported? Some applications are restricted to work only on the latest OS versions due to used APIs and integrated services. These applications won’t install and work on the older ones and typically users should upgrade their device with some newer OS version to get the app working. This is crucial to get the right OS-device combination and make sure application works on these variants.
How Was The Data Collected?
We collected the data from mobile web access. Mobile web usage should be the most unbiased as the web is used on every device, among young and old users, low income and high income, globally in every country, social media users, shoppers, news readers, sports fanatics, men and women: they all use the mobile web daily. Download the comprehensive mobile device listing for mobile app testing.
Our data is collected from web usage data and is based on global web traffic data from hundreds of thousands of websites based on the user agent information. Based on this you can get your test device matrix in order.
Naturally, it’s easy to say that the comparison of Android and iOS devices is straightforward and most media does the comparison based on Samsung Galaxy and Apple iPhone latest versions. But if you look at some this data provided about the failures, there are lots of differences when it comes to both, Android and iOS devices. Among Android phone vendors, Samsung handsets experienced the highest rate of failures at 43% globally. The Galaxy S6 was the handset that experienced issues most commonly in Samsung’s device roster, while the iPhone 6 experienced the most malfunctions of all of Apple’s smartphones.
Interested in Full Detailed Report with Device Listings?
We have gone through all the data and we have broken down the data based on countries and what are the most popular device models used. If you need more information, or if you want us to deliver the report to you in Excel format, please contact our sales (sales at bitbar.com) and we’ll be happy the share all data + findings with you. You can also download the report from mobile app testing resources page.
Happy Testing folks!