Skip to content

You know about the importance of testing your mobile application on real devices. Your customers don’t use emulators, so you can’t rely on them to get tangible test results. However, sometimes, you lack time to test your app before its release date in detail. What should you do? Should you focus on testing a variety of operating system versions or testing as many different device models as possible? Let’s take a closer look at this question and discuss which of the presented options is more important for real device testing.

Why Do You Need to Test Your App on Different OS Versions?

First, we should think about why it’s necessary to test your app on different operating systems and versions. The most obvious reason that comes to mind is that your users have different OS versions installed on their phones and tablets. We can find an exceptionally high amount of different Android OS versions in the market (also called Android fragmentation). Also, we need to remember that newer versions of Android or iOS aren’t installed on older devices. At some point, mobile devices are no longer supported – but your customers still have them in their pockets.

If you wish to make your application accessible and functional to many people, you can’t solely focus on testing on the newest OS versions!

Some OS versions and their updates may not have a significant impact on your app. Therefore, your developers and QA must stay up to date on what will happen in future updates. You should be ready to retest your application on new OS versions, taking into account all significant changes.

Speaking of staying up to date: Apple has just released a separate OS for its tablets, introducing even more diversity in the testing landscape. So from now on, you probably have to test your application on the iOS and the newly introduced iPadOS.

Why Is It Essential to Check Your App on a Variety of Device Models?

As you well know, we can find far more device models than OS versions out there in the wild. They all come with different screen resolutions, operating systems, battery types, memory, CPU power, and more. We can barely keep up with the evolution of hardware and specifications built into smartphones and tablets.

That’s why you need to test a variety of device models and make sure your mobile application works for the majority of your users. You need to keep an eye out for new models and act accordingly when older ones disappear from the market. Always keep in mind that you can’t expect your users to have the latest devices.

Make sure you test on real devices to get tangible test results. For example, you need to ensure that your app doesn’t drain the phone’s battery. You can hardly test that on an emulator!

What Part of Your Testing Process Can You Neglect to Avoid Testing Bottlenecks?

As we’ve discovered so far, we can’t merely say testing on different OS versions is more important than testing on a variety of device models. Both are crucial for real device testing. Thus, we can’t look at these two aspects separately, but always together.

However, when time is short, you may have to decide which part of your testing process to neglect. We recommend that you create a list of combinations of devices and operating system versions on which to test your app. Place the most important smartphones and tablets at the top of this list, followed by the less essential ones. Whenever you can’t test on all devices and operating system versions as planned, start by checking the combinations from top to bottom in this list. That way, you’ll make sure that your app works flawlessly, at least on the most critical devices.

Creating this list requires you to do some research and dig into market data. Get as much information about your audience as possible. See which devices are most popular in certain countries, or for specific age groups. Follow the same steps to determine which OS versions to go for. Also, consider researching screen sizes, device age, manufacturers, upcoming devices and OS versions, etc. – get to know your audience and their use of mobile phones and tablets better. Always keep the list up to date, and make use of it regularly!

How to Make Real Device Testing More Efficient

In an ideal world, you would not have to prioritize your list of devices. Instead, you could easily test on all the smartphones, tablets, and operating systems your customers use. Although we’ll probably never be able to check everything, we can still come very close to this perfect scenario. There’s always room for improvement and boosting efficiency!

Here are our two favorite strategies to save time and make testing more affordable and enjoyable:

1. Automate your test cases

We love automation! We also believe that you can’t test efficiently if you only perform manual tests. First, determine which test cases you can automate – most likely, you can’t automate everything, but strive for as much automation as possible! Doing so will save you a lot of time, allowing your developers and QA to focus on other important and more pleasant tasks.

2. Test on an unlimited number of real devices in parallel

Automated tests still need time to execute. Consequently, you can imagine how much time is required to run these tests on one device at a time. Thus, you need to run your automated test cases on many devices together. The more mobile devices you can test at once, the more time you have left for other tasks! This also means that you can dramatically reduce your testing time if you can run tests on an unlimited number of devices concurrently. We know this may sound a bit abstract, so we did the math for you in this article.

Boost Your Efficiency with a Cloud-Based Testing Service for Real Device Testing

What’s more essential for real device testing – OS version or phone model? Finally, we can answer this question with confidence: They’re both essential! In this article, we’ve been thinking about how to test your mobile app as thoroughly as possible, even in the case of bottlenecks.

  • Make a list of the devices and OS versions to determine which combinations are most critical to test. Prioritization is key!
  • Automate your test cases, aim to make testing more efficient, and don’t let manual testing become a bottleneck.
  • Run these automated tests on an unlimited number of devices concurrently, save time, and release high-quality apps.

At Bitbar, we help you make your testing processes as efficient and practicable as possible. We invite you to give our real device cloud a shot for free. See for yourself how real device testing with unlimited concurrency improves your mobile application’s quality!

Image by Steve Johnson via [source link] (copyright-free)

Sarah Mischinger

Tech Writer