Parallelism and Concurrency in Mobile App Testing

Bitbar, the mobile devops company. Logo, large

Dear Testdroiders,

One of the cornerstones in mobile app testing is the efficient use of test automation, with real hardware, making all that to work for you efficiently, and seamlessly integrate with your development and testing process. You know, the whole beef with test automation is that you can simultaneously run tests on different devices, preferably tens or even hundreds of those, to get comprehensive understanding of real behavior of your app on real platform. As discussed, there are myriad of different combinations when you consider OS version, hardware setup with all its diversity, networks, user conditions and many other things. Running something in concurrent – or parallel – mode will get you great details how things actually perform. Let’s take a look at some fundamentals of mobile app testing and how to use parallelism / concurrency to deliver you a significant value.

parallel-concurrency-thumb-2

We should start by defining what we mean as concurrency and parallelism when it comes to mobile app testing. As there are lots of debate whether these are the same thing or not, in general, concurrency is understood as a concept where two or more tasks are started, executed and finalized in overlapping time periods. For example, having multiple threads running on a single-core chipset. The concept of parallelism is probably more straightforward as it is where two or more tasks literally run at the same time. For instance, having multiple threads running on a multicore chipset.

When mapped into mobile app testing context both of these would be eventually the same: running your test scripts simultaneously on variety of real mobile devices and getting results. The most distinctive difference would be when you get those results and whether devices start a test run as soon as it has been initiated. However, the key point is that concurrent test runs do not necessarily run in parallel. For example, if the device picked up for a test run is not available, your test run maybe be queueing and once the device becomes available you’ll get tests finalized – and naturally all tests runs separately and only when test run is done.

parallel-concurrency-2

Agile Mobile App Development and Testing

Developing mobile applications is very different from developing PC software and even embedded software. Mobile development is meant to be agile, and lots of great tools and practices have been developed for that agility. However, doing something manually — such as testing an app — is never agile, which is why test automation has shown tremendous growth among app and game developers, speeding up their doings and yielding robust and better results. Manual testing is neither providing you the core benefit what comes with test automation – the parallelism. You can simply run one device at time.

Screen Shot 2016-02-02 at 10.53.34 AM

All Testdroid product have been built for test automation. However, sometimes for sake of exploratory testing, the manual test session is required. The remote manual access to devices on our data centers complements this offering and occasionally developers want to make sure their apps work just like they’ve planned – and this is where our Testdroid Interactive is extremely useful.

Ideal Characteristics of Mobile Test Automation

Test Automation relies real mobile devices (Android, iOS) and offers the possibility to test mobile apps and websites instantly and effectively using test automation frameworks. Our infrastructure is build to support unlimited number of simultaneous test runs, meaning that you can select any number of devices for your test run.

The true benefit of test automation comes from these simultanous test runs, automated access, coverage, and runs on devices, repeatability nature of tests, and identical and accurate tests – that could probably never get to accurate with manual testing.

  • Open Standards – Test automation should rely only on open standards that are widely used in the industry. Open standards mean there are no vendor lock-ins, source code is always open and available, and developers can incrementally develop new tests and focus on compelling features with their apps.
  • Automation without Complexity – Automated testing can increase the depth and scope of tests and significantly improve software quality. Lengthy and thorough tests — often not doable with manual testing — can be run automatically. Automated tests can easily execute thousands of complex test cases during every test run, providing coverage that is simply not possible with manual testing.
  • Cost Efficient and Effective Use – Depending on the size of the project and the application, test automation will always provide a good return on investment. Once tests have been created, automated tests can be run over and over again at no additional cost. Test automation typically reduces the time required to run repetitive tests from weeks to hours. This is a significant time-saving that translates directly into cost-saving.
  • Integration with Agile Process and Tools – To get the most out of your efforts and maximize testing coverage, select the most robust and most cross-platform method – and hook it up with your continuous integration, development and deployment system. Once you build an app, the system can automatically push it for testing on devices on cloud. This saves a lot of time, produces instant results and developers can fix any issues instantly.

In order to automate your mobile testing, you can use different types of procedures and methods to accomplish the real device coverage, compatibility and full functional testing. For example, there are three ways to produce tests for any mobile application:

  • Handwritten Tests – Typically, this is the best choice when you know what you’re doing and when you have programming-capable people doing the test scripts. Plenty of options are available for test automation frameworks, tools and integration — both commercial and open-source offerings. Having your engineers write all of the test scripts will take up some time and tie up resources, but it will get you exactly what you want: well-structured, thorough scripts that test precisely the aspects of your app that you want to test.
  • Record and Playback Approach – This approach is less error-prone because nothing needs to be written in code, but it is typically more limited in functionality. Tests can be quickly recorded and then played back again and again against different OS versions and device configurations. These tests focus on user interactions and user-driven activities. Some things might fall beyond a test’s scope (such as integration with other technologies and compatibility with other software).
  • Automatic Test Exercisers – Automatic test exercisers provide a great way to smoke-test applications. No specific tests are needed; rather, the focus is on testing user-interface logic (such as opening menus, clicking buttons, swiping and multi-gesture actions). Automatic test exercisers yield the least exact results but provide quick feedback on each iteration of an app.

In order to evaluate what test automation framework works you the best please take a look our comparison of each and if needed please consult us to find you and your app the most suitable test automation framework.

Enjoy the Parallelism / Concurrency with Your Test Runs!


5 Things to Consider When Adopting Mobile App Test Automation

Learn these aspects to improve test efficiency and effectiveness.

Download