Skip to content

The Mobile DevOps practices and Continuous Integration/Delivery together enable Agile development. Basically all these together are fully complementary. As we’ve been discussing Mobile DevOps is a methodology and a set of (good) practices where Continuous Integration/Delivery is more like an implementation of that. And for mobile app testing it has a lot of great things to offer.

The most important things in mobile app testing are the efficient use of test automation (using real devices), the right test automation frameworks, and efficient bug catching/handling and notifying developers about those. When it comes to parallelism and concurrent, simultaneous test runs on real devices, that’s what test automation together with CI/CD system is all about.

Mobile DevOps with Continuous Integration Enables Agile

The CI/CD systems and processes built around it push code out to testing as often as the developers need it to happen. Typically this is done several times per day or at least daily (or nightly) – and sometimes it can take days to get a new build out for device testing.  In a nutshell, CI/CD enables efficient use of Agile methodologies. You can download our free DevOps manifesto to learn how to enable simultaneous device use with a DevOps approach.

The effort of maximizing the efficiency and effectiveness will easily improve overall productivity. What this means in mobile DevOps is that test automation must be included in the process and connect it with continuous integration. Each and every build should be tested against the real environment.

Then, what is the real environment? With help of test automation developers, testers and anyone behind the system can ensure the device compatibility is meet and defects are instantly spotted out. When the problem occurs, developers should get an indication of that instantly, with all possible details and hints where the problem lays.

This speeds up of the delivery pipeline can push organizations to get instant results of their Mobile DevOps culture. The culture itself can reduce costs, as things get fixed earlier, and more importantly – make the outcome more robust and enable better quality.

How Mobile DevOps Can Enable Concurrent Mobile Test Runs

There is a myriad of testing targets when you consider how many different OS versions, hardware setups with all its diversity, physical networks, user conditions and experiences that must be delivered. Running something in simultaneously will get great details of how things actually perform.


We should start by defining what is meant by simultaneous test runs – and generally parallelism – in mobile app testing. As there are lots of discussion about whether these are the same thing or not, in general, concurrency is a concept where two or more tasks are started, executed and finalized in overlapping periods of time. For example, having multiple threads running on a single-core chipset. The concept of parallelism is probably more straightforward as it is where two or more tasks literally run at the same time. For instance, having multiple threads running on a multicore chipset.

Is that all the same when looked from mobile app testing point of view: running test scripts simultaneously on a variety of real mobile devices and getting results? The most distinctive difference would be when you get those results and whether devices start a test run as soon as the test has been launched.

However, the key point is that concurrent test runs do not necessarily run in parallel. For example, if the device picked up for a test run is not available, your test run maybe be in the queue and once the device becomes available you’ll get tests finalized – and naturally all tests are executed separately and only when test runs are done.

Parallelism Explained in Picture

“Nothing That Is Done Manually Is Hardly Agile”

Developing mobile apps is very different from developing desktop software and even embedded software. Mobile app development is meant to be agile, and lots of great tools and practices have been developed for that agility. However, doing something manually — such as testing an app — is never agile, which is why test automation has shown tremendous growth among app and game developers, speeding up their doings and yielding robust and better results. Manual testing is neither providing you the core benefit what comes with test automation – the parallelism. You can simply run one device at a time.

Screen Shot 2016-02-02 at 10.53.34 AM

NOTE! Despite all this, sometimes for a sake of exploratory testing, the manual test session is required. The remote manual access to devices complements this offering and occasionally developers want to make sure their apps work just like they’ve planned – and this is where our remote manual access to mobile devices is extremely useful. But don’t get stuck on it – that will eat all your productivity!

The Process of Mobile DevOps and What’s Included

The actual development is closely related to continuous integration. The agile approach made continuous integration very popular as it enhanced the way developers work, collaborate and integrate their doings together, in a natural way. The continuous integration with regular daily routines produce lots of data about the mobile application, regressions and compatibility of each developer’s contribution, and discovering issues early exposes integration flaws, even incompatibility of developer’s changes, and makes better time consumption and schedule estimating easier and more accurate.

Any tools and additional practices around the following process can enable simultaneous tests done on devices. The code is developed, maintained and built for test automation runs. And after all test iterations are finalized the deployment to production can happen. The deployment to production in mobile app context means publishing the release for its users.

DevOps Process and Its Steps

There are only a few great DevOps tools mentioned in the picture above, but there are plenty of other great ones too. But let’s focus on test automation a bit more.

An Ideal Test Automation for Mobile DevOps

Mobile test automation utilizes real mobile devices and offers the possibility to test mobile apps instantly and effectively using test automation frameworks. The infrastructure should be built to support an unlimited number of simultaneous test runs, meaning that users can select any number of devices for their test runs.

The actual benefit of test automation comes from simultaneous test runs, automated access, coverage, and runs on devices, repeatability nature of tests, and identical and accurate tests – that could probably never get to accurate with manual testing.

There are basically 4 requirements for organizations to meet the ideal flow with development, testing and deployment, and those are as follows:

1. Use Open Source and Standards 

Test automation should rely only on open standards that are widely used in the industry. Open standards mean there are no vendor lock-ins, the source code is always open and available, and developers can incrementally develop new tests and focus on compelling features with their apps.

2. Only Automation Boosts Productivity

Automated testing can increase the depth and scope of tests and significantly improve software quality. Lengthy and thorough tests that are often not doable with manual testing can be run automatically. Automated tests can easily execute thousands of complex test cases during every test run, providing coverage that is simply not possible with manual testing.

3. Cost Efficient and Effective Use 

Depending on the size of the project and the application, test automation will always provide a good return on investment. Once tests have been created, automated tests can be run over and over again at no additional cost. Test automation typically reduces the time required to run repetitive tests from weeks to hours. This is a significant time-saving that translates directly into cost-saving.

4. Integrate the Flow with Complimentary Tools

To get the most out of your efforts and maximize testing coverage, select the most robust and most cross-platform method – and hook it up with your continuous integration, development and deployment system. Once you build an app, the system can automatically push it for testing on devices in the cloud. This saves a lot of time, produces instant results and developers can fix any issues instantly.

In order to automate your mobile testing, you can use different types of procedures and methods to accomplish the real device coverage, compatibility and functional testing.

Ville-Veikko Helppi

Mobile Testing Product Expert