Skip to content

Mobile is more important than ever. The increasing competition with mobile apps and user acquisition has pushed companies to act quickly with its development and operations. Getting that app ‘right’ in the first place, and working properly with a myriad of different device configurations is a task that should be considered way before the mobile app is published. In this DevOps culture, the process and tools have a significant role. We’ll take a deep dive into definitions of DevOps and what it means in the mobile app testing context.

devops in mobile app testing

DevOps is no doubt one of the buzzwords for today’s agile business culture. DevOps have been adopted widely in a variety of different types of software companies building yet more different types of products. What DevOps actually mean and what are the implications of this approach for mobile app development and testing? As defined by Wikipedia:

“Practices that emphasize the collaboration and communication of both software developers and other information-technology (IT) professionals while automating the process of software delivery and infrastructure changes.” Wikipedia

What drives companies towards this type of DevOps thinking? First of all, there are always business requirements and demand that drives the adoption of any sort of process or culture changes within companies. With DevOps approach the apparent benefits for business are the full adoption into the agile methodology, getting things done faster and with focused mentality on support, operations and services. And when these two are combined businesses want to get both as efficiently as possible.

Despite the bar has been set high for mobile apps to be successful, there are lots of different characters, things that should be considered when adopting a DevOps approach for mobile app development and testing. These requirements are driven by app developers, business owners, and most importantly, the end-users. Get our free ebook to adopt DevOps and succeed in your mobile projects.

What is Mobile DevOps?

DevOps thinking has become an established part of companies and development teams relying on agile methodologies. The combination of agile methodologies and DevOps are in essence to provide continuous integration and continuous deployment to speed up the integration, deployment and testing, plus improve the quality of the application. Getting the mobile app instantly from the build system to a real device where the application can be thoroughly tested improves the overall process velocity and makes the bug fixing easier and faster.


The DevOps in mobile means incorporating the continuous integration and testing process tightly with the delivery and deployment, and wrapping up everything in continuous operations. Developers – and development in general – focus on building the actual product while relying on their own relevant tools for them. The quality assurance (QA) is a stakeholder that is interested (and responsible) in testing those regressions, performance of the product, and preferably automating as many aspects of the testing flow as possible. The operations maintain the continuous build, integration, deployment and delivery environments, and take care of those releases. When combining all these three different functions into one holistic approach Mobile DevOps can be used as a definition.

3 Rule of Thumbs in Mobile DevOps for App Testing

Mobile DevOps is an approach that focuses on communication, integration, deployment, automation and measurement of collaboration between app developers, testers (QA folks) and the operations. The whole thing is to acknowledge the interdependence nature of this approach and rely on the right tools, methods and infrastructure in order to develop, deliver and test apps faster and more efficiently.

Develop and Test Against Real Environment

The most important thing for mobile developers is that the end-product works across all its target devices. Using emulators or simulators are acceptable in the earliest phase of the development, but when the app becomes more sophisticated and have the features that end-users will see, the only acceptable way is to test mobile apps on real devices.

Nowadays, it’s not a cost issue and, nevertheless, organizations spend significant sums on their hardware infrastructure. It’s well justified to not to try to save in the most important hardware cost – the real mobile devices – to get the app thoroughly and comprehensively tested out. As said, not everything should be acquired by the organization itself, but using a cloud service that contains the right set of mobile devices is a perfect fit for today’s mobile DevOps toolchain.

Deploy and Test Frequently (Many Times, Every Day)

Agile methodology praises the use of continuous integration and delivery tools. There are plenty of great open source as well as commercial tools available that help organizations to build, deploy, test and release their apps. The most meaningful value-add to this flow is the mobile test automation that can accommodate frequent builds to testing context and enable instant bug-hunting.

The continuous development and testing integration can enable companies to respond to market and competitive changes in a timely manner. Reusing the code, back-end implementations, APIs and other infrastructure to build a new version of the mobile application is easier, as well as testing of that when infrastructure is in place. The frequent deployment and testing also enhance the ability to automate functional, compatibility and performance testing of the application, against an array of devices and different form factors.

Continuously Consider and Validate Quality Characteristics

The modern mobile DevOps is (all) about continuous everything. The DevOps approach should always strive to improve things, process, culture, and even reconsider what DevOps tools should be in use. The continuous evaluation of efficiency and productivity can get insights into real performance issues and how to make things smoother and work seamlessly in the next regression.

Changing the DevOps tools should be considered every now and then, especially if there are any doubts about the current environment, its capability to deliver and work as efficiently as possible. And this doesn’t apply only for DevOps tools but also how mobile devices and device infrastructure is used. The requirements are constantly moving and that primarily due to demand by end-users. The quality characteristics of the mobile app are set from the first line of code until it’s on the hands of end-users, and everything that happens between that should be measured, validated and fixed – if needed.

3 Business-Critical Mobile DevOps Considerations for App Testing

From the business perspective, the ability to quickly introduce new app features in it, and bug fix releases is one of the most critical tasks for companies with a mobile-first strategy. In essence, the core of DevOps methodology aims at speeding up the app development, delivery and process by getting developers and operation specialists to collaborate throughout the end-to-end app development and deployment process.

Continuous Integration and Continuous Delivery

Continuous integration and delivery can be measured with various different metrics. However, what are the most important ones for mobile app development and testing matter the most. For example:

  • Frequency of deployment and tests executed (against each regression).
  • How many new features, lines of code or other integrations have been deployed by each build.
  • What is the actual time between the development (when it started) and test finalization?
  • Percentage and the total number of failures vs. successful tests.
  • The urgency to get the mobile app published.
  • Performance of automation. Do you need to tweak in something manually?

Now, these metrics are valid for every regression of the app. Each and every time when source code is modified and a new build is deployed, it should be thoroughly tested. With test automation that’s not a challenge but really a smooth process that gets instant results without any manual intervention.

DevOps Infrastructure for Mobile App Testing and Monitoring

When talking about mobile apps it’s not only about the APK or IPA deployed on the device, but more often there is a tremendous number of dependencies to other applications, backends, and network(s). Developing, deploying and testing in the infrastructure context also improves insights into how well the entire environment does and how the application performs. In this context, it’s also much easier to drive iterations to make usability, UI and user experience aspects better: If something lags or performs bad in the system it will have an impact to all other pieces of the system.

Monitoring mobile apps, websites, APIs and all other relevant parts are also important. While the app may perform just fine there are other entities that don’t necessarily provide the same level of performance. Accurate performance analysis with the real data helps to improve the overall development flow.

Comprehensive Dev&Test Process and Mobile App Delivery

It’s always important to remember that organizations that use agile methodology should reduce things that get done manually. Things that get done manually (especially testing) are never agile. The agile development sprints that include test automation will be faster and generally the adoption of the right tools, methods and technologies speed up the development and testing tremendously. Agile development is the epicenter of today’s mobile app development process, culture, and should be reflected in tools used by developers.

We’ll continue with the mobile DevOps theme in our next blog, so stay tuned and keep testing! 

Ville-Veikko Helppi

Mobile Testing Product Expert