Skip to content

Mobile DevOps has very different expectations, criteria and requirements when it comes to successful performance monitoring and general health-level of mobile apps, websites and APIs. The industry has somewhat neglected the fact that monitoring should happen in real locations over REAL networks with real infrastructure. The methods and tools build around mobile monitoring have been focusing too much on Real-User Monitoring (RUM) and unfortunately, this approach provides less useful and unreliable data about the actual performance – and the user experience. In this blog, we’ll take a look at the benefits and the real value of synthetic mobile monitoring, and how it can provide reliable and useful data about the performance, health-level and eventually, the user experience.

What Successful Mobile DevOps Must Have: Mobile Approach

The Mobile DevOps approach, tools and new methods build around continuous integration (build), test automation (testing) and deployment are all driving new monitoring approaches forward as well. The simplicity of Mobile DevOps approach can be highly benefits for many organizations as it can break down barriers and silos between mobile development and operations, and also provide clarity to the overall development process and enable steps that are highly beneficial for mobile product development lifecycle.

Successful mobile devops must be mobile-first

An ideal mobile app development flow provides a constant, well-integrated and seamlessly working incremental steps for a clear, cohesive and continuous process that today’s mobile app development needs. One remarkable benefit of DevOps approach is that eventually things, processes, mindsets and doings aren’t that different whether product development is towards a mobile app, game, website or a desktop app. Mobile has its own nature and this must be aligned with used tools. Among the other things, there are other requirements and criteria for these two different targets. For instance, the reliability of used toolset has been one of the key characters to improve efficiency and help to leverage agile process and significantly scale up with the new, mobile-focused tools. Manifesto to learn how to adopt Mobile DevOps approach to your mobile team. When looking into Mobile DevOps process and its steps, things don’t happen in the waterfall model, but the process itself is highly iterative and all these tasks (and use of DevOps tools) happen in cycles. Code, Build, Test and Package (C, B, T and P) – can be seen as a simple agile sprint and therefore the later tasks could be also included in each sprint. This can be extremely beneficial if certain activities are included in those sprints. mobile app development and production For example, a few years ago testing was pretty much the last activity organizations exercised for their products. The worst thing was that organizations accepted the model of having testing to be done as the final task for the product. This made many of mobile-first and generally other organizations as well to think about how testing could be done as part of the development, and how test automation could be harnessed for each and every build coming out from continuous integration.

“Just a few years ago, it was OK to accept testing as the last task for any mobile product. Today, things can be done in much more productive manner, with cost savings and faster time-to-market, using mobile test automation. When do we see the same trend in monitoring mobile apps under development?

Shift-Left or Shift-Horizontally

Monitoring has been one of those tasks that organizations do AFTER the product has been developed. That’s a great practice, but as there are great ways to monitor app performance and health, and get a deeper insight of why certain things behave slow or don’t perform at all, monitoring could be part of agile sprints as well. Monitoring and how it is done and who is in charge of it can be seen as an indicator in organizations that have adopted Mobile DevOps. There are lots of things that can be monitored: application details, customer/user experience, and strictly performance indicators. Doing a proper and deeper analysis of monitored data will help to understand how these metrics can be improved and what are the most critical ones to get user experience in order. A Need to Monitor Mobile Apps Monitoring the performance of mobile apps, websites and APIs gets efficient when every build – whether produced during the sprint or even daily basis – gets tested in the real context. Mobile monitoring can help to address and spot issues in performance, user experience, and compatibility during the testing part of the sprint. But the best outcome can be achieved when app, website or API is monitored in the real network, using a real user environment, and without doing ANY modifications for getting the product monitored. DevOps has different requirements and expectations when it comes to monitoring, which traditional monitoring tools cannot meet. Integrating mobile monitoring to be part of the Mobile DevOps lifecycle can bring monitoring into each sprint. When a team is developing a product, they typically have goals and criteria for quality – and this is the phase where performance, user experience, and robustness goals can be included as part of the sprint. Learn how to get started with Mobile Monitoring for Apps, Games and Web in this award-winning ebook.

Synthetic Mobile Monitoring Provides the Solution

In general, there are two types of monitoring that can be performed for mobile products. One common way to use real users as part of the monitoring is called Real-User Monitoring. This includes basic crash reporting, unscripted monitoring, and basically spots problems that real users have while using the app, website or API. Naturally, this sort of monitoring is not possible while your app, website or API is under development, and it is not a good practice to expose non-ready product for users. Synthetic mobile monitoring, which is commonly called as directed monitoring, provides scripting capabilities for a mobile app, website, and API, making it possible to spot those issues before the product goes out of the door. In addition that synthetic mobile monitoring can be used early in the development, testing, and whole DevOps process, it also provides capabilities and ideal characteristics to get a comprehensive understanding of potential performance and user experience issues that the recent regression of the mobile product has. It’s basically simulating users and directs the path taken through the product with help of test automation scripts. Using synthetic mobile monitoring as a part of the agile sprint is as a natural fit to DevOps process as integrated testing against daily builds.

Data Produced by Synthetic Mobile Monitoring

What synthetic mobile monitoring can answer in terms of compatibility, user experience, performance, and robustness? As a result information about the uptime, connectivity, and incompatibility, as well as raw performance capabilities are getting, and that’s one of the most important business reason to use synthetic mobile monitoring during the agile process. In a nutshell, synthetic mobile monitoring does provide highly useful data and provides an instant answer to the following questions:

  • General Accessibility: Is my mobile website up? Does my mobile app launch properly?
  • Mobile Performance: How fast is my mobile website in real user conditions? How quickly does my app launch?
  • WiFi vs. Mobile Network: What metrics vary between fixed and mobile network? How can slow performance be improved?

These aspects and data about performance can help any mobile app, website or API to be improved to work faster and provide a better user experience. But for some mobile verticals, such as e-commerce, retail, media streaming, and many more, business and usage of critical factors define if the service is usable by its target audience.

By measuring those business metrics synthetic mobile monitoring provides vital information about the following aspects:

  • Business Critical Transactions: Are payments and other business-critical transactions working properly?
  • Competition: How do my product stack against the competition? How fast is my mobile app, website or API versus competitors?
  • Focus Areas: Which one should be focused in terms of development – mobile app or website? Or should you provide both? Which one is more important for success?

When it comes to details of performance all data must be quantifiable. It is very important to know which part of the user experience flow aren’t producing competitive or even adequate results. Getting all data about potential bottlenecks, whether related to third-party components, mobile network or anything else should be getting as soon as possible.

Synthetic mobile monitoring can provide fine-grained details of problem areas, integrations and provide insights into how these can be tackled:

  • Bottlenecks of Performance: Is there any slowdown in launch and execution – and if so, what are those bottlenecks and how can those be fixed?
  • Integration Aspects: Are all third party components, SDKs and APIs working as expected? What is the performance impact of those for user experience?
  • Performance Optimization: How to strike the correct balance of performance vs. cost? What is the most optimal setup for my app, website or API?

So… What do you use for monitoring your apps/websites/APIs today?

Ville-Veikko Helppi

Mobile Testing Product Expert