Mobile DevOps has significantly different requirements, expectations and criteria for usefulness when it comes to monitoring mobile performance and general health-level of mobile apps, websites and APIs. Those more traditional monitoring tools and methods have hard times to provide useful and reliable data about performance and user experience. Whereas Mobile DevOps approach can be highly useful to integrate with, it breaks down barriers between mobile development and operations, and also provides clarity to process and steps for mobile product development lifecycle.
We’ll take a look at how monitoring will be used while mobile apps and websites are under development, what the critical metrics organizations will get during the product development process, and how synthetic mobile monitoring can be an integral part of the DevOps activities.
An ideal development flow provides constant, well-integrated and seamlessly flowing steps that provide clear, cohesive and continuous process for product development. What has been thought of the benefit of DevOps approach is that is no difference in the actual process whether product development is done for desktop or mobile. However, tools that are used vary a lot. In addition, there are other requirements and criteria among these two different targets. For instance, reliability of used toolset has been one of the key component to improve the efficiency and help leveraging agile process and to scale up with the new, mobile-focused tools. Download our free DevOps manifesto to learn how to incorporate this approach to your mobile team.
When looking into Mobile DevOps process and its steps, things don’t happen in waterfall, but process itself is highly iterative and all these tasks (and use of devops tools) happen in cycles. Code, Build, Test and Package (C, B, T and P) – can be seen as a simple agile sprint and therefore the later tasks could be also included in each sprint. This can be extremely beneficial if certain activities are included in those sprints.
For example, few years ago testing was pretty much the last activity organizations exercised for their products. There is absolutely nothing wrong with it, but many organizations realized that testing under the development and each regression can spot out issues much sooner and make it possible to fix problems earlier, resulting as cost-savings and faster time to market. The phenomena is also known as ‘Shift-Left’ that means that some step of the process moves closer to the beginning and is exercised ‘earlier’ in the development flow.
Could monitoring be part of each sprint? And if yes, what metrics should Mobile DevOps monitor while application or website is under development?
The Benefits of ‘Shift-Left’ for Monitoring
Monitoring has been one of those tasks that organizations do AFTER the product has been developed. That’s a great practice, but as there are great ways to monitor performance, health of the app/web, and get deeper insight of why certain things behave slow or don’t perform at all, monitoring could be part of agile sprints as well.
Monitoring and how it is done and who is in charge of it can be seen as an indicator in Mobile DevOps adopted organization. There are lots of things that can be monitored: application details, customer/user experience, and strictly performance indicators. Doing proper and deeper analysis of monitored data will help understanding how these metrics can be improved and what are the most critical ones to get user experience in order.
Monitoring performance and general health-level of mobile apps, websites and APIs is getting efficient when each and every build – whether produced during the sprint or even daily basis – are getting tested in the real context. Mobile monitoring can help to address and spot out performance, user experience and compatibility issues during the testing part of sprint. But the best outcome can be achieved when app, website or API is monitored in the real network, using real user environment, and without doing ANY modifications for getting the product monitored.
DevOps has different requirements and expectations when it comes to monitoring, which traditional monitoring tools cannot meet. Integrating mobile monitoring to be part of the Mobile DevOps lifecycle can bring monitoring into each sprint. When team is developing a product, they typically have goals and criteria for quality – and this is the phase where performance, user experience and robustness goals can be included as part of the sprint.
Synthetic Mobile Monitoring
In general there are two types of monitoring that can be performed for mobile products. One common way to use real users as part of the monitoring is called Real-User Monitoring. This includes basic crash reporting, unscripted monitoring, and basically spots problems that real users have while using the app, website or API. Naturally, this sort of monitoring is not possible while your app, website or API is under development, and in general it is not a good practice to expose non-ready product for users.
Synthetic mobile monitoring, which is commonly called as directed monitoring, provides scripting capabilities for mobile app, website and API, making it possible to spot those issues before the product goes out of the door. In addition that synthetic mobile monitoring can be used early in the development, testing and whole DevOps process, it also provides capabilities and ideal characteristics to get a comprehensive understanding of potential performance and user experience issues that the recent regression of the mobile product has.
“Synthetic mobile monitoring will be an integral part of the DevOps approach and will be used as part of the agile process.”
Synthetic mobile monitoring monitors apps, websites and API end-points like those would be used by real users. It’s basically simulating users and directs the path taken through the product with help of test automation scripts. Using synthetic mobile monitoring as a part of the agile sprint is as natural fit to devops process as integrated testing against daily builds.
Data Produced by Synthetic Mobile Monitoring
What synthetic mobile monitoring can answer in terms of compatibility, user experience, performance and robustness? As a result information about the uptime, connectivity and incompatibility, as well as raw performance capabilities are get, and that’s one of the most important business reason to use synthetic mobile monitoring during the agile process.
In a nutshell, synthetic mobile monitoring does provide highly useful data and provides an instant answer to the following questions:
- General Accessibility: Is my mobile website up? Does my mobile app launch properly?
- Mobile Performance: How fast is my mobile website in real user conditions? How quickly does my app launch?
- WiFi vs. Mobile Network: What metrics vary between fixed and mobile network? How can slow performance be improved?
These aspects and data about performance can help any mobile app, website or API to be improved to work faster and provide better user experience. But for some mobile verticals, such as e-commerce, retail, media streaming, and many more, business and usage critical factors define if the service is usable by its target audience.
By measuring those business metrics synthetic mobile monitoring provides vital information about the following aspects:
- Business Critical Transactions: Are payments and other business critical transactions working properly?
- Competition: How do my product stack against competition? How fast is my mobile app, website or API versus competitors?
- Focus Areas: Which one should be focused in terms of development – mobile app or website? Or should you provide both? Which one is more important for success?
When it comes to details of performance all data must be quantifiable. It is very important to know which part of the user experience flow aren’t producing competitive or even adequate results. Getting all data about potential bottlenecks, whether related to third party components, mobile network or anything else, should be get as soon as possible.
Synthetic mobile monitoring can provide fine-grained details of problem areas, integrations and provide insights of how these can be tackled:
- Bottlenecks of Performance: Is there any slowdown in launch and execution – and if so, what are those bottlenecks and how can those be fixed?
- Integration Aspects: Are all third party components, SDKs and APIs working as expected? What is the performance impact of those for user experience?
- Performance Optimization: How to strike the correct balance of performance vs. cost? What is the most optimal setup for my app, website or API?
Synthetic mobile monitoring can answer all of these questions, and be in use when mobile app/website/API is still under development. With modern Mobile DevOps tools your team is now able to get all these metrics, even in fine-grained details, and use real mobile networks. And all of that just in minutes around the world.
What do you use for monitoring your apps/websites/APIs today – and what are the biggest hurdles your team goes through while doing it? Share your thoughts and experiences in the comment section below.
Finally, we would like to invite you to participate in our monitoring survey. The survey aims to get your input about different locations, networks, and other important details of how mobile monitoring can be done. We’re looking forward to get your feedback!