When it comes to measuring mobile application and website health, the most basic – but also the most important – metrics are availability and performance. These metrics can be obtained by monitoring app, website or API running in real conditions, utilizing real networks and connection to back-end services. But how can users get their app, website or API running across the globe without putting that in hands of end-users?
By monitoring your mobile app, website or API either by running synthetic tests periodically or by looking at end user experience on aggregate level using real user monitoring (RUM) provides comprehensive understanding of health of your mobile product at any given point in time.
There are two main approaches that are primarily used to monitor web applications:
- Synthetic monitoring, also called directed monitoring.
- Real-user monitoring, also called passive monitoring.
The depth of these used techniques and tools range from simply pinging a server to solutions that drive a complete end-to-end transactions.
In this introduction we’ll highlight some of the fundamental differences when measuring the end user experience on mobile networks vs. fixed internet connectivity, and the use cases where mobile monitoring using the real user environment has tremendous value to offer.
Desktop-First vs. Mobile-First
The nature of web traffic has drastically changed in the past few years. For instance, currently up to 70% of the traffic to any popular website is from mobile (and primarily with Android and iOS phones and tablets). Additionally another 30-40% of the traffic to your back-end comes from your native Android and iOS apps. This means that some 80% of total users for many popular services (especially ecommerce or entertainment) come from mobile.
In general, variety of development tools can be used across different use cases. Some tools work well for development of desktop-oriented use cases and equally well for mobile targeted products. However, when it comes to mobile, many of those foundations that make mobile-first enablement are fundamentally different. The architecture of desktop-powered environments is still x86 driven, hardware components used in devices are very different (regardless of used software components), connectivity uses differently powered hardware, displays, memory, and many other hardware components are simply different on desktop vs. mobile.
One significantly different foundation on desktop and mobile is the used network. Whereas mobile networks are very much different from fixed network powered desktop world, the importance of testing and analyzing the actual performance on the right environment is emphasized, and getting the exact user experience, performance level, measured is no job for emulated/simulated network connections, but it should happen on real-user conditions.
Use Cases of Mobile Monitoring
Given the significance of mobile terminated web and application traffic we have been really surprised to found out that all monitoring tools on the market are made for desktop-first scenarios and none of these tools are addressing the specific requirements of services that are predominantly used on mobile devices over mobile networks. These include:
How Content Transfers and Renders on Mobile Networks
Checking how mobile content really transfers and renders through most popular mobile networks varies depending on many factors. The actual network speed is a combination of the network congestion on the cell level of each carrier’s network and the signal strength of any given location. But that’s not the only problem, but carrier-level content compression, bandwidth throttling, and even content filters make the actual content transfer and content rendering different on mobile.
The need to use real mobile networks and user conditions is imminent and many mobile application and website developers, DevOps and IT professionals know the pain of being dubious of what actually happens with their product on certain mobile networks, in certain locations.
Analyzing Real Performance as Experienced by End-Users
When it comes to analyzing those two important metrics – performance and availability – there are few things that should be highlighted. The basic connectivity with fine-grained details of what happens in communication stack is important for getting holistic view of things happening on the transport layer. It’s not only how quickly the content is downloaded on mobile, or how quickly mobile device communicates with the mobile network, but also how quickly that content is getting rendered and that can only be measured only on real environment. Furthermore, another very important measurement point is that whether the rendered content is accurate and correct, or if there has been any compromises due to slow connectivity, performance of the device with app/web content, or simply no-access to back-end.
Performance Impact of APIs, Services and SDKs
Measuring what is the actual performance, the real performance and experience that end-users get, won’t be tackled with throttling network speed or simulating it inside the corporate firewall. Testing your mobile service from outside your own firewall and actually on real mobile network means looking at the many things that can break between a user and your service. In addition, lots of modern APIs and third-party components are included in mobile products that cannot be analyzed in the performance content unless executed on the real user environment. Being able to analyze what is the real end-user impact of using a third-part API, SDK, or even Ad-network gives the advantage to enable faster and better performance, and optimize exactly those right things affecting performance.
Diverse Set of Browsers, OS versions, Devices
Running and monitoring performance in the real environment helps proactive mitigation of the impact of different browsers, OS versions, and even different device variants. Primarily, developers aim to build things performing perfectly on their development environment. But as we know from the mobile app world, users have diverse set of different browsers and devices that make mobile apps and websites performing differently, the capability to measure and optimize in real end user environment can provide that vital information on how performance can be boosted, and how to optimize your service to work across different sets of browsers, devices, and networks.
It is also very important to do this measuring systematically and on continuous basis because even if you don’t change anything on your service, the devices get webkit updates, OS updates, carriers changes their network configurations and any of these can break mission critical functionality on your app, website or API end-point.
Performance of Ad Networks and Paid Content/Distribution
Nobody wants to pay for things that are sloppily delivered, not correctly reaching and targeted for its audience. The mobile ads shown as part of today’s apps, games and websites is that content that corporations of all sizes spend significant dollars on. Ensuring that return-on-investment is achieved and your brand is well-presented to your target audience mobile monitoring has a true value to offer. Ensuring that those ads you are paying for are actually presented correctly on EVERY location where you decide to boost your virtual presence, mobile monitoring can help to get that confidence.
The Lifecycle of Mobile Applications
Today’s mobile platforms are suggesting (and yes, not only suggesting but often forcing) users to get up to date with the latest versions of apps, browses and operating systems. For instance, downloading the latest versions happens pretty much on background, but without exceptions, always hits the performance of the system. Making sure download works, update gets through, and user sees the process seamless, effortless and fast, is highly important. All this happens over 80% through those real mobile networks.
Introducing Bitbar Monitoring
For the past year, we’ve been working to get this groundbreaking technology infrastructure with massive scalability to your hands and to be part of your development, operations, or DevOps toolbox. Despite today’s DevOps toolbox is pretty full with all kinds of tools from configuration management to development, testing and integration, and delivery management, the monitoring and performance optimization part has been missing a true mobile first tool that can get the confidence of how well mobile apps and websites perform under real conditions.
With these core problems in mind we have started developing game changing mobile first synthetic monitoring service called Bitbar Monitoring. If you are facing these same problems, please sign up to our monitoring newsletter and beta list.
We’ve reached our internal beta status and this website will explain how we will approach the problem, and how our product will tackle those challenges seen and experienced by today’s mobile app and web developers.
Stay tuned – this seriously cool product will be out soon!
Get all essentials about XCTest framework and learn how to get started with it for cloud testingDownload