A good and smooth performance of your mobile app cumulates as a great user (or preferably, customer) experience. Today, we’re honored to have Doug Sillars covering this topic in his guest blog – “How to Automate Mobile Performance Testing” – and announce something new with their Application Resource Optimizer. Also, you may want to check Doug’s book on this theme. Enjoy!
Automating Your Performance Testing
As you build your mobile application, you (hopefully) run many tests to cover new features (and of course, look for bugs or crashes). In many cases, these tests are executed automatically, and you only get alerted when they are completed (and hopefully all of them passed!)
When it comes to testing your mobile application for performance (by which I mean the speed at which your application responds to the user, and the battery and data usage of your app), all of the testing tools (to date) have required manual testing. Unfortunately, we all know that manual testing can be tedious, and is often one of the first things cut during a demanding development cycle.
The AT&T Application Resource Optimizer (ARO) is a free tool from the AT&T Developer Program for Android and iOS developers. It monitors the network traffic coming in and out of your test device and gives you recommendations on how to optimize the traffic being sent. Until very recently, it could only be run as a manual test, and we saw this as a shortcoming to our service.
My team here working on ARO knew that anything we could do to make testing easier. We think that making performance testing an automatic part of the build and testing process will increase the amount of performance testing. And if we can increase the testing, we will be less likely to see code that dramatically slows down our apps make it into production.
“We think that making performance testing an automatic part of the build and testing process will increase the amount of performance testing.”
So this week, we are happy to announce the release of ARO 5.0 with a new SDK that allows control of your performance testing from your automation test suite. Our goal is to help you “add performance audits” to your existing testing – by adding 3 simple commands into your automated test.
The SDK also runs from the command line, and it is incredibly simple to run:
sh arocli.sh --startcollector vpn_android --output /Users/Demo/tracename --video yes
This command starts the collector (using the Android VPN method of collection) saves the data in “tracename” and also records the video. By calling this before your automated test, you have started the process. Now you can run the automated test that you have already built. When your test script is complete, send a
command to the same terminal to stop the ARO data collection. The analysis step can then be performed:
sh arocli.sh --analyze /Users/demo/tracename --output /Users/demo/tracename/report.json --format json
Here we took the trace located in the tracename directory and created a JSON report (HTML reports are also available) so you can programmatically parse the JSON file for the analysis results and determine the next step as part of your DevOps process.
Our goal is to help you add performance testing of your application into already existing automated testing frameworks (or if you have no automated testing, to make ARO easy incorporation into your app testing frameworks.) For manual tests, you can also generate a human-readable HTML report instead.
sh arocli.sh --analyze /Users/demo/tracename --output /Users/demo/tracename/report.html --format html
Also, you can tell me how you test your mobile app for performance bottlenecks by tweeting me (@dougsillars). If you are interested in testing your mobile app for performance issues (or trying out our new automation tools, please download ARO).
Happy Performance Testing!
About the Author
Doug Sillars is a lead performance architect at AT&T, where he has helped thousands of developers improve app performance. He has Ph.D. in inorganic chemistry, studying how to make chemical reactions faster, but has pivoted, and been working in mobile for 12 years. He usually lives on a farm on an island north of Seattle with his family, chickens, goats and other creatures. However, last year Doug and his family began traveling through Europe and will be visiting many different cities and countries for this year.