Guest blog post – Using Testdroid Cloud to reproduce memory leaks

Bitbar, the mobile devops company. Logo, large

This is a first post in a new series of guest blog posts where Testdroid Cloud users give examples how they use Testdroid Cloud to make their day to day Android Development easier. The first guest blogger is Kevin Rohling, founder/ceo of cisimple. cisimple hosts Continuous Integration services for iOS and Android applications

If you’re an Android developer, let’s face it, you’ve got some frustrations in your life. You probably spend 10+ hours a day in Eclipse, which has been scientifically proven to make most sane people want to jump off bridges. Then you’ve got those Apple fan boys in your office always going on about how their tablet is bigger than yours. And as if that weren’t enough, once you get your app out the door you find that it’s greeted by the near infinite number of device permutations that handset manufacturers have been able to invent. The selection process resulting in this landscape of fragmentation can only be the market research equivalent of pulling the lever on a slot machine and waiting for the right keyboard size, screen dimension and geometric shape to pop up. Unfortunately, there’s not much you can do about the Apple fan boys in your office (legally anyway) but there’s at least some good tools to help with your fragmentation nightmares.

A few months ago I was bitten by the fragmentation bug, although at the time it felt more like a fragmentation tiger… You know, the kind that crashes your app on your customers’ devices but won’t reproduce on any of YOUR devices. Of course, those bugs that you can’t reproduce are damn near, if not completely, impossible to fix. In fact, even finding out that they even exist can be half the battle! At the time, I was working at a company where we had just updated our Android application to pull down bigger, shinier images to better showcase our product. These were stunningly beautiful images. The company had hired professional photographers. They were super hi-res, but scaled to just the right size to save on bandwidth. Even when you ran the app on tablet devices you had to strain to see the resolution break down. We were stoked to releasing something awesome.

The day after we went live with that super hi-res bundle of awesome we started noticing OutOfMemoryException’s showing up in our Airbrake logs. Lots of them! The first thing we did was try to reproduce the bug locally and no dice. No crashes on any of our devices! We were left with no choice but to pull it from the Marketplace (yes, this was before they changed the name to Play). Now I’ve got to give credit where credit is due at this point because I had no hand in actually fixing this particularly nasty bug. That was the amazing handiwork of a good friend and former co-worker named Justin Schultz. In fact, he wrote a great blog post about it here.

That said, how do you know you’ve actually fixed a bug that you can’t reproduce? Well, in all seriousness you don’t. You have to reproduce it. And that’s why we decided to give Testdroid Cloud a shot. We knew we couldn’t reproduce the bug on any of our devices but Testdroid Cloud has a huge farm of devices and our hope was that it would repro on at least some of theirs. The first step we took was to use Testdroid Recorder to setup a test that would navigate to the precise screen in our application that was most likely to cause the crash. Then we pushed the test out to Testdroid Cloud and waited for it to run. It took a while for the test to run on all the devices but we were able to watch the results start coming in. Then, sure enough, we started seeing OutOfMemoryException’s showing up in the device logs. We had our repro! The next step was for Justin to work his magic, implement a fix for the bug and re-test. The next run through the bug was gone! We repeated this experiment twice more just to feel confident before deploying back to production. We kept a close eye on our logs for the next few days but it quickly became clear that the fix had worked.

I highly recommend both Airbrake and Testdroid. There are a lot of crash reporting solutions out there but Airbrake has a super clean interface and it made finding this bug much easier and faster than it would have been otherwise. That directly translates into more happy people and fewer sad/angry people. Also, Testdroid is fantastic and they’re filling a genuine need in the Android ecosystem. There are just too many devices out there for developers to confidently build apps knowing which devices they’re going to work on in the wild. Their solution makes it possible to get real, actionable data that developers can use to improve the performance of their application. If we hadn’t used Testdroid we might have floundered with several unsuccessful fixes resulting in more crashes and unhappy users.