We’re back with a new guest blog! Today’s guest is Danny Preussler, an engineer for Groupon from their Berlin team. Groupon is a mobile-first company with excellent mobile know-how, with people on board like Michael Burton, the creator of Roboguice, and Carlos Sessa, the man behind the one of the must-reads for every Android enthusiasts – “50 Android Hacks”.
Allright, so let’s read what sort of things Danny wants to share with us about mobile app testing – and using Testdroid.
I am an engineer for Groupon in Berlin. Our team in Berlin is quite new and recently took the responsibility of an existing application – which naturally we didn’t want to break right away.
Testing is very important part of our doings. As Groupon products are available in over 20 languages all over the world there must be serious testing done before any rollouts into app stores. In this context, we typically start with unit testing (with Roboelectric) to verify things – or as exploratory testing to understand functionality written by others. Then we move forward with integration testing: testing the app as a whole entity.
In most companies mobile app testing involves a lot of manual testing on real devices, and this can be hard to be replaced completely, but we wanted to reduce this effort as our team in Berlin was new and lack of QA resources. So, some kind of automated tool was needed. Groupon engineers have a lot of experience in mobile testing; i.e. they created “Robo Remote“ an open sourced project based on Robotium (find out more here: http://www.meetup.com/
As we started from scratch in Berlin we had a look at the new Espresso, a framework announced at last years Google Test Automation Conference (GTAC) and just published some months ago. It aims on solving a lot of problems with Android test automation and actually writing tests with Espresso is very simple; the API is limited to a minimum but flexible enough to be extended.
As Test is normally just a single line there with the typical structure:
ViewMatcher -> ViewAction -> ViewAssert
Espresso also had the goal to fix the flakiness Android tests had before. There is no single WaitFor() method in the API. ViewActions are automatically performed on UI thread and before any checks with ViewAssertions are made it will be waiting until the application is idle (the UI thread is idle and all background tasks are finished) but no millisecond longer. This makes Espresso tests much faster than Robotium or simple InstrumentationTests.
No question about it – fast and stable tests are the foundation to work agile in mobile app development & testing. Our firsts Espresso tests where written in a few hours but where would we run them? They should be part of our continuous Jenkins build but would we run them on emulators?
That’s a typical way of doing; Jenkins supports a plugin that allows a matrix of devices to be created for testing. But the Jenkins that was currently in use was headless. No chance in running our emulator builds there.
So how about using real devices? The Jenkins was not running in our office but in a datacenter. Who would connect all these devices? Who can fix them if they have issues and need a reboot? Handling a device farm for test automation is a tough thing; even Google engineers admitted at GTAC that you should not try to do this. So leave this job to someone who does the work perfectly – and keep those devices up and running. As Testdroid announced the support for Espresso, we took a look at it.
The first manual upload to Testdroid was very easy. Just pointing out the file and everything is ready. Thanks to Testdroid’s screenshot taking functionality we found issues on devices with non-typical screen sizes. Instantly.
After this, it was time to add our Jenkins build. Testdroid already provides a plugin for the new Android build system – Gradle. The only thing we had to do was to add the dependency:
apply plugin: 'testdroid
And change the instrumentation runner from normal Android JUnit Tests to the new Espresso runner (something you always have to do when using Espresso, if it’s in the IDE or the manifest file):
With few more lines you can configure the devices that should be used and the language they should run on.
projectName 'Groupon Merchant'
For the future this easy language selection will help us test all the tens of languages – easily and instantly – before the rollout.
That’s all one has to do for letting the continuous build pushed to the cloud. Any build would be send to the cloud and the developer will get notified about the result by email whenever the tests are finished.
Testdroid also offers a Java SDK with lots of samples hosted on Github. Those showed us how we could implement additional tasks as downloading JUnit results or screenshots after the tests.
The combination of Testdroid with Espresso looks very promising for real device testing for us.