We’re back with a new guest blog! Today’s guest is Danny Preussler, an engineer for Groupon from their Berlin team. Groupon is a mobile-first company with excellent mobile know-how, with people on board like Michael Burton, the creator of Roboguice, and Carlos Sessa, the man behind one of the must-reads for every Android enthusiast – “50 Android Hacks”.
All right, so let’s read what sort of things Danny wants to share with us about mobile app testing – and using Bitbar Testing.
I am an engineer for Groupon in Berlin. Our team in Berlin is quite new and recently took the responsibility of an existing application – which naturally we didn’t want to break right away.
Testing is a very important part of our doings. As Groupon products are available in over 20 languages all over the world there must be serious testing done before any rollouts into app stores. In this context, we typically start with unit testing (with Roboelectric) to verify things – or as exploratory testing to understand functionality written by others. Then we move forward with integration testing: testing the app as a whole entity.
In most companies mobile app testing involves a lot of manual testing on real devices, and this can be hard to be replaced completely, but we wanted to reduce this effort as our team in Berlin was new and lack of QA resources. So, some kind of automated tool was needed. Groupon engineers have a lot of experience in mobile testing; i.e. they created “Robo Remote“ an open-sourced project based on Robotium (find out more here: http://www.meetup.com/
As we started from scratch in Berlin we had a look at the new Espresso, a framework announced at last years Google Test Automation Conference (GTAC) and just published some months ago. It aims at solving a lot of problems with Android test automation and actually writing tests with Espresso is very simple; the API is limited to a minimum but flexible enough to be extended.
As Test is normally just a single line there with the typical structure:
ViewMatcher -> ViewAction -> ViewAssert
Espresso also had the goal to fix the flakiness Android tests had before. There is no single WaitFor() method in the API. ViewActions are automatically performed on UI thread and before any checks with ViewAssertions are made it will be waiting until the application is idle (the UI thread is idle and all background tasks are finished) but no millisecond longer. This makes Espresso tests much faster than Robotium or simple InstrumentationTests.
No question about it – fast and stable tests are the foundation to work agile in mobile app development & testing. Our firsts Espresso tests where written in a few hours but where would we run them? They should be part of our continuous Jenkins build but would we run them on emulators?
That’s a typical way of doing; Jenkins supports a plugin that allows a matrix of devices to be created for testing. But the Jenkins that was currently in use was headless. No chance of running our emulator builds there.
So how about using real devices? The Jenkins was not running in our office but in a data center. Who would connect all these devices? Who can fix them if they have issues and need a reboot? Handling a device farm for test automation is a tough thing; even Google engineers admitted at GTAC that you should not try to do this. So leave this job to someone who does the work perfectly – and keep those devices up and running. As Bitbar announced the support for Espresso, we took a look at it.
The first manual upload to Bitbar Testing was very easy. Just pointing out the file and everything is ready. Thanks to the screenshot taking functionality of Bitbar Testing we found issues on devices with non-typical screen sizes. Instantly.
After this, it was time to add our Jenkins build. Bitbar Testing already provides a plugin for the new Android build system – Gradle. The only thing we had to do was to add the dependency:
apply plugin: 'testdroid
And change the instrumentation runner from normal Android JUnit Tests to the new Espresso runner (something you always have to do when using Espresso, if it’s in the IDE or the manifest file):
With a few more lines you can configure the devices that should be used and the language they should run on.
projectName 'Groupon Merchant'
For the future this easy language selection will help us test all the tens of languages – easily and instantly – before the rollout.
That’s all one has to do for letting the continuous build pushed to the cloud. Any build would be sent to the cloud and the developer will get notified about the result by email whenever the tests are finished.
Bitbar also offers a Java SDK with lots of samples hosted on Github. Those showed us how we could implement additional tasks as downloading JUnit results or screenshots after the tests.
The combination of Bitbar Testing with Espresso looks very promising for real device testing for us.