Groupon wanted to give their customers the best possible experience when using their mobile apps. A great experience is a fast experience. Achieving a fast experience required and still requires ongoing performance testing. At first, Groupon put in a purchase order for an infinite number of monkeys. But that was ultimately was denied. So they had to invest in test automation instead.
Instead of monkeys with stopwatches, Groupon used a combination of RoboRemote and Bitbar Testing. RoboRemote is a remote control test framework for Android that utilizes uiautomator and Robotium to control the application under test. RoboRemote sets application state closes or restarts the application, monitor android debug bridge (ADB) logs and performs other steps necessary for repeatable and reliable tests.
Frameworks like Robotium or uiautomator are great, but on their own, a Test Automator has a limited ability to do fun things like upgrading the app under test, clear app data or even reboot the device/emulator during a test. With RoboRemote (and other remote control frameworks) an Automator has a lot more freedom with direct access to ADB and the ability to run other commands on the host system. This flexibility comes in handy when Groupon needed to do something like setting initial application state for their performance tests.
Bitbar Testing provided Groupon the ability to run automation on real devices whenever they wanted. With the tools provided, Groupon could see useful information like CPU and memory usage after their tests concluded. They also got peace of mind, knowing the devices will be available and managed for them. No one has to go into a room with a wall full of devices just to reboot one that was behaving erratically.
“We always know that we can walk into the office in the morning and see the results of our test automation. In other words, Bitbar Testing gives us an efficient and reliable set of monkeys for our app performance test needs.” David van der Bokke, Manager of Mobile Test Engineering from Groupon’s Palo Alto team
Testing is a very important part of Groupon’s work. As Groupon products are available in over 20 languages all over the world there must be serious testing done before any rollouts into app stores. So Groupon’s engineers typically start with unit testing to verify things – or as exploratory testing to understand functionality written by others. Then they move forward with integration testing: testing the app as a whole entity.
In most companies mobile app testing involves a lot of manual testing on real devices, and this can be hard to be replaced completely, but Groupon wanted to reduce this effort as some of their teams were lacking in QA resources. So Bitbar was once again the answer.
Handling a device farm for test automation is a tough thing; even Google engineers admitted at GTAC that one should not try to do this. So leave this job to someone who does the work perfectly – and keep those devices up and running. That’s why Groupon chose Bitbar.
The first manual upload to Bitbar Testing was very easy. Group just pointed out the file and everything was ready. Thanks to the screenshot taking functionality of Bitbar Testing, Groupon found issues on devices with non-typical screen sizes – instantly.