How to Build Robust and Reliable Automated Tests

Bitbar, the mobile devops company. Logo, large

What is the main difference between a successful and not-so-successful application or game available on Google Play? Sure, many things make a difference, but one great example is arcade car games. You can find hundreds – if not even thousands on Google Play. Why are some downloaded 100M+ times and some only thousands of times? Because of graphics? Gameplay? Features? Recommendations?

At the end of the day all of these are ‘nice-to-have’ characteristics for your application, but none of these are actually as important as robustness and reliability of that specific application on a range of different Android devices. ‘Crash’ and ‘not working’ are the most common feedback at Google Play for unstable applications/games, and this makes the next hundred million users skip that lousy one.

In our second post in our series, Deconstructing Mobile Testing Methods” , we’ll build a simple test case for one of the most used application on Android – the Twitter application. We’ll construct some basic test cases using Robotium and uiautomator to illustrate how application can be tested and easily verified that it works on more than 200+ devices.



For a reliable and continued success of application, developers should take testing more seriously, especially when their app will be targeted for global market. The localization of apps is very important and therefore, we’re swapping here for one of the most universal language for you guys, the Java.


package com.bitbar.testdroid.test;

// Array lists are the best choice if data has a known number of elements or
// small fixed size upper bound. It automatically expands as data is added.
import java.util.ArrayList;

// Activity, a single focused thing that the user can do. Almost all activities
// interact with the user, so Activity class takes care of creating a window.

// This class provides functional testing of a single activity.
import android.test.ActivityInstrumentationTestCase2;

// Basic building blocks for user interface with the widget packages.
import android.view.View;
import android.widget.EditText;
import android.widget.TextView;

// The Robotium Solo class – all-you-can-eat class for everything

The Robotium Solo is all-you-can-eat type of super class containing all the methods you need to test. It supports test cases that span over multiple activities, supports regular expressions and will automatically do user interface (UI) tricks when needed. Robotium Solo takes care of most expected new activities which requires less planning for test cases. Also, Robotium Solo can be used in parallel with ActivityInstrumentationTestCase2. The test cases are written from a user perspective and technical details are not needed.



// The UiDevice class provides access to state information about the device. This can be
// also used to simulate user actions on the device, such as Home and Menu button presses

// Representation of user interface elements

// Mechanism for tests to describe the UI elements

// UiDevice instance and UI test cases should extend this class

The uiautomator API line-up is very straightforward, clean and simple while providing a versatile set of classes for developers to use. These classes, with interfaces and exceptions, allow developer to capture and manipulate UI components on the target application very easily.


In Robotium, ActivityInstrumentationTestCase2 class can be used to test several Activities. The ActivityInstrumentationTestCase2 allows use of the full Android system infrastructure and the functional tests for an activity can be written using this class. The communication with the Android infrastructure is done via the Instrumentation class which can be access via the getInstrumentation() method. ActivityInstrumentationTestCase2 starts the activity in the standard Android context, similar as if a user would start the application.


// Named class extended with
public class NewRobotiumTest extends ActivityInstrumentationTestCase2

    // Static constant for launcher's activity classname
private static final String LAUNCHER_ACTIVITY_CLASSNAME = "";

    // The representation of ordinary class found in the class hiearchy
private static Class<?> launchActivityClass;

    // Try-catches for com.bitbar.testdroid.test.NewRobotiumTest.
static {
try {
launchActivityClass = Class.forName(LAUNCHER_ACTIVITY_CLASSNAME);
} catch (ClassNotFoundException e) {
throw new RuntimeException(e);

    // Define Solo
private Solo solo;

    // Let the compiler know what we are doing here
public NewRobotiumTest() {
super("", (Class) launchActivityClass);

    // Override annotations
public void setUp() throws Exception {
solo = new Solo(getInstrumentation(), getActivity());

public void tearDown() throws Exception {

    // Method used to find View specified by its id
protected View findViewById(String id) {
return findViewById(solo.getCurrentActivity().getResources().getIdentifier(id.replaceAll("", ":id/"),
null, null));

    // Finds a view that was identified by the id attribute from the XML
protected View findViewById(int id) {
View view = solo.getView(id);
if (view != null)
return view;

        ArrayList views = solo.getViews();
for (View v : views) {
if (v.getId() == id) {
return v;
return null;

The FindViewByID function provides an enhanced View finder. It first tries to find View from Activity, then from all Views under ViewRoot. For comparison, uiautomator doesn’t require much at this stage.


public class NewuiautomatorTest extends UiAutomatorTestCase


One of the most interesting differences to developer using Robotium or uiautomator is the use of R class. The R class used in the Robotium example is a special class generated automatically at build time that provides user-accessible references to resources in terms of static, final constants of the R class. The R class is managed automatically by the Android plugin for Eclipse.

Screen Shot 2013-06-04 at 9


// Public void for the operation
public void testRecorded() throws Exception {

    // Wait for the text 'Hello.' to be shown for newbie
if (solo.waitForText("Hello.")) {

        // R class ID identifier for 'Sign in' - and click it

            // R class ID identifier for entering username
solo.enterText((EditText) findViewById(""),"username");

            // R class ID identifier for entering password
solo.enterText((EditText) findViewById(""),"password");

            // R class ID identifier for clicking log in

            // Wait until log in is done

        // Activate the text field to compose a tweet
solo.clickOnView((TextView) findViewById(""));

        // Type the tweet
solo.enterText((EditText) findViewById(""), "Testdroid is awesome!");

        // Tweeting!

As mentioned, is the dynamically generated class, created during the build process to dynamically identify all assets (from strings to widgets and layouts). Note that this is Android-specific and doesn’t have much to do with Java language constructs.

The uiautomator, on the other hand, uses more descriptive, actual texts. What can be done with R classes in Robotium, can be done easily in uiautomator with resource IDs based on labels, texts, descriptions and so on.


// Public void for the operation
public void testSignInAndTweet() throws Exception {

    // Starting application:

    // Press Home button to ensure we're at the beginning/homescreen

    // Select 'Apps' and click button
new UiObject(new UiSelector().description("Apps")).click();

    // Select 'Twitter' and click button
new UiObject(new UiSelector().text("Twitter")).click();

    // Locate and select 'Sign in'
UiSelector signIn = new UiSelector().text("Sign In");

    // If button is available, click it
UiObject signInButton = new UiObject(signIn);
if (signInButton.exists()) {;

        // Set the username
new UiObject(new UiSelector().className("android.widget.EditText").instance(0)).setText("username");

        // Set the password
new UiObject(new UiSelector().className("android.widget.EditText").instance(1)).setText("password");

        // Click 'Sign in' button
new UiObject(new UiSelector().className("android.widget.Button").text("Sign In").instance(0)).click();

        // Wait Sign in progress window
getUiDevice().waitForWindowUpdate(null, 2000);

        // Wait for main window
getUiDevice().waitForWindowUpdate(null, 30000);


    // Starting a new tweet:
new UiObject(new UiSelector().description("New tweet")).click();

    // Typing text for a tweet
new UiObject(new UiSelector().className("android.widget.LinearLayout").instance(8)).setText("Awesome #Testdroid!");

    // Tweeting!
new UiObject(new UiSelector().text("Tweet")).click();


In order to take a screenshot of current window and store it as PNG Default scale of 1.0f (original size) and 90% quality is used. The screenshot is adjusted per screen rotation for both of the testing methods.

Robotium and uiautomator

// A screenshot of current window and store it as PNG


Do not overlook the performance aspect of your mobile application lifecycle. Whether your application is native, web-based or hybrid, you need to test for performance on various devices, platforms and with various configurations to ensure optimal user experience for people downloading your application. Failing to address these aspects will inevitably result in brand degradation, business losses, and costly R&D iterations to fix.

We took these tests and ran those on more than 200 Android devices. Just at a glance, here are some results from two popular free devices at Testdroid Cloud, Sony Xperia Z C6303 and Alcatel One Touch 983:

Screen Shot 2013-06-11 at 9.38.44 AM


As shown in examples, both of these frameworks can do the job for Twitter application. But as applications are becoming more complex, the way more important question – again – is that how efficiently can those be used and how well those fit to your development and testing work. Well, that’s the topic we are diving into next week, so see you again right here! All comments are more than welcome!