Goodbye Handwritten Test Cases!

Bitbar, the mobile devops company. Logo, large

Since the dawn of time, humans have always been toolmakers. When we are faced with a problem, trivial or repeatable issue, we tend to seek out a tool that helps us overcome that pesky dilemma. If you think about it, many of the issues we face in software development aren’t very different and for us here at Bitbar, we like the fact that we create tools for humans to solve problems and make automated testing easier (and trivial).

A solid example of the man and tool paradigm is Testdroid Recorder. A trivial task and not necessarily so difficult to do by handwriting gets done automatically with Testdroid Recorder. The most significant difference is that handwriting that test case could easily take days, even weeks. But with Testdroid Recorder, the very same functionality gets recorded, tested and played back in minutes. This tool really does all the heavy lifting for you instantly with no need to reinvent the ancient and still used today, pulley!

PulleyBitbarFINAL

In this post in our series of Deconstructing Mobile Testing Methods, we’re deep diving into creation of test cases without writing any code (or pulleys!)

Get Ready – The Prerequisites

To be sure this test is done correctly, we need to have some basic components properly installed in your machine. Because we will do a small Android application testing exercise, we decided to use source code version of application. But it’s definitely okay to use just an APK to generate tests for your application. Here are some requirements and installations to get started:

1) You need to have Android SDK/ADT installed and at least one Android device for deploying and testing your application.

2) Use a real world example but just in case you don’t have your own project set up yet, we’ll use Open Sudoku which can be found from GitHub. So go ahead and download the following package:

=> https://github.com/romario333/opensudoku

3) Extract the package somewhere locally in your machine.

4) Then, install Testdroid Recorder. The instruction can be found from here or please follow the instructions on the video below:

5) All done – congratulations!
You’ve successfully installed one great tool that has helped so many humans overcome loads of problems related to writing test cases by hand.
And, guess what? You saved a lot of time.


Getting Started

With the following steps, we’ll make sure everything is ready for recording test cases in your environment. First, ensure the project you are about to test runs fine as Run -> Android application. As we’re using OpenSudoku here, this should work out of the box and no need to verify it. Follow the next steps:

1) Open Android SDK/ADT or Eclipse where you installed Testdroid Recorder.

2) Create a new workspace.

3) Import OpenSudoku to your workspace by right-clicking on Package Explorer view and select Import.

4) From General menu item select Existing project into Workspace. Click Next.

5) Click Browse, locate opensudoku-master folder from your local drive and then click Open.

6) Click Finish and you’ve successfully imported project to Project Explorer.

Recording Your First Automated Test

Check to be sure the device you want to use to record the test is connected to your computer or the system you’re using to run the test. Alternatively, you can run the tests on a virtual device. But, drawing from the experience we’ve gained, running tests can be very slow with the emulator and isn’t recommended. Running the same test, even on real device, through the cloud service is much faster!

1) Right-click OpenSudoku project in Project Explorer view and select New -> Other. You should see the following in the menu:

Screen Shot 2013-06-24 at 11.12.08 PM

 2) Select First Recording under Android – Testdroid menu item.

3) When the authorization window will be shown, insert your Testdroid Cloud credentials in order to authorize access to Testdroid services, and click Authorize.

4) In case you don’t have an account with Testdroid Cloud, click the link provided on page to get one.

Screen Shot 2013-06-24 at 11.17.14 PM

 5) Click Finish.

6) Select OpenSudoku on My Android Project menu. The test project with recorded actions will now generated for you (name of the project in Output project).

Screen Shot 2013-06-24 at 11.19.36 PM

 7) Click Next. The following screen will appear:

Screen Shot 2013-06-24 at 11.24.51 PM

 8) Press Record button. Now the wizard is launching your application project to your device and when you see Recording started… comment appear on the screen, you’re ready to go! Just use your application in the way you want it to be tested and the generated code is shown on the screen.

9) When you are finished, simply click Stop and Finish on the recording screen. The wizard takes care of creating a new test class for you, installing Robotium library and adding it to your Java build path.

What gets recorded?

After some play with OpenSudoku application the recorder has done all the hard work and followed user’s interaction with the application. Let’s take a look at some of pieces of test that was generated.

First, Testdroid Recorder uses extension of Robotium’s Solo library.

import com.bitbar.recorder.extensions.ExtSolo;

private ExtSolo solo;

The record execution starts with waiting the activity of the application:

solo.waitForActivity("FolderListActivity");

solo.sleep(4900);

assertTrue("Wait for button (text: Close) failed.",solo.waitForButton("Close", 20000));

solo.clickOnButton("Close");

Screenshot_2013-06-25-00-36-01

 After this user had pressed a close button after some delay.

Moving forward and the menu comes to the screen where user needs to select the difficulty level for the game:

solo.sleep(6800);

assertTrue("Wait for list (index: 0) failed.", solo.waitForList(0, 20000));

solo.clickInList(1, 0);

solo.waitForActivity("SudokuListActivity");

Screenshot_2013-06-25-00-14-49

After selecting the difficulty level, a list of 30 available Sudokus are shown.

solo.sleep(4200);

assertTrue("Wait for list (index: 0) failed.", solo.waitForList(0, 20000));

solo.clickInList(1, 0);

solo.waitForActivity("SudokuPlayActivity");

solo.sleep(7500);

assertTrue("Wait for button (text: Close) failed.", solo.waitForButton("Close", 20000));

Screenshot_2013-06-25-00-19-04

Selecting any of these will take user to play screen and if there for the first time, the Welcome screen will show up. User clicks Close and again another pop-up window will show up:

solo.clickOnButton("Close");

solo.sleep(1600);

assertTrue("Wait for button (text: Close) failed.", solo.waitForButton("Close", 20000));

solo.clickOnButton("Close");

solo.sleep(4000);

Two-Clicks

Recording gestures is also extremely useful. When the user performs any type of gesture, for example with multiple fingers, Testdroid Recorder records all the activity and generates the test code accordingly.

gestureGroup_1();

private void gestureGroup_1() throws Exception {

     float points[][] = { { 0.5852f, 0.2618f }, { 0.5880f, 0.2618f },

                      { 0.5880f, 0.2618f }, { 0.5861f, 0.2618f },

                      { 0.5861f, 0.2618f }, { 0.5861f, 0.2618f },

                      { 0.5861f, 0.2618f }, { 0.5861f, 0.2618f },

                      { 0.5861f, 0.2618f }, { 0.5861f, 0.2618f },

                      { 0.5861f, 0.2618f }, { 0.5870f, 0.2618f },

                      { 0.5954f, 0.2624f }, { 0.6102f, 0.2630f },

                      { 0.6528f, 0.2663f }, { 0.6889f, 0.2697f },

                      { 0.7231f, 0.2748f }, { 0.7565f, 0.2838f },

                      { 0.8019f, 0.3046f }, { 0.8537f, 0.3356f },

                      { 0.8620f, 0.3412f }, { 0.8620f, 0.3412f },

                      { 0.8991f, 0.3643f }, { 0.9167f, 0.3880f },

                      { 0.9222f, 0.4105f }, { 0.9185f, 0.4268f },

                      { 0.9120f, 0.4369f }, { 0.8991f, 0.4533f },

                      { 0.8870f, 0.4651f }, { 0.8722f, 0.4775f },

                      { 0.8639f, 0.4837f }, { 0.8602f, 0.4870f },

                      { 0.8593f, 0.4887f }, { 0.8593f, 0.4893f },

                      { 0.8593f, 0.4893f }, { 0.8593f, 0.4893f },

                      { 0.8593f, 0.4893f }, { 0.8593f, 0.4899f },

                      { 0.8593f, 0.4899f }, { 0.8593f, 0.4899f },

                      { 0.8593f, 0.4904f }, { 0.8593f, 0.4904f },

                      { 0.8611f, 0.4910f }, { 0.8611f, 0.4910f },

                      { 0.8611f, 0.4910f } };

                      solo.multiDrag(points);

     }

The user has pressed Back button in Android menu and this gets recorded as follows:

solo.sleep(3600);

solo.goBack();

solo.waitForActivity("SudokuListActivity");

Scrolling of menu is performed and gets recorded as follows:

solo.scrollListToLine(0, 11);

Taking a screenshot can be performed from Testdroid Recorder by pressing a “Take screenshot” button. The code gets generated as follows:

solo.takeScreenshot("cz.romario.opensudoku.test.FolderListActivityTest.testRecorded_scr_1");

Sometimes users need to click long on any UI element and for this purpose LongClick can be used:

solo.sleep(4900);

solo.clickLongOnScreen(solo.toScreenX(0.386f), solo.toScreenY(0.680f), 1722);

How to run recorded test?

To run your newly created test in Eclipse, use Run As -> Android JUnit Test. You should see your application being tested and the JUnit view shows results.

Screen Shot 2013-06-25 at 1.22.03 AM

Conclusion

In this example we recorded Robotium tests using project sources. However, you don’t have to have application sources to be able to record test – it’s possible to record tests using APK file. We recommend you build test cases that consist of many smaller parts, rather than one huge monolithic test case. Smaller units are easier to handle and it is easier to find out why the test has failed. Also, the tests is easier to manage during the development when the application goes through significant changes.

We’ll wrap up the series next week with ….sneaky aren’t we?

See you again in July!