How to Get Started with Robot Framework (with Example)


Dear Testdroiders,

Robot Framework is a generic keyword-driven test automation framework for acceptance level testing and acceptance test-driven development (ATDD). Surprisingly, many of you have been using this framework (in the past) for a basic acceptance testing that extend the system level testing capabilities with specifications and test cases associated with the actual app testing. However, if you are not familiar with Robot framework or if you look for a way how to use it in Testdroid Cloud, I’ll walk you through of some basic things in this blog.

Screen Shot 2016-08-02 at 10.47.46 PM

What is Robot Framework and How Does It Work

Robot Framework is extremely easy to setup, use and modify to get both Android and iOS apps tested. The test syntax that it uses is based on keywords and these keywords are quick to edit and further configure to make a match with the application under test. Furthermore, testing capabilities provided by Robot Framework can be easily extended with the test libraries which can be implemented using Python, Java and even some other languages are supported.

The nifty thing with Robot Framework is how those mentioned keywords work. Users can quickly create new keywords, either using the existing example ones or by writing everything from the scratch.

If you look for more generic information about Robot Framework, there are plenty of great examples and online documentation in GitHub.

The Basic Setup of Robot Framework

To get started with Robot Framework basically you need Python and pip installed. If you have the environment properly configured you can just execute the installation command lines:

pip install robotframework
pip install robotframework-appiumlibrary

After you have installed Robot Framework, you can take a look at the example where we use our Bitbar Sample Application with very basic Robot Framework test examples. For the cloud run you basically need a Python script and shell script that makes sure the test execution on cloud end works fine. For instance, you need to add the pip installation in your shell script to ensure everything gets properly installed on your cloud session as well.

In this example, we’re using Appium as a basis to run the test and make sure everything gets logged for further inspection.

## Cloud setup
echo "Starting Appium ..."

/opt/appium/appium/bin/appium.js --log-no-colors --log-timestamp --command-timeout 120 >appium.log 2>&1 &

# Wait for Appium to fully launch
sleep 10

ps -ef|grep appium

echo "Extracting"

echo "Installing pip"
curl -O
python --user

echo "Installing requirements"
~/.local/bin/pip install -r ./resources/requirements.txt --target .

## Start test execution
echo "Running test"
pybot -x TEST-all tests/android_example.robot

This should be all what you need for a shell script. To create a compatible tests you need to add few lines to ensure all paths are recognized and test cases can be found from the right fileset. The file structure for test package can be something as follows:

Screen Shot 2016-08-03 at 6.39.26 AM

The following example includes both Android and iOS application in the test package (under resources->app folder). The test definition can be found under ‘tests’ and testing libraries are included under libs->examplelib. In addition, you can easily and quickly tweak and create your own test libraries with the instructions shown on the Github documentation of Robot Framework.

What Data Can Robot Framework Produce

When running a Robot Framework test in Testdroid Cloud users do not need to do any significant tweaks for their test runs (or job configs). The test automation and execution works the same way as with any other framework, and results are quickly available from any of those Android and iOS devices.

If you are starting out test automation with Testdroid Cloud and planning to use Robot Framework, simply create a project (Android or iOS), upload your application (APK or IPA), upload the test package, and select whatever devices you want to use for a test run. Just in few minutes the test run will be done (naturally this depends on the length of your test case) and the following data can be fetched from a test run:

Screen Shot 2016-08-02 at 10.04.31 PM

This example includes three basic test cases (we’ll come back to those in the next chapter of this blog). However, after the test run is finalized you have now all the data in one, compact view of Testdroid Cloud. If you click any of those files shown in overview you’ll get full log opened in the view below:

Screen Shot 2016-08-02 at 10.04.10 PM

For instance, the shell script file that we introduced in the first chapter of this blog extracted the, installed the PIP, setup other tools and finally executed the example tests. The output of the log can be also inspected in this view – and for example it will look something like this:

Imports done
Simple Smoke Test - Correct Answer | PASS |
Simple Smoke Test - Wrong Answer | PASS |
Simple Smoke Test - Wrong Answer 2 | PASS |
Tests.Example | PASS |
3 critical tests, 3 passed, 0 failed
3 tests total, 3 passed, 0 failed
Tests | PASS |
3 critical tests, 3 passed, 0 failed
3 tests total, 3 passed, 0 failed
Output: /home/ubuntu/test/output.xml
XUnit: /home/ubuntu/test/TEST-all.xml
Log: /home/ubuntu/test/log.html
Report: /home/ubuntu/test/report.html

In addition, to get comprehensive understanding of how the app (with test) really does on the physical device, you’ll also get performance data about the CPU and memory consumption:

Screen Shot 2016-08-02 at 10.04.21 PM

Okay, that’s some fundamentals for creating test runs using Robot Framework and managing the data after tests are done. Let’s take a look of how easy those test cases are to build and what you need to know about Robot Framework to use it efficiently with Testdroid Cloud.

How to Create Robot Framework Compatible Test Cases

First, some generic settings are defined for application and test session. Resource links to a file where all common procedures are provided in the test package. Test Teardown provides instruction of what system should do when test session is done:

*** Settings ***
Resource                 ...${/}resources${/}common.robot
Test Teardown            Close Application

For the variables, creator of a test should define some basic things here. In the Bitbar Sample Application there are name field (editText) and one button (button1). IDs and actions can be defined in this section:

*** Variables ***
${NAME}                  John Doe
${INPUT_NAME_FIELD}      id=com.bitbar.testdroid:id/editText1
${SUBMIT_BUTTON}         id=com.bitbar.testdroid:id/button1

The example application looks like this:

Screen Shot 2016-08-02 at 10.23.28 PM

Now, let’s look at those test cases (using Android). To create three different test cases for the application, we start with a simple smoke test where correctness of selection is validated. Correct answer in this scenario is when option to use Testdroid Cloud is clicked and name is entered in the text field. Two other test cases indicate the wrong answer with radio buttons (Option to buy 101 devices and ask mom to help you). Test cases are as simple as follows:

*** Test cases ***

Simple Smoke Test - Correct Answer
   [Tags] cloud
   Set Up And Open Android Application
   Input Name ${NAME}
   Select Option Use Testdroid Cloud
   Submit Selection
   Validate Correct Answer

Simple Smoke Test - Wrong Answer
   [Tags] 101
   Set Up And Open Android Application
   Input Name ${NAME}
   Select Option Buy 101 devices
   Submit Selection
   Validate Wrong Answer

Simple Smoke Test - Wrong Answer 2
   [Tags] mom
   Set Up And Open Android Application
   Input Name ${NAME}
   Select Option Ask mom for help
   Submit Selection
   Validate Wrong Answer

When test execution is started, Robot framework first parses the test data. After that it uses keywords provided with the test library/libraries to interact with the system.

In the keyword example, there is an input name, select of options (3 radio buttons) and submit of a selection (=a button press). In addition, keywords section can include validation of correctness. In the example below, selecting the right radio button uses xpath to find a name included in the actual UI element:

*** Keywords ***

Input Name
   [Arguments]      ${name}
   Input Text       ${INPUT_NAME_FIELD} ${name}
   Hide Keyboard

Select Option
   [Arguments]      ${option}
   Click Element    xpath = //*[contains(@text, '${option}')]

Submit Selection
   Click Element    ${SUBMIT_BUTTON}

Validate Correct Answer
   Wait Until Page Contains      Congratulations ${NAME}       5s

Validate Wrong Answer
   Wait Until Page Contains      Wrong Answer                  5s

Conclusion and Download Link

Robot Framework is very handy and easy to use. Some of you have been using it in the past so in case you have your tests implemented, and you have your Android / iOS app ready for test, give this framework a try at Testdroid Cloud. You’ll be surprised how smoothly things work and how quickly you can do acceptance testing. Furthermore, here is the full test package available for a download (and tweaks).

Have fun with Robot Framework at Testdroid Cloud!

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.