How to Launch Test Automation Runs from Command Line

Bitbar API

Many developers work on their console view significant portion of the day. Getting something done quickly and efficiently is one of those benefits that working directly from command line provides for our every day lives. Even test automation test scripts can quickly be started from the command line and in this blog we’ll take a look at how to get this done quickly and efficiently just using a bash / command line.

Don’t forget to read it entirely! There is a really nice helper script included in this article that will surely help you with your Appium testing!

REST API Access to Real Devices on Cloud

Since the launch of Testdroid Cloud (Bitbar Public Cloud nowadays) the API access was an absolute must for many of test automation users. It basically provides an instant way to launch test runs, manage all aspects of projects, devices (groups of devices) and fetch back test results with rich dataset. This way users were able to get test runs started and see the progress on their local environments, plus integrate any of their development and test management tools with our device farm of real Android and iOS devices.

The use of continuous integration tools were also integrated with the API and efficiency after each and every build from build to test was enormously improved. If you are looking for a way to integrate continuous integration tools with real devices on cloud, take a look at our REST API documentation.

The Requirements for the use of API

The REST API provided for Bitbar Public Cloud is extremely easy to adopt and use. The full API reference guide provides all information about your Bitbar account management, projects, test runs, devices, framework supports and bunch of other great features that makes test automation effortless with physical devices on the cloud service.

  • Bitbar Testing Account – You need an account in Bitbar Public Cloud to test your Android and iOS apps on devices. Everything presented in this blog can be also done manually (e.g. manual testing with remote access to devices). If you don’t have an account, you can create an account here.
  • API Key – Once you have logged into service, you can get your personated and unique API key from the top right menu (Profile -> My Account -> Access via API):

API key access to Bitbar Public Cloud

  • Device Group(s) – To get your tests running on certain devices, you should create a device group where those devices are identified. The easiest way to create a device group is landing on the service, dragging and dropping desired devices on a device group. After this, you can use these devices simply based on device group ID.

Different Options for Command Line Utilities

Bitbar Public Cloud doesn’t limit what bash or command line tools (or programming languages) you want to use. The light-weight API takes care of all gateway management, accessing devices and managing test runs and projects for you.

Project Creation with REST API

Let’s look at the simple example of how to use cURL to authenticate your access, and use POST to access a project or create it:

$ curl -H "Accept: application/json" -u <API_KEY>: -X POST -d "type=<PROJECT_TYPE>" -d "name=<project name>" https://cloud.testdroid.com/api/v2/me/projects

For the you should copy-paste your unique API key and for the you could set any of existing project types with Bitbar Public Cloud. This example creates an Appium Server-Side project under your account:

$ curl -H "Accept: application/json" -u M4vN5mVKFewM9jPK0cJw9t2NgiZjOf4J: -X POST -d "type=APPIUM_ANDROID_SERVER_SIDE" -d "name=TestAppiumProject" https://cloud.testdroid.com/api/v2/me/projects

As a response message you’ll get something like this:

{"id":106673414, "archivingItemCount":120, "archivingStrategy":"DAYS", "common":false, "description":"", "name":"TestAppiumProject", "sharedByEmail":"ville-veikko.helppi@bitbar.com", "sharedById":6105119, "type":"APPIUM_ANDROID_SERVER_SIDE", "createTime":1480503902000, "archiveTime":null, "frameworkId":541, "archivingStrategyDisplayValue":"120 days"}

And if you look from Bitbar Public Cloud, the API call has created a project under your project listing:

project view at bitbar public cloud From the JSON response you can parse the project ID (106673414, in this example) for further use. More advanced version (and very cool way) to get only the project ID as a response would go as follows:

$ curl -H "Accept: application/json" -u M4vN5mVKFewM9jPK0cJw9t2NgiZjOf4J: -X POST -d "type=APPIUM_ANDROID_SERVER_SIDE" -d "name=TestAppiumProject" https://cloud.testdroid.com/api/v2/me/projects  | sed -e 's/.*id\":\([0-9]*\).*/\1/'

Verifying Device Groups

You can nicely pull down the list of all device groups by using the following command:

$ curl -H "Accept: application/json" -u <API_KEY>: https://cloud.testdroid.com/api/v2/device-groups

It’s always a good idea to verify that device group exists before trying to initiate test runs on devices included in a specific device group. This can be done quickly with a following command:

$ curl -s -H "Accept: application/json" -u <API_KEY>: https://cloud.testdroid.com/api/v2/me/device-groups/<DEVICE_GROUP_ID>

As a response you’ll get list of device groups that are available for your user account. For instance, device group ID 14 is the free device group with Android devices. The response will look like this:

{"id":14,"creditsPrice":0,"deviceCount":3,"displayName":"Free Android devices","name":"free-android-devices","osType":"ANDROID","userId":null}

Launch Appium Server-Side Tests from Command Line

It’s very typical use case that users want to launch their Appium tests from local environment, but pushing both app and test package into Bitbar Public Cloud using a command line.

For the Appium Server-side test runs, the tests need to be uploaded as a zip package, where at the root of the zip is the “run-tests.sh” script that calls the Appium tests as on your local computer. Note that “run-tests.sh” is different for iOS and Android tests. Example “run-tests.sh” scripts for Android and iOS can be found at Github.

In addition, you can find few handy helper scripts from Github that assist you with creating and packaging tests. For example, “launch-tests.sh” helps you to get test run started. “createiOSZip.sh” and “createAndroidZip.sh” will quickly zip a test package for you. The following line will upload your test for a specific project:

$ curl -H "Accept: application/json" -u <API_KEY>: -X POST -F "file=@<ZIP_FILE>" "https://cloud.testdroid.com/api/v2/me/projects/<PROJECT_ID>/files/test"

You can also check if the test run is finalized by using a command line:

$curl -s -H "Accept: application/json" -u <API_KEY>: "https://cloud.testdroid.com/api/v2/me/projects/<PROJECT_ID>/runs/<TESTRUN_ID> | sed -e 's/.*state\":\([A-Z"]*\).*/\1/'

And when it is done, you can pull all the results back to your local environment for further inspection:

$ curl -s -H "Accept: application/json" -u <API_KEY>: -X POST "https://cloud.testdroid.com/api/v2/me/projects/<PROJECT_ID>/runs/<TESTRUN_ID>/logs.zip

NOTE! The test run logs are not available instantly after test run is finalized, but you might need to wait a second for them up to show up!

One Command for Appium Test Run Launch

At the very same Github repository, you can find a launch-tests.sh file. We’ve created this to help you to launch a test run just using one line on shell. Here is an example:

$ bash launch-tests.sh -a -p "My Android Server Side Appium" -k M4vN5mVKFewM9jPK0cJw9t2NgiZjOf4J -g "HTC+Huawei" -f ../../../apps/builds/BitbarSampleApp.apk

The usage and parameters for launch-tests.sh are as follows:

-a/i – defines whether you are starting your test run on Android or iOS devices (and what type of device group you are trying to use)

-p <PROJECT_NAME> – can to choose a specific profile. If not given, a new project will be created.

-g <DEVICE_GROUP_NAME> – in the above example there is a “HTC+Huawei” device group on my settings. It’s worth noticing that you don’t have to use a device group ID but the name will work just fine.

-k <API_KEY> – API key for authentication and access between your local environment and Bitbar Public Cloud back-end.

-t – for creating and uploading a new test zip file.

-f <APP_FILE_PATH> – for uploading a new app file.

This script has helped many developers to get their Appium tests started quickly and only using command line. We hope it will help you too!

Happy Testing Folks!

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close