Skip to content

We walked through the basic setup and installation of Calabash for Android and iOS in the previous blog, and this time we’ll take a look at how you can build Calabash tests for your mobile application. If you have followed the example showcased in How to Setup and Get Started with Calabash you have properly generated the file structure for your first Calabash tests on Android and iOS. There are lots of different (best) practices of how to create your Calabash tests, and we’ll use an example of a basis to illustrate how these can test templates can be generated and further developed.


This blog takes a look at what is the Calabash architecture and how to create tests using that architecture, what cucumber means in the context of test creation, and basically how to create your first Calabash test. We have an example application available for Android and iOS, plus you’ll find a link below where the test script can be downloaded. Furthermore, we’ll briefly discuss running tests on a cloud service that provides comprehensive results, data, logs, performance metrics, and highly sophisticated reporting, even with the actual Calabash logs. Don’t forget to download a copy of Calabash 101 to access it anytime.

Calabash Architecture for Test Creation

There are differences when it comes to Calabash running on Android and iOS. The basic test and test creation flow, however, remains the same. The recommended process to get test scripts done varies but the good rule of thumb is that you first create your features. All this information can be found in the example below and even example provides a basic example of the feature.

Then you test your feature and execute it to see what it actually does. Then you can create those Step Definitions. Step definition is basically a Ruby code that gets executed when you run your test. The following picture illustrated the relation between Feature, Scenario and Steps.

Screen Shot 2016-04-07 at 2.23.31 PM


A scenario specifies a single behavior or use case within a given feature that is comprised of various Steps. For example, the following scenario describes the behavior of ensuring that a credit card input field has the correct length of digits:

Scenario: Run whole app
  Given my app is running
  And I touch the "Start" button
  Then I take picture
  Then I press "More Info"

Steps usually begin with one of the keywords Given, When, Then, And, and But, however, they don’t have to, they can use * in place of those keywords. In fact, Cucumber does not distinguish between them (or *). They are instead meant to provide a language hint based on cause and effect to the stakeholders as to what is being described.

As such, simply recognizing their language implications are enough to use them effectively. However, for a detailed examination of these keywords, see the Cucumber Wiki entry on them.


A Feature is rarely defined by a single behavior. For this very reason, Scenarios can be grouped together logically under a Feature Definition. Feature definitions are typically given a name and an optional, short description. For example:

Feature: Test the entire app
Scenario: Log in to app
  Given my app is running
  Then I take picture
  Then I use the native keyboard to enter “” into the “your name” text field
  Then I use the native keyboard to enter “myPassword123” into the “password” text field
  Then I press "Sign In"
Scenario: As a valid user I can start using the app
  I wait for text "Hello"
  Then I wait for activity "HomeTabActivity"
  Then I press view with id "menu_compose_tweet"
  Then I enter text "Testdroid" into field with id "edit"
  Then I press view with id "composer_post"

Steps and Step Definitions

Step definitions are like code-behind for the scenarios defined in step definition scrips. They provide the glue that makes them runnable in the application. Their function is to translate readable texts into runnable actions. For example, step definitions for used example could be as follows:

require 'calabash-android/calabash_steps'
Then /^I set the screen to portrait$/ do
    perform_action('set_activity_orientation', 'portrait')
Then /^I set the screen to landscape$/ do
    perform_action('set_activity_orientation', 'landscape')
Then /^I hide the keyboard$/ do

We’ll take a look at steps “I set the screen to portrait/landscape” and “I hide the keyboard” examples below.

Calabash Architecture / Infrastructure – How Calabash Works on Real Devices

The great part of Calabash is that it is a cross-platform framework and works really well for both major platforms, Android and iOS.

On Android, Calabash uses the Android instrumentation test server (separated application) that is based on ActivityInstrumentationTestCase2. All Android tests are executed on the server-side and users can flexibly use predefined or custom steps. Looking at the picture below, Ruby client library translates Features and commands defined for those features to instrumentation calls. If the application uses Webview, the execution is injected through JavaScript.

With iOS, Calabash installs an HTTP server as an instrumentation package that then listens to commands from the Calabash server. Tests are also executed on the server-side and each test case is described in Cucumber. Slightly different than with Android, the Calabash iOS converts Cucumber commands to ‘FRANK’ method calls that then get executed. The Webview support is done the same way than with Android.



Getting Started with Test Creation

To save time, we’ll use an example application and ready test package. The file contains a functional test script. You can easily create a similar file set for using calabash-android gen command. If you are looking for Calabash-iOS Tests you can basically use the same test package but you need to fetch the .IPA file.

Screen Shot 2016-04-07 at 4.46.22 PM

As the application is relatively simple, we’ll start by creating a Feature, written in Cucumber. The possible scenarios for user interaction in the above example would be that user 1) clicks through the radio buttons, 2) types a name in input field, 3) hides (or after hiding, asks to show) the keyboard, 4) changes the orientation of the screen, plus asking test to take screenshots at any given moment.

Feature: Test UI Components
  Scenario: Test Radio Buttons
    Then I press view with id "radio0"
    Then I take screenshot
    Then I press view with id "radio1"
    Then I take screenshot
    Then I press view with id "radio2"
    Then I take screenshot
  Scenario: Input Text and Change Orientation
    Then I set screen to portrait
    Then I take screenshot
    Then I set screen to landscape
    Then I wait
    Then I enter text "Hello Calabash" into field with id "editText1"
  Scenario: Test Keyboard
    Then I take screenshot
    Then I hide keyboard
    Then I take screenshot
    Then I wait
    Then I show keyboard
    Then I take screenshot
  Scenario: Click Button
    Then I press view with id "button1"

If you look at my_first.feature it basically contains the same steps as shown break-down in the scenarios above.

Running Calabash Tests and Interpreting the Results

The reporting and test results are important for understanding how the application does under the testing. These reports and results should yield all fine-grained details about the test execution, performance, and provide users with actionable and rich information about tests and test runs. Furthermore, it’s important to get that test result data as real-time as possible, and those are stored based on each application regression, test executed against the specific version of the app, and that you have a way to compare results based on each app and test against the older ones.

First package your test zip file to include all files that have been either generated (e.g. calabash-android gen) and modified/added while you created the test. The folder structure (regardless if you have added new tests, steps, features, etc.) should look like this:

Screen Shot 2016-04-05 at 11.56.16 AM

After you have created a package of it, just log in to Bibar Testing and upload your APK or IPA, and this zip file. If you are not familiar with the interface, there is a full-blown guide on how to use real mobile devices on the cloud for testing. This guide will provide you with all necessary steps to access devices, get tests executed, and finally provides you with a comprehensive report of both, test details and Calabash report.

calabash logcat log

calabash logs

Naturally, when your Calabash test script is more complex you probably want to check also performance aspects of the app running on a specific device. The same view provides you the details of CPU consumption, memory allocation/deallocations, and even FPS metrics on how apps do.

Happy Calabash Testing Folks!


Ville-Veikko Helppi

Mobile Testing Product Expert