Skip to content

Every now and then a new test automation framework emerges, it quickly gains attention from developers and testers. This is happening right now with the Detox framework for automating mobile tests.

A New Framework

Detox is a JavaScript mobile testing framework that is built into the application and the test execution starts with app launch. This makes test execution really fast and robust as no external additional tools are needed to orchestrate and synchronize during the test execution.

As for the popularity of Detox, it can be easily summarized.

  • First, it comes to the number of users. There are plenty of seasoned JavaScript web developers using libraries such as ReactJS, while experienced mobile programmers are scarce.
  • Some existing testing frameworks that looked still really promising a year ago are now fading out and users are forced to move to other frameworks. This also happened to Calabash which is no longer under development.
  • The current framework leader Appium originated from the web app testing and was later widely utilized in mobile app testing. It was great for testers as one could use the same concept in web testing with Selenium and in mobile testing against real devices or simulators. The problem with Appium is that launching tests is rather slow with installing and starting communication with the Webdriver.

Running Detox in Real Device Cloud

Bitbar Cloud has always been about openness, providing great freedom for end-users to choose how to test their application. This applies also to running Detox tests on Bitbar Device Cloud.

Of course, because of the current Detox limitations, you’ll need to run your iOS tests using simulators, until the support for real iOS devices is released by the Detox developers.

Follow these steps below to start an iOS simulator Detox test run on Bitbar Cloud. Before that, let’s check how your `` should look like.

  1. You should already have locally a `package.json` file listing your needed npm packages to use with your Detox test. You will need to include this file in your test package zip and upload it to Bitbar Cloud.
  2. Install all npm packages required to run your Detox tests: `npm install`. This uses your package.json file to select all the required packages.
  3. Next, you’ll need to install the Apple simulator with brew: `brew tap wix/brew` && `brew install applesimutils` as Detox is not supporting real iOS devices yet.
  4. Your script is now ready to start the app and test execution! `"${PWD}/node_modules/.bin/detox" test –configuration ios.sim.release –loglevel verbose > detox.log 2>&1`

Once everything is ready, we can move to Bitbar Cloud for test creation.

  1. On Bitbar, create a new Appium iOS Server Side project and name it as, for example, “Detox iOS”.
  2. Start a new test run:
    1. Upload a small dummy app. The testing framework requires you to upload some apps, even if this is not the app that you’ll be testing. The app to be tested is uploaded to the test suite zip.
    2. The test zip package needs to include everything required for running your test. This includes your app with built-in Detox tests and your `` that installs Node dependencies and launches the app under test.
  3. In test run’s advanced options increase the “Test time-out period” to 20 minutes. Installing the applesimutils takes a bit of time. We’ll include this as default in future releases.
  4. Start your test run!

Once your test is ready, you can find all the results in Bitbar Cloud, screenshots, video recording, logs and device performance to compare execution between devices. One thing worth noting, when executing tests on simulators, you won’t get the correct performance or video recordings.

Here’s a working sample made by Jarno from our Customer Success team if you want to get your hands dirty:

Detox Tests Console Log The Console.log for Detox tests on Bitbar Cloud

Final Thoughts

Test automation has historically been thought through a manual testers’ point of view. It should be possible to interact with the application as it would be used by a real tester. This is the reason for the success of current mainstream frameworks (eg. Appium). But is this really a valid point still today with DevOps and the “shift left” concepts? With developers ever more engaged with creating and maintaining test suites, why not make use of the internal workings of the application when testing it?

This is what Detox is doing, moving away from black-box testing to gray box testing, where developers make use of the application’s internal state to drive tests. If you have Detox tests already and are in need of a mobile device cloud, you can already try to automate the tests through our free trial here.


Niko Cankar

Bitbar Product Owner