Setting up XCUITest on Bitbar’s Device Testing Cloud

Setting up XCUITest on Bitbar Device Testing Cloud

This is the last post in our XCUITest101 series. As of now, we have set up scalable XCUITest and run them on Continuous Integration server. The test executed in the CI server has used headless simulators, however, we also need to set our tests to be executed on real devices. There are many vendors in the market who provides the real devices in the cloud but Bitbar is the pioneer who started supporting XCUITest in the cloud before anyone else. In this post, we will see how we can use the Bitbar’s real device cloud to run our XCUITest tests on physical devices.

Bitbar Real Device Cloud 

Bitbar provides a mobile app testing solution with physical Android and iOS devices in the cloud, allowing mobile developers and tests to run automated and manual tests for mobile apps. They have added support to run XCUITest in the cloud in the early stages. There are other vendors who started supporting XCUITest once they released the UIAutomation is deprecated and other testing frameworks like Appium, Calabash either broke or needs more work to get supported by XCUITest API. Bitbar also provides an option for setting up the Virtual Private Cloud (VPC) so that companies can run the private cloud in their own network. In the case of executing XCUITest tests on the Bitbar real device cloud, you can sign up for the free trial and we will cover how to set up your XCUITest in this post. 

Preparing Your App and XCUITest Tests

Bitbar has comprehensive documentation to help you get started with running XCUITest tests on their platform. There is also support for crash reporting so that we can debug the crashes easily. I would strongly recommend you take a quick look at the documentation to understand the process in general. In a summary, it’s a two-step process, we need to create an .ipa file for our app and .zip file for our XCUITest bundle. It’s a very easy process and can be done via User Interface or Command line. Let’s explore both steps in details. 

Create an IPA Package

In order to create an IPA package for our app, we just need to build our XCUITest101 app for testing from Xcode. In the Xcode 10, select Product —> Build For —> Testing 

You can also do this by using xcodebuild and using build-for-testing option. You can find the details here

Once the build is finished, we have to extract the XCUITest101.app and XCUITest101UITests-Runner.app files from the Derived Data directory. Depending on where the derived data is stored, we can get those files. By default, it’s now stored in  ~/Library/Developer/Xcode/DerivedData, let’s copy these files from there to Desktop. (Note: You can copy and paste to any convenient location.)

Alternatively, from the Xcode we can go to the products XCUITest101.app right click and select “Show in Finder”, this will open the finder and we can copy XCUITest101.app and XCUITest101UITests-Runner.app and paste to Desktop. Now let’s run the following command to create an IPA file.

$cd ~/Desktop
$ zip --symlinks -qr "XCUITest101.ipa" XCUITest101.app/

After executing this command, you’ll see that this XCUITest101.ipa file is generated on Desktop. 

Create an XCUITest Zip File

Now that we have created the .ipa file for our app, let’s create a .zip file for the XCUITest bundle. Go to Desktop where XCUITest101UITests-Runner.app file is stored, right-click on the file and click on the “Compress XCUITest101UITests-Runner”.

You can do this by using a command line: 

zip -r -y XCUITest101UITest-Runner.zip XCUITest101UITests-Runner.app/

This will create an XCUITest101UITest-Runner.zip file. 

At this point, we have both XCUITest101.ipaand XCUITest101UITest-Runner.zipfile ready to run our XCUITest tests on Bitbar’s real device cloud. 

Run XCUITest tests in the Cloud 

Once you have created an account with Bitbar Cloud and signed into the platform, you can create different test runs to run automated XCUITest tests on various iOS devices available in the cloud. Bitbar has a comprehensive user manual which can guide you through all the available features of the platform. In order to run XCUITest tests, we can create a new project “XCUITest101” and set up a Test Run using Test Run Creator. Now we just need to select the XCUITest framework and upload the files we have created. 

There is also an option to select the devices of your choice and customize the test run with different parameters as mentioned in the user manual here. The test run will trigger XCUITest tests on the specified devices. Once the tests are finished, we will see the detailed reports of the tests.

Trigger Bitbar Test Run from CI Servers 

It’s very common that most of the iOS projects execute XCUITest tests from Continuous Integration servers, so it’s very important to trigger the tests against real devices from your CI server as well. Bitbar Cloud provides powerful REST API to support testing from CI servers. If you are using Jenkins, there is a Run-in-Cloud plugin which can be easily integrated with your Jenkins flow. You can use this plugin through Jenkins UI or the Jenkinsfile pipeline. If you are using other CI servers like Travis CI, CircleCI, Bitrise, Nevercode etc, we can write a custom trigger using the REST API.

In order to trigger the XCUITest tests, we need to get API key from Bitbar Cloud. The API KEY can be found in the My Account section. We can use REST API directly or use Ruby or Python client to trigger test runs. In our case, we can start a new project using 

$ curl -s "${CLOUD_URL}"'api/v2/me/runs' -H "Content-Type: application/json" -u "${API_KEY}": -X POST --data '{"osType":"'"${OS_TYPE}"'","projectId":"'"${PROJECT_ID}"'","frameworkId":"'"${FRAMEWORKID}"'","files":[{"id":"'"${APP_FILE_ID}"'", "action":"INSTALL"},{"id":"'"${TEST_FILE_ID}"'", "action":"RUN_TEST"}],"deviceGroupId":"'"${DEVICE_GROUP_ID}"'"}' | jq '. | if .id then .id else .message end'

You can read more about mobile CI workflow of how to use Bitbar API here

Conclusion 

Using an online device cloud like Bitbar Cloud for iOS test automation can be easy and cost-effective, since it comes with a lot of hassles and a large bill to acquire all the needed devices for dedicated test automation. We don’t need to code sign app to get XCUITest tests running on Bitbar Cloud. You can check out what devices and services Bitbar offers to make it an easy and painless process. Leave your comments below and share your experiences of using real device cloud to automate XCUITest tests.


An Essential Guide to XCTest Framework for iOS App Testing

Get all essentials about XCTest framework and learn how to get started with it for cloud testing

Download