Setting up XCUITest for iOS Continuous Integration

So far in the XCUITest 101 series, we have written some XCUITests and explored the XCUITest API. The next step is to run them for continuous testing so that we can get instant feedback on our latest code. The practices of Continuous Integration allow us to schedule and run the test on a regular basis or whenever there is code change. In this post, we will explore how to use XCUITest tests on a continuous integration server.

XCUITest Command Line Execution

In order to set the XCUITest tests on the Continuous Integration Server, we need to be able to run XCUITest tests from the command line. There are various ways to do this. The native tools xcodebuild can be used for running XCUITest tests or we can use Fastlane scan tool for executing tests from command line. We will cover both methods briefly. Before we schedule the test from the command line, we should know the following things about our Xcode project. In our XCUITest101 app, we have the following things:

XCUITest Scheme: The scheme that has been configured for running XCUITest tests is XCUITest101

It’s very important to make sure the scheme is executing the XCUITest target and it’s shared. In our case it looks perfect: 

Xcode Project/Workspace: We don’t have Xcode workspace but our Xcode project XCUITest101.xcodeproj/ which can be used for the command line argument. 

Now we have all the required information to set up XCUITests from the command line. 

Using xcodebuild Tool 

Apple’s native xcodebuild tool can be used for analyzing, building and testing iOS apps from the command line. The full command will look like this: 

xcodebuild -project XCUITest101.xcodeproj/ -scheme XCUITest101 -destination 'platform=iOS Simulator,OS=12.0,name=iPhone X' clean build test CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO

In the command above, we have passed various parameters required to execute the XCUITest tests from the command line. There are so many other parameters available for the build to test or test without the building. Please refer the Apple’s official documentation on xcodebuild for the details. After executing this command, we will see that the tests have been executed and test results are logged on the console. 

Using Fastlane Scan 

Another easy approach to executing the XCUITest tests from the command line is by using the Fastlane scan tool, in which we don’t need to write the long commands. However, we need to install Fastlane tools using Rubygem. Let’s install Fastlane tools like this: 

$ gem install fastlane

This will install all the dependencies required for running the XCUITest tests from the command line. Once the installation is successful, we can use the following command to execute the XCUITests from the command line.

$ fastlane scan --scheme XCUITest101

This command will execute all our XCUITest tests from the command line and generate test reports in the test_output directory. Reports are generated in HTML and XML format, which can be easily configured on the CI servers.

Once we are able to execute the tests from the command line, then we can stick this command on any CI server. The CI server can be in-house or cloud-based. 

Setup the CI Server 

There are various services available in the market that can be used for iOS continuous integration. The most common CI services being used for iOS app testing are Jenkins, TeamCity, Xcode Server as in-house CI solutions. Services like TravisCI, CircleCI and BuddyBuild, Bitrise and Nevercode can be used as cloud-based CI servers. We can use any free service but for this post, we will use TravisCI which gives you free services for all open-source project on Github. It would be handy if you already have a GitHub account and an XCUITest project on Github, then we can sign up for TravisCI services using the Github credentials. TravisCI will guide you through the process of enabling CI service for your project. Basically it’s a straightforward process – all you need to do is to sync GitHub account which will display all of our repositories and then you need to add .travis.yml  file at the root of the project. Our configuration file will look like this: 

language: objective-c
osx_image: xcode10
branches:
  only:
    - CI
    - master
before_install:
  - gem install fastlane --no-ri --no-rdoc --no-document
install: true
script:
  - fastlane scan -s XCUITest101

This configuration will use an Xcode 10 image to perform the build. Note that it installs the Fastlane and executes the script mentioned above. Once the build is finished, we can see the sample output as below:

The entire build output can be found on the TravisCI here

Note: The Source Code for this demo is available on the XCUITest101 Github repo in the CI branch. 

Conclusion 

With minimum configuration, we now have managed to set up the CI server for our XCUITest tests to get continuous feedback of our iOS application. The Travis CI service is just an example but we can set up XCUITest tests on any available CI server in the world. We can even trigger XCUITest server to Bitbar’s real device cloud. We will cover that in the next post. 


Manifesto: Everything about Mobile DevOps

Learn how to properly adopt DevOps approach for your mobile team and get the most out of it.

Download

  • Farrukh Naeem

    Thanks for this awesome article @Shashikant . What if I want to run some specific test which are in a separate class or under a separate folder?