Appium vs. XCUITest for Automated iOS Testing

  November 06, 2019

Thorough testing of your app must be on your team’s agenda to make your iOS app a success. Especially UI tests help you ensure your app looks the way it should on different devices and operating system (OS) versions. Following many Appium tips and the idea of our previously published article about Appium vs. Espresso, we’ll discuss here, which framework works better for automated iOS testing: Appium or XCUITest?

Meet Appium – the All-Rounder

Appium is an open-source test automation framework with an active community behind it. You can use it to test your native, hybrid, and mobile web apps, which is one reason why it’s so popular with both app developers and QA engineers. Using the WebDriver protocol, it drives iOS, Android, and Windows apps. Does this sound familiar to you? In fact, Appium uses the Selenium WebDriver API, with which you may have already tested web applications.

Appium’s Advantages
1. It’s language agnostic

Basically, Appium is a Node.js server waiting for commands to be sent via a REST API. To work with this API, you can use different client libraries, depending on the tech-stack you want to use. Consequently, Appium supports all WebDriver-compatible languages, such as Python, Java, PHP, C#, and Ruby. Depending on which OS you want to test, you need to install and configure the corresponding implementation of WebDriver – for iOS, this is the XCUITest Driver.

2.Reuse test cases across platforms

Unlike XCUITest, Appium supports testing on multiple platforms, such as Android. If you already manage a single codebase for your app – e.g., because you use React Native or Apache Cordova to develop your product – it makes sense to use Appium to do the same for your test code.

3. Benefit from your experiences with Selenium

Since Appium is based on WebDriver, we can instantly see the similarity with Selenium – one of the most popular test automation frameworks for web applications. That’s why using Appium is a great tool for QA engineers, as it’s straightforward to use for anyone who worked with Selenium at some point.

Appium’s Disadvantages
1. Test runs are rather slow

Being the all-rounder among the mobile test automation frameworks comes with a price tag. Due to its architecture supporting multiple platforms and technology stacks, the tests are rather slow. As you can imagine, it takes time to send your commands to a server, which then translates them into platform-specific actions!

2. Your UI tests may lack precision

Your testers need to be able to trust test results, right? Therefore, it is unfortunate that Appium UI testing can be flaky, i.e., the same test may fail or pass with the same setup and conditions.

3. It can be hard to setup

Especially for the inexperienced, Appium can be hard to configure locally. Using XCUITest with XCode is much more straightforward. However, this is because Appium is a third-party framework and not officially supported by Apple. That being said, even though setting things up can be a bit challenging, with proper guidance, it can be a breeze!

4. You may run into compatibility issues with XCode

Appium is a third-party framework (it’s not native to XCode), but it relies on the API provided by XCUITest. For this reason, Appium may not work with the latest version of XCode after you download it. Behind the framework is an active community that strives to keep it up-to-date. Nevertheless, it takes some time until errors that have occurred after the update have been resolved.

Appium Sample Test:
driver.findElement(By.id("com.example.app:id/radio0")).click();
driver.findElement(By.id("com.example.app:id/radio1")).click();
driver.findElement(By.id("com.example.app:id/radio2")).click();
driver.findElement(By.id("com.example.app:id/editText1")).click();
driver.findElement(By.id("com.example.app:id/editText1")).sendKeys("Simple Test");
driver.findElement(By.name("Answer")).click();
// or alternatively with
driver.findElement(By.id("com.example.app:id/button1")).click();

Meet XCUITest – the iOS Specialist

XCUITest is Apple’s native UI testing framework and is built on top of XCTest. Most app developers are already familiar with XCTest from writing unit tests for their classes and components. It runs inside the XCode IDE, and testers can write tests in either Objective-C or Swift, making it the perfect match for any iOS developer.

Let’s talk about its pros and cons, advancing in our Appium vs. XCUITest for automated iOS testing discussion.

XCUITest’s Advantages
1. It executes iOS tests faster

XCUITest gets developed by Apple to test iOS apps with Swift or Objective-C in the XCode IDE. This means the framework must concentrate on doing one job and doing it exceptionally well: testing iOS apps. The framework’s architecture was built so that tests can be performed quickly and reliably – often it’s favorable to do less but to do it better than others.

2. You can trust test results

After reading the above, it becomes clear why XCUITest can deliver more consistent test results than Appium. Thanks to its streamlined architecture, the framework provides reliable test results with few to zero flakiness.

3. It’s straightforward to set up and maintain

Nobody and nothing is perfect, and maybe you can find bugs in XCode or XCUITest after updating the tool to the latest version. Something like that can always happen. In general, however, you don’t need to worry about XCUITest having compatibility problems with XCode since they’re both developed under Apple’s release standards.

Additionally, the setup process is far more comfortable than with Appium – learn more in our guide.

4. Have everything in one place

Because XCTest is part of the XCode IDE, your iOS developers can create their own tests more straightforwardly. For example, they can check whether UI tests still pass after changing the app’s code without a hassle. Using Apple’s native testing framework allows you to have your app’s source code and test code in one place. In this way, you can maintain your codebase with more ease and practice less complicated workflows.

XCUITest’s Disadvantages
1. It’s obviously language-specific

You can only write XCUITest test cases using the programming languages Swift or Objective-C. Consequently, this framework fits perfectly with your iOS developers, which isn’t a disadvantage in itself. However, your QA engineers may need to learn a new programming language to write tests! XCUITest is an excellent way for iOS developers to test more frequently during development, but you have a limited choice of tech stacks.

2. You’re bound to a specific platform

Naturally, you can’t test your Android or mobile web apps with XCUITest because it was designed for iOS only. As a result, your developers and QA engineers need to create and maintain multiple codebases if you want to publish your app for different platforms. This also means that you need to dig into other test automation frameworks that rely on other technology stacks, making your whole setup more arduous to maintain.

Also, note that you can only install XCode on macOS! If your developers use Windows, for example, they can’t run XCUITests natively on their machines.

As for programming languages, XCTest/XCUITest is fully compatible with both Objective-C and Swift.

Code sample with Objective-C:
- (void)testClicksOnRadioButtons {
 [tester tapViewWithAccessibilityLabel:@”Radio1”];
 [tester tapViewWithAccessibilityLabel:@”Radio2”];
 [tester tapViewWithAccessibilityLabel:@”Radio3”];
 [tester enterText:@”Simple Test”
 intoViewWithAccessibilityLabel:@”editText1”];
 [tester tapViewWithAccessibilityLabel:@”Answer”];
 }
Code sample with Swift
testClicksOnRadioButtons() {
let app = XCUIApplication()
app.radiobutton[0].tap()
app.radiobutton[1].tap()
app.radiobutton[2].tap()
app.staticTexts[“Simple Test”]
app.button[0].tap()
}

Appium vs. XCUITest – Which One Should You Choose for Automated iOS Testing?

The answer is: It depends. If you only need a solution for automated iOS UI testing, XCUITest is the best choice. Compared to Appium, it delivers test results faster and more reliable. It already comes with XCode and is straightforward to configure. In short, you get all the benefits you expect from a dedicated UI testing solution.

However, if you’re developing for multiple platforms, Appium may be the better choice. XCUITest depends on XCode, which runs only under macOS. Also, if you’re building your app with frameworks and libraries like Apache Cordova or React Native, you probably don’t want to split your codebase for UI testing. What’s more, Appium can also be the better fit for smaller teams where not everyone is familiar with Swift/Objective-C.

As you can see, for automated iOS app testing, it depends on your app, team, and preferences whether you choose Appium or XCUITest. Regardless of your choice, you need a place to run these automated tests! Bitbar supports both Appium and XCUITest. Try our real device cloud for free and see for yourself.