The State of The Art in iOS 10 Test Automation Frameworks

State of the Art with iOS Test Automation

The iOS 10 has rolled into Apple devices and has already got a significant install base on different Apple devices out there. Historically, the pace has been even faster when it comes to getting the latest and greatest iOS version adopted by 75% of device users. In addition, the iOS 10 adoption hasn’t gone as smoothly as prior ones for those who use it for test automation and are using iOS 10 with their existing test scripts and assets.

In this blog we’ll look at the state of the art with iOS 10 test automation and what mobile app developers should consider when building their infrastructure and using de-facto frameworks for their test automation needs.

What Has Changed in iOS 10?

The recent launch of iOS 10 and its sub versions has caused lots of issues for iOS app developers and especially, iOS test automation. The whole ecosystem of developers and testers have been forced to tweak their platforms, test scripts and infrastructure (plus development tools) to get things running smooth from continuous integration to real device testing.

The MixPanel report on iOS adoption reveals that over 75% of users have already upgraded to the iOS 10 base version or any of its subversions (or beta versions). As said, this hasn’t happened as quickly as we’ve expected to see much faster adoption of iOS versions on users devices.

This ‘slowness’ in adoption also creates fragmentation on iOS and causes applications to behave differently, even on the same device (with the same hardware) but running the different OS version. The only thing app developers can really do is to make sure that app is thoroughly tested on all possible variants, and this is where test automation has its strengths.

MixPanel Report on iOS 10 Adoption

Impact of iOS 10 on Xcode Development Environment

The biggest change in Xcode development environment is the deprecation of UIAutomation. This has had an impact on various defacto open source test automation frameworks, such as Appium and Calabash.

For example, many of iOS test automation frameworks used by mobile app developers are built on top of UIAutomation and as Xcode 8 replaced UIAutomation with Xcode’s new test framework – XCUITest – the old Xcode Instruments based test scripts need to be deprecated.

QUICK TIP! If you running a test script built for prior Xcode versions, all uia_* calls will raise an error with the latest Xcode. How can you tackle this and what are the options to keep your internal test automation process running? We’ll shed some light to the state of the art with iOS 10 test automation in this blog.

The Requirements of iOS 10 for Mobile Test Automation

First of all, it’s always recommended to use real iOS devices (iPhones, iPads etc.) for testing. And with test automation, using Appium, Calabash or simply built-in XCUITest you can quickly and easily build your test scripts to ensure application quality meets the user expectations.

As stated before, UIAutomation has been deprecated from the latest Xcode and if you still use it – or you use some older version of e.g. Appium – there are things that you should be aware of. UIAutomation was supported with Xcode 6.x and 7.x and you won’t have any issues with running iOS 8.x and 9.x installed devices on that environment.

The XCTest is a framework that was introduced back in the Xcode 5 and it’s a modern version of OCUnit. If you are still running OCUnit tests there is a reference guide to walk you through migrating OCUnit tests to XCTest by Apple. However, XCTest can be still used with the latest Xcode 8.x versions.

XCTest serves as a basis for KIF (Keep It Functional) test automation framework which allows an easy creation of test automation scripts for iOS applications. The KIF builds and performs the test scripts using a standard XCTest testing target, and it will work just fine on the latest Xcode 8.

State of the Art with iOS Test Automation

Then, let’s move to the cross-platform frameworks.

Calabash iOS 10 requirements are pretty straightforward and you can find a step-by-step instructions on that blog how to make sure things work fine with any iOS 10 variant. Download our Calabash 101 to learn basics and advanced tips to leverage the power of Calabash for mobile app testing.

A New Way to Use Appium/Selenium with Real Devices on Cloud

Appium, however, do require some attention and test automation functionality depends on a certain version of Xcode and iOS. For example, if you are using older version of Appium for iOS testing (version below 1.6) the framework is supported only with Xcode 6 and Xcode 7. This comes from the UIAutomation dependency and it will only support iOS 8.x and 9.x versions.

In order to get your Appium iOS tests and test scripts (with minimal or no tweaks at all) running on iOS 10.x you need to upgrade to the Xcode 8 – and then your Appium scripts will also work fine on the older iOS versions as well. However, it’s good to prepared to do minor tweaks on your test scripts as iOS versions have evolved and brought some new functionalities and APIs that change how to UI looks and feels for user (or for test automation script). Generally, Appium iOS testing typically does require some tweaks if you have used those test scripts with Appium Android testing.

If you are looking for a recommendation on which iOS devices (with certain iOS version) you should test your app, do check this report and get the comprehensive listing of which Apple devices are used in certain countries:

The Most Popular Devices Globally for Mobile App Testing

Happy iOS Test Automation, Folks!


The Beginner’s Guide of Using Appium for Mobile App Testing

Learn all the basics about Appium, how to set it up and how to use it for mobile app testing.

Download

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close