The iOS 10 has rolled into Apple devices and has already got a significant install base on different Apple devices out there. Historically, the pace has been even faster when it comes to getting the latest and greatest iOS version adopted by 75% of device users. In addition, the iOS 10 adoption hasn’t gone as smoothly as prior ones for those who use it for test automation and are using iOS 10 with their existing test scripts and assets.
In this blog we’ll look at the state of the art with iOS 10 test automation and what mobile app developers should consider when building their infrastructure and using de-facto frameworks for their test automation needs.
What Has Changed in iOS 10?
The recent launch of iOS 10 and its subversions has caused lots of issues for iOS app developers and especially, iOS test automation. The whole ecosystem of developers and testers have been forced to tweak their platforms, test scripts and infrastructure (plus development tools) to get things running smoothly from continuous integration to real device testing.
The MixPanel report on iOS adoption reveals that over 75% of users have already upgraded to the iOS 10 base version or any of its subversions (or beta versions). As said, this hasn’t happened as quickly as we’ve expected to see much faster adoption of iOS versions on users devices.
This ‘slowness’ in adoption also creates fragmentation on iOS and causes applications to behave differently, even on the same device (with the same hardware) but running the different OS version. The only thing app developers can really do is to make sure that app is thoroughly tested on all possible variants, and this is where test automation has its strengths.
Impact of iOS 10 on Xcode Development Environment
For example, many of iOS test automation frameworks used by mobile app developers are built on top of UIAutomation and as Xcode 8 replaced UIAutomation with Xcode’s new test framework – XCUITest – the old Xcode Instruments based test scripts need to be deprecated.
QUICK TIP! If you running a test script built for prior Xcode versions, all uia_* calls will raise an error with the latest Xcode. How can you tackle this and what are the options to keep your internal test automation process running? We’ll shed some light to the state of the art with iOS 10 test automation in this blog.
The Requirements of iOS 10 for Mobile Test Automation
First of all, it’s always recommended to use real iOS devices (iPhones, iPads etc.) for testing. And with test automation, using Appium, Calabash or simply built-in XCUITest you can quickly and easily build your test scripts to ensure application quality meets the user expectations.
As stated before, UIAutomation has been deprecated from the latest Xcode and if you still use it – or you use some older version of e.g. Appium – there are things that you should be aware of. UIAutomation was supported with Xcode 6.x and 7.x and you won’t have any issues with running iOS 8.x and 9.x installed devices on that environment.
The XCTest is a framework that was introduced back in the Xcode 5 and it’s a modern version of OCUnit. If you are still running OCUnit tests there is a reference guide to walk you through migrating OCUnit tests to XCTest by Apple. However, XCTest can be still used with the latest Xcode 8.x versions.
XCTest serves as a basis for KIF (Keep It Functional) test automation framework which allows an easy creation of test automation scripts for iOS applications. The KIF builds and performs the test scripts using a standard XCTest testing target, and it will work just fine on the latest Xcode 8.
Then, let’s move to the cross-platform frameworks.
Calabash iOS 10 requirements are pretty straightforward and you can find a step-by-step instructions on that blog how to make sure things work fine with any iOS 10 variant. Download our Calabash 101 to learn basics and advanced tips to leverage the power of Calabash for mobile app testing.
Appium, however, do require some attention and test automation functionality depends on a certain version of Xcode and iOS. For example, if you are using an older version of Appium for iOS testing (version below 1.6) the framework is supported only with Xcode 6 and Xcode 7. This comes from the UIAutomation dependency and it will only support iOS 8.x and 9.x versions.
In order to get your Appium iOS tests and test scripts (with minimal or no tweaks at all) running on iOS 10.x you need to upgrade to the Xcode 8 – and then your Appium scripts will also work fine on the older iOS versions as well. However, it’s good to prepared to do minor tweaks on your test scripts as iOS versions have evolved and brought some new functionalities and APIs that change how to UI looks and feels for users (or for test automation script). Generally, Appium iOS testing typically does require some tweaks if you have used those test scripts with Appium Android testing.
If you are looking for a recommendation on which iOS devices (with certain iOS version) you should test your app, do check this report and get the comprehensive listing of which Apple devices are used in certain countries:
Happy iOS Test Automation, Folks!