In this July, Apple published a check list for app developers breaking down iTunes App Store usage by operating system, demonstrating that most of the iOS user base is running the latest version of the OS. According to Apple’s metrics, 94 percent of its customers use iOS 6, five percent are on iOS 5, and the remaining one percent are on a prior iteration. As Apple users are about to get a massive hit with new iOS7, let’s look at the some of the reasons why testing of iOS apps and games is becoming even more important for developers.
The diagram retrieved from https://developer.apple.com/devcenter/ios/checklist/ on 16 September 2013.
Why Testing on iOS Devices?
As stated, being much less fragmented than Android platform, developing for iOS doesn’t automatically mean more robust and better reliability of apps and games. In fact, all apps should be tested across the iOS device ecosystem and with as versatile set of iOS devices – iPhones, iPads and even iPods – as possible. This will significantly improve the stability and spots out the issues that app includes. And this should be done BEFORE pushing out for publication – you know, bad ratings are starting to flow in as soon as you publish a buggy app. Some of the most significant issues every app developer needs to consider:
- Application stability and compatibility. Despite of being more homogenous platform (compared to Android), iOS also suffers from stability and compatibility issues between the platform versions. And not only those platform updates, but also application stability across different iOS devices and bugs related to app itself are frustrating users. Ideally, app/game should be tested on an array of iOS devices to ensure the smooth behaviour.
- Security & vulnerability issues. Actually, it doesn’t matter on how much time developer invests in designing and implementing their apps/games. The bugs are still inevitable and exists everywhere. The security and vulnerability issues allow hackers to attack on critical information and steal crucial private information.
- Bugs – memory leaks, performance issues. In the most common cases apps and games are crashing because of bugs, directly related to memory leaks or performance issues. Blocks of allocated memory can cause issues to users not only when running app but for apps launched later on.
Getting Started with iOS Test Automation
The second one is the Jasmine framework:
To get started, you can download the example zip file from here. Alternatively, navigate to bitbar.com/testing -> Resources -> APIs, Plugins & Libraries, and download Example iOS Test Case for iOS (Jasmine). As well as with this example, the example that can be found at Testdroid Cloud – ios-test-skeleton – you will have the files/folders as follows:
- ‘ext’ directory – Jasmine framework is located here
- ‘reporters’ directory – the reporter(jasmine.uiautomation_junit_reporter.js) is located here and will be used to create jUnit .xml file with test results.
- ‘specs’ directory – the test specifications are located here
- ‘suite.js’ file – the main file of test scripts. This is the file which is passed to UI Automation tool to run tests. It generally shouldn’t be changed but new test specifications should be added only for instance:
Structure of A Test Script for iOS Testing
Every test case starts from describe() function (in Jasmine, it is used similarily as TestCase class in jUnit). This function takes 2 parameters: description/title of the test case and the function to be executed. Inside this function user should place another function(s) – it(). This function in Jasmine is something similar to TestMethod in jUnit. It also takes 2 parameters: the description/title and function to be executed.
Result of every it()function will be presented at Testdroid Cloud separately. To exercise good practice it is recommended to have every test case in separated file instead of one file with several describe()functions.
INSIDE IT() FUNCTION
Inside this method, the test steps are done. User clicks on elements, goes to interesting place in app, check important things and so on. In Jasmine instead of well-known asserts from jUnit a series of expect() functions are provided. For instance:
expect(target.frontMostApp().mainWindow().textFields().length > 0).toBe(true)
This can be used to verify if there is any UITextField object on window. In addition, it is very useful to take screenshots that can be later reviewed at Testdroid Cloud results. This can be done by using the following function:
In order to get those screenshots shown in proper order at Testdroid Cloud, it is recommended to use numeral order in the beginning of screenshot name.
WRITING TEST SCRIPTS
For writing test scripts, Instruments->Automation tool is very useful and also required to write test scripts. In this tool user can choose the target application and run scripts to see how the execution goes or record his/her actions. In order to make it run fine, it is also recommended that user reviews the generated code and playbacks it as sometimes generated code can be ‘messy’. After recording any actions user can copy-paste produced code to one of it() functions.
In addition, it is useful to use the following methods:
var target = UIATarget.localTarget();
These produce in console output the whole tree of current views and in this way user can easily find out indexes, names, etc. of the views for further action.
CHECKING TEST LOCALLY
If user wants to run the whole test project before uploading it to Testdroid Cloud, they can import suite.js in UI Automation tool and simply click on the play button.
PS. Testdroid Cloud is now hosting devices with iOS7 so check it out and hammer your apps/games against these devices.