Testing is actually a search problem: we know defects exist in our code but we just have to spot them. Unfortunately, it often means nothing about how much of time you have invested in your Android or iOS app design, or even how pedant you have been when coding your masterpiece: human errors are unavoidable and bugs are there.
Testing of your mobile app – and outcome of those tests – represents the ultimate review of the requirements specification, the design, and the code. As we did in one of our prior Best Practice blogs, this one also continues on the path of suggesting separating your test cases/scripts into smaller unit and using good old object-oriented approach for testing as well.
As underlined before, testing should be started in the earliest phase possible in your mobile app development. From our experience, those app developers who start testing even before the full set of requirements have been defined and the development process has started, are likely to produce a better quality, robust app to marketplace.
As we know, object-oriented programming concepts are different from conventional programming and have become dominant with object-oriented programming languages. The fundamental entity is the class that provides an excellent structuring mechanism. It allows programmer to divide a system into well defined units which may then be implemented separately.
But object-oriented testing has changed quite a lot in years and nowadays great example is mobile app testing. The process differs quite a lot from older Unit -> Integration -> System -> Validation type of flow. Sure, there were other steps like function, performance and acceptance testing, but those are combined here under System testing.
Mobile app testing is nowadays more about “testing everything in parallel”. You don’t necessarily test all the functions and different bits and pieces separately and in different phases. For example, Unit testing is something that should happen during the entire product development lifecycle. Basically the same applies for integration testing – e.g. how these different features have affected to your mobile app. The same applies with system and validation testing, which can be seen as making your mobile app compatible with an array of different Android or iOS devices, and making it work on all different variations of OS versions, hardware setups, and so on.
Starting with unit testing, the Pragmatic Unit Testing in Java with JUnit by Andy Hunt and Dave Thomas was released over 10 years ago but it still provides a good overview to unit testing. Some things are still applicable to modern mobile app testing. For example, one nice example is the “Right – BICEP” approach:
The “Right – BICEP” Approach
- Right – Are your testing results right? How to validate your test results, and does the expected result match what the method does? In a nutshell, if you don’t know what “right” would be – how can you test? For versatile validation of your tests “rightness” you can use data-driven test cases.
- B – Are all the boundary conditions correct? Boundary conditions could be see as follows: garbage input values, badly formatted data for input, empty or missing values, values out of range, duplicates, unexpected orderings and so on. Boundaries could be checked with CORRECT (below).
- I – Can you check and test the inverse relationships? For example, if your method does something that has an inverse, you should apply the inverse.
- C – Are you able to cross-check results using other means? Can you do something more than one (=your) way? Are there overall consistency factors you can check?
- E – can you force error conditions to happen? Testing some of these are very straightforward: invalid parameters, out of range values, running out of resources (memory, disk space etc.), network failures, high system loads and so on. Some of them are more challenging, like implementing exceptions and interrupts to your app.
- P – are performance characteristics within bounds? Could your routine be optimized and how does the performance change when data input grows.
- C – Conformance – does the value conform to an expected format?
- O – Ordering – is the set of values ordered or unordered as appropriate?
- R – Range – is the value within reasonable minimum and maximum values?
- R – Reference – does the code reference anything external that isn’t under direct control of the code itself?
- E – Existence – does the value exist (e.g. is not null, non-zero, present in a set)?
- C – Cardinality – are there exactly enough values?
- T – Time (absolute and relative) – is everything happening in order? At the right time? In time?
Continuing with integration testing – which should focus on grouping classes that collaborate or communicate with others in some manner, it can be said that integration of operations one at a time into classes is often meaningless. Today, integration testing on apps should be seen as regression testing. This is important as each component, thread, cluster, or any type of subsystem addition to your mobile app causes a need to test. For example, most of the mobile services today have a server-client interaction, requiring a login, uploading of data (e.g. pictures) and downloading of data (e.g. news, data, images). When developing these kinds of services, users need to ensure that all their changes done in code, to either server side or client side do not break the functionality of the service.
EXAMPLE: Put Often Used Functions in Their Own Test Methods
When testing your mobile app sometimes annoying pop-up – and also more important things like login screen – shows up. This can mean you need to build a workaround in your test automation scripts. Let’s focus on a login example. As you need to login with using your actual credentials and if you have logged in to app already, the login screen doesn’t necessarily pop up. This can be because of application data exists and app recognizes you – or simply you are running consecutive test runs where login needs to happen only at the beginning.
For example, you could encapsulate this to you
setUp() (same could apply to
tearDown() etc.) script to make sure script won’t stop there and try to find UI elements for login: