Automating mobile game testing can really make a big difference in how thoroughly (and easily) people can get their Android and iOS games tested. One of the most prominent approaches has been image recognition together with Appium. Today, I’ll walk you through how to modify the existing template/example with AKAZE and OpenCV libraries and how to build the latest versions of image recognition libraries.
It’s never highlighted enough but testing mobile games requires a lot of user inputs, most of which are clicks, gestures or interactions with certain UI elements with the right timing. This naturally can be a very laborious task to do manually and therefore test automation has a great value to offer. You basically make sure the test script does the ‘right things’ and then you (or your build system/scripts/continuous integration environment) keep executing it across a variety of different setups, and the gameplay goes on.
As there are lots of diversity with both Android and iOS in terms of OS versions, display sizes, memory, performance and many other characteristics from mobile devices, test automation build for testing must be scalable and reliably interact with graphics content and gameplay. Because of these requirements, plus screen resolution variances, image recognition provides an easy way to automate mobile game testing.
In addition to preferred setup and test automation frameworks, we’ve been talking about the best practices for testing mobile games, how mobile game testing differs (and not only due to the graphics) from the regular mobile app testing, what are the must-to-have architecture and infrastructure as well as what different processes you should consider using for mobile game testing.
Mobile Game Testing with OpenCV and AKAZE for Image Recognition
So far this combination has been probably the easiest and the most reliable software approach to get mobile game testing done. Rather than performing interactions in random order we’ve taken Appium as part of this equation and provide simple test script templates that you can start with. However, we’ve provided OpenCV and AKAZE examples only for certain platform so you might need to rebuild those libraries to work in your local environment.
These instructions have been tested on Mac (Yosemite and El Capitan) and these should work fine pretty much on all major Linux distributions.
The jsoncpp is a C++ library for manipulating JSON values and basically this will be used together with AKAZE. This will provide you all stuff for C++ recompilation and it’s suggested to use amalgamated source to include this in your builds.
First, you need to recompile the latest version of jsoncpp as this is required by AKAZE. Ensure you have (at least) Python 2.6 installed and working properly. Then, just follow these steps to get jsoncpp installed:
git clone https://github.com/open-source-parsers/jsoncpp.git cd jsoncpp python amalgamate.py
By default, the following files are generated and are required to recompile new AKAZE binaries:
dist/jsoncpp.cpp: C++ source file that needs to be added to your project.
dist/json/json.h: header file to be used in your project.
dist/json/json-forwards.h: header that includes a forward declaration of all jsoncpp types.
- You will also need to build and copy the
libjsoncpp.alibrary to your local lib directory.
To copy the newly built library to your local lib directory, follow these steps:
mkdir build cd build cmake .. make cp src/lib_json/libjsoncpp.a /usr/local/lib/
The first thing to do with OpenCV is to clone the OpenCV bundle and check out version 2.4. For new versions and enhancements, please check the status of the latest versions from OpenCV Github repository. To get started, follow these steps:
git clone https://github.com/Itseez/opencv.git cd opencv git checkout 2.4 mkdir build cd build/ cmake -G "Unix Makefiles" .. make -j8 make install
This creates Java .jar files for the image recognition sample. The latest .jar files can be then installed with the following command:
mvn install:install-file -Dfile=path/to/opencv/build/bin/opencv-2412.jar -DgroupId=opencv -DartifactId=opencv -Dversion=2.4.12 -Dpackaging=jar
The pom.xml file needs to be updated with installed OpenCV version (2.4.12 in this example) as follows:
... <dependency> <groupId>opencv</groupId> <artifactId>opencv</artifactId> <version>2.4.12</version> </dependency> ...
There haven’t been recent changes in the feature set of AKAZE but in order to get it built for your setup, follow these steps:
git clone https://github.com/bitbar/akaze.git cd akaze # copy previously built versions of jsoncpp to 'src/lib/jsoncpp/', 'src/lib/json/' and src/lib/jsoncpp.cpp mkdir ./src/lib/jsoncpp/ cp -r path/to/jsoncpp/dist/json ./src/lib/jsoncpp/ cp -r path/to/jsoncpp/dist/jsoncpp.cpp ./src/lib/ mkdir build cd build cmake .. make cp bin/akaze_* path/to/image-recognition/akaze/bin/
You have now up-to-date versions of each used libraries installed. The next thing to do is to verify that everything works and basically you can use the example test from the image-recognition sample directory:
mvn -Djava.library.path=/usr/local/share/OpenCV/java/ test
Real-World Example – How to Automate Mobile Game Testing
If you are interested to look into a specific example of how mobile game test automation has been implemented, here is one awesome step-by-step description of what happens and what needs to be done in each step.
Happy Game Testing Folks!