Connectivity is one of the key aspects when building mobile games. Typically, mobile games communicate over the internet with various different entities; app markets (e.g. Google Play), directly with other players and their devices, or your own back-end servers. When it comes to mobile game development, both indies and major game studios are struggling to make an ideal environment for testing massive multiplayer setup. This is one of the areas where test automation can provide you with significant benefits.
We discussed a couple of weeks ago how test automation can provide significant benefits to mobile game testing with help of image recognition/comparison sort of features. In fact, test automation can also help when it comes to multiplayer testing, cross-platform testing, and general connectivity testing. Not only spotting out those bugs and issues, but also significantly reduces the cost of failure, saves time through its repeatability and earlier verification, and leverages resource productivity. Using test automation for mobile game testing already in the earliest phase of development is a good way to gain those benefits and effectively use resources.
But you also need good infrastructure for doing efficient testing for connectivity of your game. Naturally, there are plenty of options where and how testing will be conducted, but let’s take two the most common ones as an example – in-house environment and cloud-based setups. For setting up an in-house test environment, you will need servers, USB hubs, plenty of real mobile devices (phones, tablets etc. whatever you plan to support with your game), appropriate speed WiFi, and so on. Here are a good tip and check-list what you need if you built this internally.
Using a cloud-based setup is easy and effortless, and you can use various different techniques to test the connectivity aspects of your game. For example, different types of performance tests can be used. However, the most important thing in cloud-based testing is that you have plenty of devices in use – as a versatile set of different OS versions, chipsets, display sizes, memory, and other form factors. This enables the simultaneous smoke and compatibility testing when you do connectivity and performance testing for your game.
Whether deciding to go with either of these options, WiFi infrastructure is another very important infrastructure area that is often overlooked when creating large scale mobile test automation environments. You can get to about 10 maybe even 15 devices with any WiFi infrastructure without any problems but as the number of devices in your WiFi network adds up, so do the challenges. Those issues will start appearing when data is transferred. This gives the game developer an excellent way to optimize the data passing, infrastructure software and decide what data should be passed between servers and clients. Not everything needs to be communicated/sent back and forth. In general, it is a good rule of thumb that if WiFi cannot handle 15 devices sending all data between server and client (your game), you should seriously consider optimizing the data passing.
Quick Check-List for Testing Connectivity
1. Optimize the data communication between servers and clients. You can set your mobile game to fetch or pull a large amount of data from your back-end server. Especially if you plan to build an MMO game you need to optimize every piece of your data communication. Despite unlimited data plans are pretty common nowadays, you should still for sake of user experience try to optimize all possible data passing.
2. Test simultaneously on several devices. This is only possible by using real devices to see if the game states are properly synchronized on all devices and all device resources (players/users in real life) have equal access to all game resources and that there are no lags in-game response, inputs or outputs, for all players. Test automation can be a huge help with this as test scripts can be made ‘intelligent’ and play against each other. For example, with image recognition feature you can put game playing against another instance, on other devices.
3. How does game behave when online and offline. When the internet is available, the game might work fine. If your game is designed to work online all the time, you should plan how to fall back when the device goes offline. This can be easily ‘simulated’ on a real environment with real devices, for example, cutting down the WiFi or stop communicating with the back-end server. Typically today’s mobile games can be played offline and later on game status with all details are synchronized with the game centers, servers and so on.
4. Check for interruptions. In addition to online/offline gameplay, there are many other interruptions for games that can happen while the game is played. For example, incoming calls, device or server running out of memory, the bugs/failures in the game itself causing a state where gameplay doesn’t advance. Actually, all these are related to connectivity testing as for example network connection can switch (WiFi <-> Cellular) and cause an interruption.
5. Test for cross-platform compatibility and possible differences. If your mobile game is a cross-platform one (e.g. you have identical game for Android and iOS) naturally the use of cross-platform test automation frameworks is great news for you. Regardless, if there are small differences, you can still use these frameworks.
6. Test for location-based details. If your game is using location-based services (e.g. just pulling out the info in which country you are in), you should test for this feature. With test automation, you can quickly set up a device with a specific language/country settings to match the localized device/environment/user.
7. Combine different forms of testing. While you are doing connectivity testing, you can also do the instant smoke testing how your game works across different devices. Again, with test automation, this is easy, effortless and provides you instant results on how the game performs.