Mobile Game Testing – Part #4: Testing for Connectivity

Bitbar, the mobile devops company. Logo, large

Connectivity is one of the key aspects when building mobile games. Typically, mobile games communicate over the internet with various different entities; app markets (e.g. Google Play), directly with other players and their devices, or your own back-end servers. When it comes to mobile game development, both indies and major game studios are struggling to make an ideal environment for testing massive multiplayer setup. This is one of the areas where test automation can provide you significant benefits.

We discussed couple weeks ago in our Best Practices in Mobile Game Testing blog series’ the second post – UI and Functional Testing with Image Recognition how test automation can provide significant benefits to mobile game testing with help of image recognition/comparison sort of features. In
Mobile Game Connectivityfact, test automation can also help when it comes to multiplayer testing, cross-platform testing and general connectivity testing. Not only spotting out those bugs and issues, but also significantly reduces the cost of failure, saves time through its repeatability and earlier verification, and leverages resource productivity. Using test automation for mobile game testing already in the earliest phase of development is a good way to gain those benefits and effectively use resources.

But you also need a good infrastructure for doing efficient testing for connectivity of your game. Naturally, there are plenty of options where and how testing will be conducted, but let’s take two the most common ones as an example – in-house environment and cloud-based setups. For setting up an in-house test environment, you will need servers, USB hubs, plenty of real mobile devices (phones, tablets etc. whatever you plan to support with your game), appropriate speed WiFi, and so on. Here is a good tip and check-list what you need if you built this internally.

Using a cloud-based setup is easy and effortless, and you can use various different techniques to test the connectivity aspects of your game. For example, different types of performance tests can be used. However, the most important thing in cloud-based testing is that you have plenty of devices in use – as versatile set of different OS versions, chipsets, display sizes, memory and other form factors. This enables the simultaneous smoke and compatibility testing when you do connectivity and performance testing for your game.

Whether deciding to go with either of these options, WiFi infrastructure is another very important infrastructure area that is often overlooked when creating large scale mobile test automation environments. You can get to about 10 maybe even 15 devices with any WiFi infrastructure without any problems but as the number of devices in your WiFi network adds up, so do the challenges. Those issues will start appearing when data is transferred. This gives the game developer an excellent way to optimize the data passing, infrastructure software and decide what data should be passed between servers and clients. Not everything needs to be communicated/sent back and forth. In general, it is a good rule of thumb that if WiFi cannot handle 15 devices sending all data between server and client (your game), you should seriously consider optimizing the data passing.

Quick Check-List for Testing Connectivity

1. Optimize the data communication between servers and clients. You can set your mobile game to fetch or pull a large amount of data from your back-end server. Especially if you plan to build a MMO (massive multiplayer online) game you need to optimize every piece of your data communication. Despite unlimited data plans are pretty common nowadays, you should still for sake of user experience try to optimize all possible data passing.

2. Test simultaneously on several devices. This is only possible by using real devices to see if the game states are properly synchronized on all devicesCheck list for Connectivity testing and all device resources (players/users in real life) have equal access to all game resources and that there is no lags in game response, inputs or outputs, for all players.  Test automation can be a huge help with this as test scripts can made ‘intelligent’ and play against each others. For example, with image recognition feature you can put game playing against another instance, on another devices. Take a look at here.

3. How does game behave when online and offline. When internet is available game might work fine. If your game is designed to work online all the time, you should plan how to fall back when device goes offline. This can be easily ‘simulated’ on real environment with real devices for example cutting down the WiFi or stop communicating with back-end server. Typically today’s mobile games can be played offline and later on game status with all details are synchronized with the game centers, servers and so on.

4. Check for interruptions. In addition to online/offline gameplay, there are many other interruptions for games that can happen while game is played. For example, incoming calls, device or server running out of memory, the bugs/failures in game itself causing a state where gameplay doesn’t advance. Actually all these are related to connectivity testing as for example network connection can switch (WiFi<->Cellular) and cause an interruption.

5. Test for cross-platform compatibility and possible differences. If your mobile game is a cross-platform one (e.g. you have identical game for Android and iOS) naturally the use of cross-platform test automation frameworks is a great news for you. Regardless, if there is small differences, you can still use these frameworks.

6. Test for location-based details. If your game is using location based services (e.g. just pulling out the info in which country you are in), you should test for this feature. With test automation you can quickly set up a device with a specific language/country settings to match localized device / environment / user.

7. Combine different forms of testing. While you are doing connectivity testing, you can also do the instant smoke testing how your game works across different devices. Again, with test automation this is easy, effortless and provides you quickly results on how the game performs.

Happy Testing!

How to Build a Million-Download Mobile Game

Learn tips & tricks to build better games and make users’ lives more enjoyable.