Our CEO Marko Kaasila got interviewed by Netscout for his insights and deep knowledge of the mobile testing industry. Read more to see what Marko things of today’s trends in mobile app testing and the best practices how mobile app developers, testers and organizations of different sizes can overcome problems, improve their app quality and gain a better understanding of how users use their apps.
What are the common challenges facing mobile developers today?
First and foremost, the most common challenge that mobile developers are facing today is the pressure to release software often and at an ever-increasing pace.
Mobile is eating the world and companies are making a big chunk of their revenue and customer engagement on mobile. So the pressure to deliver high-quality mobile applications and services to end users is immense. Tolerance for failure on mobile is incredibly low if the app fails, the user will uninstall and move on, most probably to a competitor’s service.
How do you and your team help solve some of these problems?
Shortening the mobile software delivery cycles and testing early is the key. Everyone knows that speed wins! Every time there is a new code committed, the entire delivery pipeline should be initiated. The new code is automatically built to the application, which is then automatically tested on real end-user devices to verify correct functionality, compatibility and adequate performance.
What we have built is a cloud-based infrastructure to automate that process. We host thousands of different kinds of devices, real non-emulated Android, iPhones, iPads and even wearable hardware, in the cloud to help our customer to deliver their applications faster and with high quality.
What are the most common mistakes or oversights you see developers making in building mobile apps?
The most common mistake is to develop in the agile process but then fall back to the old school waterfall process and do manual testing. That slows down the delivery process dramatically and leads to low developer productivity, slower speeds of innovation, and exposes you to a threat from faster-paced competition.
On a very concrete level, not addressing the complexity of the runtime environments, the differences in chipset architecture and the number of different devices your users will be using your service on is an oversight that companies often do. We are seeing a dramatic change there, but there are still companies that have a mobile strategy in place, but then fail in execution by not understanding the fundamental differences between web and mobile development.
How could these mistakes have been avoided?
The best companies avoid these mistakes by doing continuous testing. Testing is an integral part of the engineering process, and they use a lot of automation. Teams must have automated builds, automated tests and even automated deployments.
Software engineers in Test are a growing segment within software engineering. They do own the testing part of the software development, but instead of doing manual QA work, they write software that tests the products. Then they use services like Testdroid to run those automated tests continuously as the application is developed, for every code check-in, for every internal release and naturally for every production release. For example, in our platform, Testdroid, customers run automated tests on hundreds of devices simultaneously and get results back for analysis in minutes. Doing traditional manual quality assurance, getting the same results would take days, if not weeks!
Why is testing such an important component of app development?
Mobile users have a very low tolerance for failure. If the application, or mobile service chain in general, does not deliver, they will go somewhere else. So companies must invest in testing in order to guarantee high-quality apps and services. Testing costs in mobile are from 25 percent of the total mobile app development costs up to 50 percent in some segments, like in mobile gaming. If the mobile application fails in production, the companies will not only lose the investments in the development, but also the marketing costs used to promote the mobile offering.
Low quality also has a huge impact on customer satisfaction, support costs and directly to the bottom line.
What should developers be focusing on and/or looking at in the testing phase of their mobile apps?
One word: Testability. Knowing the importance of quality when delivering applications to the market, developers should consider how to make the application testable as quickly as possible. If developers start thinking to test after finishing their part, it is usually too late. Testing should be considered in each step of the development process. This way you can achieve a high level of automation and will help the entire development team to deliver applications at the speed required by the business.
How do you think most developers could improve their testing methodology?
Define the test matrix early on, meaning choosing the most important devices to test. Then choose the best testing tools for the use case and write the tests along the application. Do the unit testing on the real devices to see the compatibility and performance issues on a wide range of the runtime environments and once the UI is stabilized, implement the UI and functional tests with the chosen tool.
There are a ton of open source tools nowadays to choose from for various test phases, as well as test tools provided by the platform vendors like Apple’s XCTest and Android Espresso. On Tesdroid platform we support all of them, so we can cover all the tastes of the development teams. They should naturally use a platform, like Testdroid, where you can continuously run all the tests, no matter what tools are used.
Also after the launch, it is important to continuously monitor the performance of the applications. For example, using Bitbar Monitoring, developers can spot issues on real networks, real devices in different cities before their users do. And the best part is that developers can use the same tests that they used in the development phase to do production monitoring.
What parts of app testing seem to create the biggest headaches for developers?
It is definitely the device fragmentation. There is a wide range of different configurations in terms of devices that it is extremely challenging to create an app that works flawlessly on all of them, on Android and iOS. There are very small screen devices, large screen devices, devices with low-end single-core CPU and then monster devices with octa-core CPU. Some devices have low memory. Most applications these days rely heavily on third-party SDK’s, which not all SDK providers can test the audience in which their users are developing for, so developers in most cases must double check their frameworks.
What trends, innovations or headlines are you following closely in your world right now? Why?
I am following closely the DevOps trend, especially Mobile DevOps. Continuous delivery is number one challenge for Development and IT organizations mainly due to pressure coming from the business side. On server-side DevOps and Continous delivery are becoming mainstream. But on the mobile development, which is 10X more complex due to the very fragmented toolchain and devices it is a really difficult problem to solve. But we are determined to do that.
You can get read it also in the original place on Netscout.