Quality assurance (QA) is a critical part of the process when creating and maintaining successful mobile apps for hundreds of millions of app consumers. One of the metrics in this app creation process is naturally QA’s productivity – the success to efficiently find and filter out the problems in apps before they land on hands of users. This type of productivity of quality assurance can be measured by various metrics, for example:
1. Bugs found rate – how efficiently the QA team finds bugs during the testing process per time used on testing
2. Post launch issue rate – how many issues are found after the go-live release
3. Customer satisfaction rate – for example complaints to customer support, issues reported to mobile app stores, overall ratings of the apps
4. Quality of deliverables and issue documentation – how easy it is for developers to understand the issue and make a fix based on the documentation
5. The number of new test cases added per period of time
The challenges unique to mobile app development & QA are device diversity, different operating system versions, hardware configurations and OEM/Carrier customizations. Those aspects increase the complexity of Quality Assurance and result in longer development cycles (time-to-market), higher fields crash rates (lower customer satisfaction and uninstalls hurting your top line) and higher customer support costs (more customer complaints) and inefficient development team (dealing with customer issues vs. developing new features).
In order to improve on QA productivity, teams need to have access to real end user devices.
Every major cloud infrastructure provider offers Dev/Test tools (Develop, Build, Test, Release and Analyze) from cloud. These tools make development teams very agile & efficient: When an environment is needed, they can start the right environment and have it up and running in minimal time and with minimal effort. When you no longer need the environment, you can shut it down to free up resources. In mobile development & QA, this is also possible. Teams can provision real devices immediately from cloud and improve the productivity by factor of 2-4.
Developers and QA teams need access to real devices exactly when they face an issue during development or testing, fixing post-launch issues or customer complaints. Earlier the issues are found, lower the cost of fixing them later during the project. To maximise bug-finding efficiency, you can automate huge amount of testing, and do it on real devices after every new code checking and/or nightly builds. This will shorten your Dev/Test cycle, improve defect-fixing times and result in improved bottom line.
When using cloud based tools for mobile development and QA, the performance of your team will improve a lot: The cost is fraction of the manual testing, developers are happier (instant feedback on new code, no need to deal with purchasing when acquiring new devices), product owner is happier (higher customer loyalty, higher LTV, faster time-to-market), customer service is happier (less customer complaints) and most important of all, your customers are happier as they do not get frustrated by “not working on my device”.
“Adopting cloud based tools for your mobile development and testing is a huge opportunity for app developing companies to improve their performance. The return on investment is in weeks.”
A good check list to move forward is:
1. Consider automating most of the testing, choose any of the excellent open source testing frameworks available for the most popular mobile platforms
2. Avoid totally unnecessary vendor lock-in by using standard languages (Java, dotnet, perl, php, python, ruby), there is a pool of millions of developers using these languages to tap into
3. Automate as much as possible of the testing to improve the organisational agility (even make developers responsible for creating automated tests)
4. Use continuous integration and continuous testing, testing is a continuous process in agile and will deliver you results 24/7
5. Integrate with your test management systems to improve the collaboration and transparency throughout the organization. Tools must enable a free flow of collaboration.
6. Instant accessibility. The development teams do not have time to engage with long sales cycles, procurement processes for each tool and device. Choose a service with rapid provisioning without waiting for hardware, procurement or internal processes.
7. Finally, the most important one: Tools matter, but how you use them matters more. Tools make a difference, but how you use them will ultimately determine your level of success.
This blog concludes the CEO blog series. You can read the whole series starting here.
Learn major cost drivers in mobile testing and how to leverage test automation to improve the ROI.Download