Open test automation frameworks are constantly evolving, and at least one new test framework emerges every year and becomes the favorite one among developers. That puts more challenges on the development and testing/QA teams to make long-term planning, since investing in test automation yields results over a longer period of time and test assets cannot be ported to new frameworks frequently without losing a significant amount of the investments.
A tremendous number of things in mobile app development and testing are metric-driven. And especially when it comes to mobile app testing, metrics are highly valuable indicators of how well does the app perform, what is the general bug rate, and how many things have been exposed from the code so far. Metrics also reveal how well test automation and infrastructure used around it can expose issues in the code and how quickly and easily developers can fix those.
In this blog, I’ll describe one great example, a real-life story, of how one of our customer uses a great number of identical mobile devices, with the very same OS version, with the same OEM customization setup, to hammer their builds of their mobile app hundreds of times per day, and how each build are getting tested on those devices every time.