Using cloud solutions for assuring app quality nowadays is trending. Yet, many companies still prefer a traditional way, a.k.a an in-house solution, to handle their app testing.
The problem is that there’s no one-size-fit-all solution to meet any specific requirements. Both cloud services and in-house solutions have their merits in terms of data privacy, accessibility, cost, scalability, and etc. To make things straightforward, we compiled a complete checklist (or you can download a pdf version here) that you can use to make sure that you are able to select the right environment for your app testing.
Download this comprehensive ebook for more info about choosing the optimal environment. We updated it lately.
Data Privacy and Ownership
Here are some questions to help you consider how confidential your application and testing data is.
- Does your organization have the attributes of financial business or customer personal info?
- Is your app government or military related?
- Are you building or launching an astonishing application or game?
- Can your testing data be hosted and stored on a cloud-service?
- Do you need to share your data within a local environment?
The accessibility of a testing environment is important, especially when you pursue an agile approach. Ask yourself or your team the following questions whether the solution you are using or will use is optimal in terms of availability.
- Do you need to get the access to devices immediately or anytime you want?
- Do you strictly need a very high uptime of the solution, equal to or greater than 99.5%?
- Do you need VPN access or various network for app testing?
Like the reliability mentioned above, the support provided by either your vendor or your internal team is rather important. Below are some things you should consider.
- Do you need a 24/7 support when something unpleasant happens?
- Are you well-staffed for maintaining the devices, infrastructure and hardware?
- Is your team knowledgeable of giving a quick fix to a potential glitch?
- Do you want the support team to be accessible with any type of communication?
- Are live demonstrations what you would need when it comes to training or problem solving?
Enhancements and Update Policies
You can get the latest software every week or month, but your software is not that new tomorrow or next week. Using the most advanced software gets your job done quickly and nicely. Here are some considerations for you regarding software upgrade.
- Do you want a smaller but faster update delivered to your software?
- Or do you want a bigger but slower upgrade?
- Do you need the technical support to the addons you might want?
Normally developers live in their own app building environment most of the time and QA team takes care of the testing process. However it is highly possible that developers want to run a quick test or two from his tools.
- Are you using open source app development tools and testing frameworks?
- Will you use any proprietary toolsets in the future?
- Do you need to connect your app building environment to the testing environment?
- Will you consider to connect your continuous integration system with the solution?
- Do you use API for your app testing?
Total Cost of Ownership
Now the cost and ROI are what your top management cares the most. No one wants to pay for service that can’t produce consistent value.
- Does your company accept an IaaS mode service?
- Is SaaS service that you are looking for?
- Do you need a more flexible payment process, like subscription for a cloud solution?
- Is your company ok with the cost of acquiring devices for your in-house solution?
Scaling up your app testing is important to meet the coverage of devices and end customers. Testing on only tens of devices is way too inefficient.
- Do you want to get access to a large breadth of various devices at anytime?
- Do you need the flexibility of running on 20 devices and 200 devices?
- Is unlimited concurrent devices the feature that you need?
- Do you need a plug-and-play mechanism to scale up the testing coverage?