Using a public cloud for mobile app testing has been an exponentially growing trend. An amazing number of app developers have migrated from their local – and nearly in all cases – manual testing practices to test automation on a device cloud where devices are available 24/7. One of the top reasons has been the lower maintenance and operational costs, as well as device acquisition costs. While migration to a device cloud has many benefits, not all companies can use cloud-based services. However, there is always a ‘private cloud’ option that gives even greater benefits.
There are many reasons why cloud-based services cannot be used by certain types of companies. Banks are a great example. Due to their internal regulations, security and privacy concerns, the use of public cloud services isn’t an option within this segment. But could private cloud tackle these challenges – and if yes, how? Well, they actually can, and provide the most optimal solution for high security-concerned instances.
We can define “private cloud” as a cloud service that is operated by someone on data centers and locations outside of the actual site where development, source code, and all critical assets reside. We can also define a private cloud as an internal cloud setup that is on-premise but provides cloud-like access and solution for these types of instances internally. Typically those are called ‘on-premise solutions’.
In this comparison, we’ll deal ‘on-premise’ and ‘private cloud’ as one similar entity. Yes, they are different by many standards and measures, but from the top 10 most critical characteristics that companies consider when selecting the most suitable solution, the difference is actually pretty minimal. Especially in terms of benefits that private cloud setup can provide a similar way on-premise solutions do.
By gathering the right information app developers can make the decision whether a public cloud is good enough for their needs – or if a private cloud can still be a better, more cost-efficient and more flexible solution to use. Let’s look at the top 10 things to consider when comparing these two solutions.
10 Considerations to Choose Between Public or Private Cloud
1. Security
Security is the primary concern for many app developers that build safety-critical apps, and for example banks, insurance companies and many others that handle money transfers, personal information (that is critical by any level) and other critical data. Regardless of whether the app is meant to be data, user and safety-critical, it is naturally always important to rely on services that reliable, provide certifications and take security measures seriously.
When we compare public and private cloud implementations, it’s safe to say that the private environment provides a more secure approach. Despite it is always highly recommended to be demanding on security issues, in too many cases the security topics have been seen as a blocking issue for the use of the cloud. On-premise solutions naturally provide the most secure solution but they don’t run by themselves; they require constant attention, maintaining the infrastructure, devices, tools, integrations and many other things.
This gap between an on-premise solution and a private cloud is actually pretty insignificant nowadays. There variety of ways to build a private cloud environment to be highly secure, providing a lot of great security implementations and methods to get access between private cloud and local development environment. For example, certain data and user protection mechanisms, and other techniques and practices can be utilized in the service provider’s data centers. In addition, firewalls, intrusion detections, SSL and app security methods, 24×7 security measurements and monitoring, and the third-party certifications for security practices can be also implemented on a private cloud.
2. Data Ownership, Maintenance, and Retention
The ownership and the actual physical location of data, tests, app details, run, and many other things look to be the same when public and private cloud implementations are compared. However, private cloud instance always means that it is entirely separated from the public setup, servers, databases, and networks. Private cloud setup means that it is an entity of its own with all these features included, but only dedicated and reserved for one user (customer).
Backing up hard disks, making sure no files, data or other (e.g. user) information is left on anywhere has been one the most important reason for so many companies to use on-premise solutions. However, this again doesn’t change much with the private cloud if implemented the right way. You still do not give access to anyone outside of your organization, all the data, ownership of it and backing it up is taken care for you. Doing this locally isn’t always an easy and straightforward task and can mean some serious operational expenses. Both cloud options provide these kinds of tasks as-a-service.
A private cloud provides here very compelling offering as all data aspects, the maintenance of databases, logs, and user information is taken care for you – with security measures.
3. Performance and Availability
Performance – in nearly all of the cases – is based on the infrastructure and how things have been built. As private cloud is meant to be used only by one company, they do not get any traffic from outsiders, the infrastructure hardware is reserved and dedicated to just one company, they tend to provide the best performance in terms of computing power, fastest access to devices, test runs, results and all other assets.
For availability, a private cloud naturally means those reserved and dedicated devices only for one customer’s use. Those can be shared across different development teams geographically so they’re used at different times of the day. When comparing the availability of the device access and efficient use of the cloud service, it is essential to consider if the service and environment are critical, important or medium-level importance to you.
The uptime and availability metrics should be always high (always over 99.5%) for any good and reliable cloud service. In the case of a sudden event, there should be redundant operations that can get users up and running as soon as possible (for example, using mobile devices from various data center locations). The same applies to backup services so that users can access files, results, and data when something breaks (in cases of hardware failures, etc.). Furthermore, a reliable private cloud service provides 24×7 monitoring for performance, uptime, and accessibility for all assets needed by devs and testers.
4. Support and Self-Service
The old rule of thumb: Great product without great support is like a flower without water. Or some people see it differently: when you don’t need support, everything works like a charm.
The support in the context of the private cloud means that those instances are supported for their users, all infrastructure and devices are up and running. Support is naturally much more than DevOps-like infrastructure maintenance. Both cloud options typically are built around automation, providing a variety of different monitoring capabilities and monitoring of the infrastructure, devices, and databases are taken care of by its provider.
This type of ‘support’ covers also self-service. It’s absolutely a must to have good documentation, how-tos, tutorials, plenty of tips and tricks and best practices material available for the service users. Especially, when we are talking about mobile test automation that can be tricky and relevantly a new topic for many app developers, it is important to have all these help materials available to make sure the service users are ‘happy campers’: excellent manuals, help pages for self-service and manual options via online live chat systems with operations, social media, etc. plus an online channels to provide instant feedback on service. These are typically the highest priority within private cloud setups.
5. Enhancements and Update Policies
Both cloud models – public and private cloud are SaaS type of services and typically with private cloud setups, a ‘service roadmap’ is drafted when service is brought up. This includes all details of how mobile devices are upgraded, what new features are adopted in the infrastructure and service, how users can access the infrastructure and what update policies are adopted.
As this is very different from a local on-premise setup where users define their own upgrades locally, with both cloud options devices are either upgraded to consider all users of those or dedicated users. The biggest difference in upgrade policy is for who those devices are reserved for. With public cloud all users will be considered and typically devices should be upgraded to the latest versions of OS, OEM upgrades installed as soon as those become available, but in case of a private cloud, those are upgraded based on users preferences.
Nevertheless, both public and private cloud setup must have frequent updates for hosted devices, bug fixes or other enhancements for the infrastructure. Cloud models really provide the most efficient option as everything is taken care automatically and users don’t need to do anything. There won’t be any slowdown or downtime for the service itself, no impact on daily activities or need to do any installation or configurations for accessing the latest and greatest.
6. Integrations
Integrating the development and testing environments, frameworks and tools are nowadays pretty easy. Agile process backs up this sort of thinking and many popular CI/CD tools are easy to be extended with additional software. For example, the integration APIs with current tools provide an easy start for any integration and can bring totally different purpose tools together.
The integration is the key to efficient use of all tools and software. In this context, it is recommended that open standards and technologies are used as widely as possible to avoid vendor lock-ins, expensive maintenance costs and also time-consuming customization work between different technologies. Ideally, you should look for cloud-based solutions that provide these capabilities with API, open standards and all required bits and pieces for you to connect with your in-house infrastructure. This will make integrations simple, fast and cost-effective.
7. Usability
Good usability and great user experience are the topic priorities for cloud providers. And as the usability is also one of the top criteria for efficient and productive product development, cloud platform must provide compelling features, support for a variety of different (open source) frameworks. The open-source part is important here as it means both public and private cloud users must have access to all source, all the time. This provides the required flexibility in terms of vendors, frameworks, typically even cross-platform point of view.
The use of the service must be intuitive, easy to start, use and scale-up. When it comes to usability, it’s always a good sign when the community provides constant updates through source code commits, webinars, guidance sessions, and other online events, blogs, help systems, and self-service documentation.
One known factor to build up the confidence and provide a way much better usability experience of the service is when you have a free trial to service to experiment it further. This means transparency and openness – and typically that provides a great foundation for users to get started with the service.
8. Contractual Flexibility – Subscription vs. License Fee
No company wants to lock themselves into a deal that doesn’t provide consistent value to them. The greatest benefit of using subscription models – that both public and private cloud provider – is that you can use it when you need it. Ideally, subscriptions should provide diversity in terms of different features and capabilities – and provide a flexible plan for each type of user/need. The very same applies for migrating or switching between the plans. It is very common that while your development and testing are going on your needs for real devices increase as well.
It’s quite common that contractual flexibility is the highest when the subscription model is offered. For example, with on-premise setups, not only the costs that are coming in from operational expenses, users of that sort of environment still need infrastructure, devices, server machine, additional software installed and many other things that bring in the hidden costs. With public and private clouds the costs are fixed for a period of time.
9. Infrastructure Scalability
With on-premise solutions, you need your own devices and whole in-house hardware infrastructure. Typically, this is pretty straightforward to set up but naturally running, maintaining and monitoring the whole system takes a lot of time, needs to be done 24×7 to get an efficient environment for developers to use. Also, the scalability isn’t necessarily easy, if you don’t have appropriate products that can handle a large number of devices in use, communicate with the rest of the infrastructure and deliver you results.
Typically, the size of internal development environments/labs with real devices varies from 20 devices to several hundreds of devices. While setting up and even operating some of these we have learned a thing or two on how to build, operate and maintain an enterprise-grade test lab. If you’re seriously considering building an in-house test environment, check our 4-episode video series where you’ll find plenty of details.
10. References and Customer Recommendations
Existing customers and users are a great sign of something. Going through the existing user’s success stories, testimonials and case studies can give you the confidence of an existing system in place. Reviewing customer stories and even asking directly from them how they feel about the system is also encouraged. You’ll also learn if they had any other hiccups – not related to the software, service or adoption – but in more general, what were the extra miles in other parts of the development, integration, and adoption they had to go through.
To summarize, these are important ‘metrics’ or features to be asked: performance of the system, in terms of responsiveness and uptime, functionality and how to fully exploit all those great features, usability and what does reference customer like about it, and finally, how does the vendor of a service response and support their customers in case of issues, questions or anything that will help you to get value out of these products/services.
By asking the right questions and knowing what sort of answers to look for, cloud-based solutions are easy to evaluate and thorough understanding of what sort of value they can provide for you.