Skip to content

An increasing number of app developers are migrating from the in-house to cloud-based development environments in order to build their apps more cost efficiently, with the allure of lower maintenance and operational costs. While these are potential benefits for a variety of app developers, not all companies can rely on cloud-based environments due to regulatory, security and privacy risks. In fact, all app developers must carefully evaluate their needs before committing to either approach to avoid compliance issues and unforeseen expenses.

In-hosue vs. Cloud

Striking a balance between the use of in-house and cloud-based development environments depends on the company’s needs. Making the move from a traditional in-house development environment to cloud-utilizing service can be a tough decision for many mobile developers. We’ll look at some of the characteristics and benefits of using either approach in this blog, but don’t forget to download this ebook telling about the process of building up your own in-house testing lab.

Gathering the right information is the key to understanding which solution fit better and ultimately provides you the value. This is the second blog in our latest blog series The Investments for Mobile App Development and Testing.

1. Security

A secure app development environment is one of the most important concerns for many bigger brands building their apps on mobile platforms. In fact, many companies shy away from cloud-based environments because they fear any sort of security breaches. This is typical for banks, VPN and SSLinsurance companies and any other companies that deal with money, personal information or any other critical data. It is always a good idea to be careful, but in too many cases the security topics have been seen as a blocking issue for the use of cloud-based environments. In-house environments that do not connect to the internet are typically the most secure, however, they come also with certain challenges and users doesn’t necessarily get the best value for their development.

When relying on cloud-based development environments, it is important to know what data and user protection mechanisms, techniques and practices are utilized in the service provider’s data centers. For example, firewalls, intrusion detection, SSL and app security methods, 24×7 security measurements and monitoring, and the third-party certifications for security practices may be needed in order to ensure certain clientele to use cloud-based services instead of in-house environments.

2. Data Ownership and Retention

With an in-house development environment, the data ownership and retention are pretty clear and on your own responsibility. Backing up your hard disks, making sure your system is not accessible by outsiders, and that the all data – results, tests, source code, and naturally your app and other valuable assets are maintained by your own organization. This naturally isn’t always as easy and straightforward task – and can cost you some money as operational expenses.

Sharing the data across your organization needs to be under control but fully configurable so that all teams/individuals do have access to required data. If you are using a cloud-based development environment, one of the most important questions to be answered by the provider is how easy it is to access and configure and what the general guidelines of update policy are.

Good characteristics for a good cloud-based development environment are that all data related to you, your application, and so on, are only accessible by you and it is not possible to be accessed by a not authorized instance. You also must have 24×7 access to all your data and it must be fully transferable back to your local environment (e.g. solid integration between the instances using API).

3. Performance and Availability

Whether you are using in-house or cloud-based development environments something can always happen to the system and cause you downtime when the Performance and uptimedata, apps, tests or services are not accessible. It is essential to consider if the service and environment are critical, important or medium level importance to you. Even not able to use the internet will block the use of cloud-based service but the system built internally can still keep going. These sorts of issues related to reliability must be considered when selecting the right platform for your development.

The uptime and availability metrics should be always high (over 99.5%) for any good and reliable cloud-based service. In case of a sudden, dramatic event, there should be redundant operations that can get users up and running as soon as possible (e.g. various data center locations). The same applies to back-up services so that users can access files, results and data when something breaks (e.g. hardware failure). Furthermore, a good cloud-based service provides 24×7 monitoring for performance, uptime and accessibility for all assets needed by developers.

4. Support

It has been said that a good product without good support is like a flower without water: You may have things up and running, but to keep it running you need someone to take care of those devices, infrastructure and so on. The good product itself may allow users to get their things done, but good support fully complements the service and enables its users to fully utilize all features and benefits built in.

Naturally, there are different types of supports available for both in-house and cloud-based development environments. Some require human interaction, some are more like a self-service and enables users to find out more information. This can be very important especially in case of challenging test automation where frameworks, tools, and other technical issues may not be trivial to everyone. It’s important to have flexibility in support: excellent manuals, help pages for self-service and manual options via online live chat systems with operations, social media etc. plus an online system for submitting service tickets.

5. Enhancements and Update Policies

Enhancements and Update policiesEvery service – either delivered as an in-house solution or cloud-based SaaS type of service – must include ‘service roadmaps’. This means that products are constantly upgraded, new features are built, and how users can access this information and the new version of the software. With cloud-based development environments users don’t even necessarily notice that service has been improved but in case of an in-house system the manual software update is typically required and can expose the system to various things.

A world-class cloud-based development environment must have frequent updates, bug fixes, or just incremental enhancement builds, with new user-desired features rolled out. One of the best benefits of a cloud-based development environment is that users will get everything automatically and there is no disruption for any of their daily activities, no downtime, no need to install or configure new features.

6. Integrations

Integrating the development and testing environments, frameworks and tools are nowadays pretty easy. Agile process backs up this sort of thinking and many popular CI/CD tools are easy to be extended with additional software. For example, the integration APIs with current tools provide an easy start for any integration and can bring totally different purpose tools together.

The integration is the key to efficient use of all tools and software. In this context, it is recommended that open standards and technologies are used as widely as possible to avoid vendor lock-ins, expensive maintenance costs and also time-consuming customization work between different technologies. Ideally, you should look for cloud-based solutions that provide these capabilities with API, open standards and all required bits and pieces for you to connect with your internal infrastructure. This will make integrations simple, fast and cost-effective.

7. Usability

Good usability – also good user experience got from the use of a tool – is one of the top most important criteria for any development environment. UsabilityEspecially in cloud-based environments, it must be intuitive, easy to use and straightforward to adopt for your organization’s daily needs. We briefly discussed the Total Cost of Ownership but it is also closely related to usability as if the system is not that easy to use, configure and won’t fit in your organization’s needs it may be more expensive in terms of not getting fully utilized and used.

The live demonstrations, as well as an active community behind the cloud-based development environment, are generally good signs of a good product/service. Monthly, or even more frequent, webinars, online events, Q&As, blogs, help portals and all these additional sources of information can make cloud-based development environment more useful. Plus not forgetting active guidance on how and what kind of resources should you consider for each job!

It is also easier to provide a free trial in case of cloud-based development environment as an in-house solution might require local installations, configurations and basic adaptations to get the system up and running.

8. Contractual Flexibility

Nobody wants to lock themselves into a deal that doesn’t provide consistent value. The big benefit of using subscription models is that you can use it when you need it, but with in-house products, this isn’t always the case. Sometimes you might not have things running but the license fee remains the same. Ideally, subscriptions should provide diversity in terms of different features and capabilities – and provide a flexible plan for each type of user/need. The very same applies for migrating or switching between the plans. It is very common that while your development and testing are going on your needs for real devices increase as well.

9. Equipment Scalability

ScalabilityWith an in-house solution, you need your own devices and rest of the hardware infrastructure. Typically, this is pretty straightforward to set up but naturally running, maintaining and monitoring the whole system takes a lot of time, needs to be done 24×7 to get an efficient environment for developers to use. Also, the scalability isn’t necessarily easy, if you don’t have appropriate products that can handle a large number of devices in use, communicate with rest of the infrastructure and deliver you results.

Typically, the size of internal development environments/labs with real devices varies from 20 devices to several hundreds of devices. While setting up and even operating some of these we have learned a thing or two on how to build, operate and maintain an enterprise-grade test lab. If you seriously consider building an in-house development environment, please check out all possible information about these. Naturally, the easier option to up-scale the hardware environment – and those real devices – is a privately hosted and dedicated devices for your needs.

10. Recommendations and Customer References

Existing customers and users are a great sign of something. Going through the existing user’s success stories, testimonials and case studies can give you the confidence of an existing system in place. Reviewing customer stories and even asking directly from them how they feel about the system is also encouraged. You’ll also learn if they had any other hiccups – not related to the software, service or adoption – but in more general, what were the extra miles in other parts of the development, integration and adoption they had to go through.

To summarize, these are important ‘metrics’ or features to be asked: performance of the system, in terms of responsiveness and uptime, functionality and how to fully exploit all those great features, usability and what does reference customer like about it, and finally, how does the vendor of a service response and support their customers in case of issues, questions or anything that will help you to get value out of these products/services.

By asking the right questions and knowing what sort of answers to look for, cloud-based solutions are easy to evaluate and thorough understanding of what sort of value they can provide for you.

Ville-Veikko Helppi

Mobile Testing Product Expert