Skip to content

The most successful mobile app developers build their apps in agile and test their stuff thoroughly before going public. To get this done efficiently and in a productive manner, this naturally means that development, testing and overall product creation process must be seamlessly integrated into one cohesive process. The need for a minimum viable product drives short development cycles, but getting feedback from users (customers) is not the right way to understand how well the app does. Because of this, there are great products and solutions to help developers building better apps.


According to Gartner, worldwide IT spending is set to grow to $3.8 trillion in 2014, as organizations look to invest in growth after some years of sluggishness in growth. This huge investment gives a clear indication of technology’s value – quite simply, as IT forms the backbone of more and more crucial business functions, its proportion of the overall business budget needs to increase. However, not all dollars or euros are necessarily well spent and spur that growth. The efficiency of investment must be understood to get the best outcome for money/investment. Furthermore, Gartner predicts that investments in data centers (large ones) are about to reach $143 billion in 2014. This is a great point to consider how mobile app developers can fully utilize their own assets, build their own mini-scale  ‘data centers’ and what is the most efficient and beneficial model for them to build and test their mobile apps.

Dimininishing Return Curve

This blog summarizes the costs associated in two agile development and testing approach – an in-house solution and cloud-based solution with real mobile devices used in development and testing – by using a basic example of cost comparison in both. Also, this is the third and last blog in our blog series – The Investments for Mobile App Development and Testing.


SaaS-PaaS-IaaSPrimarily driven by costs vs. benefits, time-to-market, and easiness of use, mobile app developers are increasingly turning to SaaS (Software-as-a-Service) offerings to get their apps developed and tested on real mobile devices. Naturally, there are other definitions, such as Paas (Platform-as-a-Service) and IaaS (Infrastructure-as-a-Service) that deliver different types of hosted infrastructure, software, and complimentary stuff as a service. The purpose of all these different options is to provide access to hosted devices and services for development, testing and deployment, plus managing the process. These approaches aim to bring a better level of efficiency and enable a better and easier way to build more robust apps.

One thing that is common in all these forms of services is that the user (developer) gets the mobile devices, required frameworks (e.g. test automation frameworks), tools, APIs and everything ready to be instantly integrated with their own development environments. In today’s agile mobile development landscape, it is also appropriate to consider a shorter lifecycle to evaluate the total cost of ownership.

It is also important to consider trade-offs between CapEx and OpEx within these different options. Those trade-offs are also important in any assessment of Total Cost of Ownership. And this gives us a great comparison point between in-house and SaaS/PaaS/IaaS solutions.

Basic Example – In-house vs. Cloud

The costs of developing a mobile app vary a lot depending on various things. Building an app can basically cost nothing if you “do it yourself quickly” – or it can be comparable to the cost of a car (e.g. smaller companies) – or it can be comparable to cost of a house (e.g. brands, bigger companies building their apps). This is significant range and depends on how complex the app is, does it use any external frameworks (good and bad in the context of costs), does it require any significant infrastructure (e.g. back-end server implementations) and a myriad of other things.

There are several app frameworks that can offer cost savings (e.g. PhoneGap, Titanium). However, these savings come with their own cost. They allow developers to work in a non-native language with their own abstractions. This can create artificial limitations on both functionality and performance.

Regardless of what type of app you are building, it is difficult to compare app development costs, but much easier to do so in the context of testing that app – and how much infrastructure-related costs you. When it comes to testing mobile apps, manual testing approaches are too slow and not agile. The need for using real devices for testing, quick turnaround times, and specialized knowledge makes the testing effort successful and results as a better app. So, what does the infrastructure for development and testing cost?


Total Costs Related to Dev&Testing Environment

The basic assumption is that:

– You are building several apps (not limited here, as this infrastructure can handle all of them)
– You use 50 different devices for testing
– You need everything up and running and someone maintaining the system 24/5 – OR – just business hours

In-house setup
– Devices: ($200 x 50 = $10,000)
– Server infrastructure: ($1,000 per 10 devices -> $10,000)
– Software: ($99/month x 50 = $4,950/month -> $59,400/year)
– Setting up and deployment: 3 business days ($50/hour x 8 x 3 = $1,200)
– DevOps to manage the system 24/5: ($50/hour x 24 x 5 = $6,000/week x 50 weeks = $300,000)
– DevOps to manage the system on business hours: ($50 x 8 x 5 = $2,000/week x 50 weeks = $100,000)

Public cloud setup
– Deployment: 1 business day ($50 x 8 = $400)
– Integration: 2 business days ($50 x 8 x 2 = $800)
– License fee for 50 devices: ($1499/month -> $17,988/year)

Privately hosted cloud setup
– Deployment: 1 business day ($50 x 8 = $400)
– Integration: 2 business days ($50 x 8 x 2 = $800)
– License fee for 50 devices: ($199 x 50 -> $9,950/month -> $119,400/year)

Note! Calculations include all operational expenses for device, infrastructure etc. maintenance and monitoring. The typical life-cycle of mobile devices are 2 years and the server infrastructure 3 years. The set of 50 different Android devices can give you a coverage of  40-60% of Android volume depending on geographical location.

Installation, Configuration and Customization

The typical belief of mobile development and testing costs is that the software used for testing is something that must be tailored for each and every case specifically. However, this is not the case if the right solutions are used. Also, it is not the most expensive cost item to get the environment up and running – and fully operating.

In terms of installation of the solution, one of the biggest benefits of SaaS/PaaS/IaaS is that there is no significant software setup required as it is offered as a service. Only basic deployments and integrations with other development tools and environment may take little time, but typically it is very Install - Setup - Customizestraightforward. The installation of an in-house environment can be also very straightforward as everything is delivered in pre-packaged installation.

In terms of configuration and customization activities, the situation is very similar to the installation: the SaaS/PaaS/IaaS environment is pre-configured and doesn’t typically require any customization work. The only item that requires configuration concerns the device setups and integrating the existing development environment, tools and other software with API that communicates with remote devices.

The major difference comes in customization of the in-house environment. SaaS/PaaS/IaaS products can be updated without users even seeing that, but in-house products needs reinstallation and possible customizations when it comes to new features etc. But generally, when you have the right software in use (e.g. continuous integration + suitable testing enterprise software), the process of configuration and set up the whole environment is very easy and straightforward.


The basic calculation example shown above is pretty much in line with a prior example of how much a company spends money on different sub-items (setup, deployments, installations, hardware acquisition, DevOps etc.).

The costs related to both in-house and cloud solutions vary on the number of different devices that are used for development and testing, but the most importantly it is driven by the decision of how device roster will be maintained and monitored. If you do that (and yes, it must be done) in-house, there are significant DevOps costs but ‘outsourcing’ that to cloud-model and let someone else taking care of those devices and infrastructure will actually save you a lot of time and money. Furthermore, depending on the model you have decided to go with, either public or private cloud, the device availability is 24/7.

Ville-Veikko Helppi

Mobile Testing Product Expert