Poor-quality software has huge and growing economic consequences for organizations in the United States. But what are the actual monetary costs – and how can your technology company mitigate them?
How Do We Define Software Quality?
‘Software quality’ is ambiguous. A variety of models have existed and evolved to describe ‘quality’ in the context of software development. Factors like ongoing maintenance and stability heavily affect the perception of quality for software, over a long-term basis.
Each organization has its own requirements and, therefore, a unique definition of ‘quality’ to determine. However, there are four common aspects of software quality to consider.
- Requirement conformity
– Conforms to clearly-stated requirements
– Deviation from the requirements is regarded as a defect
– Few defects are found
- Fitness for use/purpose
– Meets user expectations and needs
– Offers strong user satisfaction
- Standard compliance
– Compliance with specific external/internal standards for relevant industries and organizations
– Conformity with required quality/process standards (EG. ISO 25000, CMMI level)
- Additional aspects
– Structural quality (EG. complexity)
– Aesthetic quality (EG. appearance)
How Do We Measure It?
As ‘software quality’ is ambiguous, so are the methods for measuring it. However, a few metrics are commonly used.
Defect trend over time
- High-quality software features a decreasing curve
- Poor-quality software features an increasing curve
Testing code coverage
- A limited metric, as it doesn’t factor the quality of the tests
Indicators of sub-par code
- Cyclomatic complexity
- Depth of inheritance
- Degree of class coupling
- The effort required to understand the function of a piece of code
What Causes Poor Software Quality?
Poor-quality software has a variety of causes that can be predicted and controlled – but only if they’re understood.
- Insufficient domain knowledge (causes poor requirements)
- Insufficient technology knowledge (causes uncertainty about component quality)
- Unrealistic schedules (caused by poor estimation practices)
- Badly engineered software (caused by poor practice and/or weak software engineers)
- Poor acquisition practices
- Breakdown in team communication and coordination
- Insufficient data for measuring software quality
How Much Does the United States Spend on Software Each Year?
To calculate the cost of poor quality software for the United States, we first need to determine how much is spent on software each year. And this is not a simple task. Fortunately, CISQ published a report, ‘The Cost of Poor Software Quality in the US: a 2018 report’ that explores this topic in detail.
How much was spent on IT products, services, and labor in 2018?
- $2 trillion+ in the United States (estimate by CISQ)
- $3.7 trillion globally (estimate by Gartner)
- $6.3 trillion globally (estimate by Apptio)
Gartner estimates the global spend on IT enterprise systems in 2018 was $3.7 trillion dollars (a 6.2% increase from 2017) based on the cost of ‘Data Center Systems, Enterprise Software, Computing Devices, IT Services and Communications Services’.
Enterprise software was forecast to experience the highest growth in 2018 at 11.1% (around $400 billion), with spending on apps and infrastructure expected to grow through 2019.
In contrast, Apptio estimates global IT costs in 2018 at $6.3 trillion by including labor (potentially 45% of IT spending). However, this figure may not include ‘embedded systems, IoT, lost market share, stock declines, legal costs, etc.—and other costs associated with problematic IT systems and services’.
CISQ estimate that by adding these missing categories to Apptio’s calculations, the total US IT spend on products, services, and labor in 2018 is likely to exceed $2 trillion, or 10% of GDP (US GDP in 2018 was $20.4 trillion, or about 23.3% of the world economy.
How Much Does Poor-Quality Software Cost?
Most technology companies don’t collect Cost of Software Quality (CoSQ) data, so they’re unable to answer two important questions:
- How much in cash value does poor-quality software cost our organization?
- What kind of ROI do we get from investing in software quality, considering the overall costs of quality and cost of ownership for software assets?
Software is different from manufactured items in a number of ways:
- High fixed costs and lower variable costs.
- Wear and tear isn’t a factor – but maintenance is.
- Hardware can be added to yield extra value.
- Inherently more complex.
- Less tangible and visible.
Poor-quality software can be grouped into four categories.
External Deficiencies and Failures
Estimated costs of external deficiencies and failures total $1.43 trillion. Leading causes include finding and fixing operational deficiencies, and massive failure consequences.
Legacy systems are dominant in most IT shops and are estimated to have cost around $635 billion in 2018
Massive failures and cybersecurity vulnerabilities
Massive failures happen at an increasing rate and cybersecurity vulnerabilities are rife throughout software infrastructure and technology stacks.
How much did malicious cyber activity cost the United States in 2018?
- $126 billion (Council of Economic Advisers)
- $1.11 trillion (CISQ extrapolated from Lean IT)
- $1.43 trillion (median average of both estimations)
Internal deficiencies and failures
Estimated costs of internal deficiencies and failures total around $500 billion. Leading causes include finding and fixing unreleased deficiencies, rework, canceled projects, and troubled projects.
Canceled IT projects
Around one-third of current IT projects will be canceled or fail significantly, which puts around $300 billion of the US IT labor base at risk. Poor software quality is a leading cause of schedule slippage and cost overruns.
Estimated project losses
- $130 billion lost in troubled projects.
- $47 billion lost in canceled projects.
The estimated costs of the technical debt total $.54 trillion. Leading causes include violations of good practices, and fixing problems that may cause future disruptions.
The estimated costs of management failures seem impossible to calculate. Leading causes include unanticipated costs, customer damages, and reactive crisis-mode management behaviors.
How Much Did Poor Quality Software Cost in 2018?
With this data available, we can estimate the total cost of poor-quality software for the United States in 2018 at $2.84 trillion.
Poor quality software cost an estimated $2.84 trillion in the US alone in 2018.
How is the total CPSQ for the United States in 2018 broken down?
- External failures and deficiencies – $1.43 trillion
- Internal failures and deficiencies – $.8 trillion
- Technical debt – $.54 trillion
- Management failures – unknown contribution at this time
How Can Your Technology Company Reduce the Cost of Poor-Quality Software?
Each organization’s challenges and definition of poor-quality software are unique. But there common causes of poor-quality software to consider.
General recommendations include:
- Identify and fix problems and inefficient processes as early as possible in the development pipeline – ideally before they happen. Agile, DevOps, and continuous testing can help achieve this.
- Commit to measuring your organization’s CPSQ, so you can make an informed business case for investing in new software quality processes and tools.
- Track the results of software quality in both your business and relevant benchmark organizations.
- Sensible areas of focus will likely include the cost of ownership, profitability, impact on human performance, innovation enablement, and the effectiveness of mission-critical IT systems.
Your organization can improve the quality of its mobile software by investing in a dedicated tool, like Bitbar’s test automation platform.