IT Project Failure Rates: Facts and Reasons

Since I wrote this article, IT Project Failure Rates: Facts and Reasons, several years ago, it continues to be one of the most read articles on my website. Thank you.

Clearly, it continues to resonate, and not for good reasons.

The picture of the frustrated fellow is indeed even more startling now than when originally posted – he doesn’t like the facts and they’re not moving in his favor.  Failure rates on IT projects remain unacceptably high, and the primary reasons remain people and people-systems.

Here are some updated facts:

• According to the Standish Group’s Annual CHAOS 2020 report, 66% of technology projects (based on the analysis of 50,000 projects globally) end in partial or total failure. While larger projects are more prone to encountering challenges or failing altogether, even the smallest software projects fail one in ten times. Large projects are successful less than 10% of the time.
• Standish also found that 31% of US IT projects were canceled outright and the performance of 53% ‘was so worrying that they were challenged.’
• Research from McKinsey in 2020 found that 17% of large IT projects go so badly, they threaten the very existence of the company.
• BCG (2020) estimated that 70% of digital transformation efforts fall short of meeting targets. A 2020 CISQ report found the total cost of unsuccessful development projects among US firms is an estimated $260B, while the total cost of operational failures caused by poor quality software is estimated at $1.56 trillion.

Reinforcing these figures is my experience coaching CIOs and technologists across organizations at all levels, who find equally shocking outcomes. Why is this and why does it continue?

Here are some thoughts on the source of ongoing failures. The reasons sound familiar and are harder to fix than the simple words used to describe them:

First, the right questions about a technology project are not asked when they should be. Before even beginning a project, be certain you can define success, else you’ll meander and never reach an agreement with your partners.

• How does the business define satisfaction?
• What value must be delivered – hence, the original promise – to match the original business case? What is acceptable alignment?
• What is the minimally accepted on-time delivery date?

Second, the process doesn’t tease out the right questions and concerns. People tend to check boxes, go through the routine, without considering the upcoming difficulties. Technology is tough.

• A clear and credible business case is lacking – we take people at their word, which can be superficial.
• Project scope and objectives are poorly defined. (This allows for a lot of wiggle room later.)
• Planning is over-optimistic.
• Zero-based budgeting is ignored.
• Sunk costs take precedence.

Third, users back away due to other priorities.

• Without their ongoing involvement, requirements definitions go either missing or haywire.
• Integration, QA and change management are relegated to those who just happen to have a free moment. Whose full-time job is it?
• User testing becomes a part-time activity.

Fourth, inadequate project management. In all the time I’ve been involved in technology projects, this has been among the weak links. It’s befuddling why, as without it, you have chaos. Imagine building a massive natural gas-fired electricity generating plant without a project management system.

• Respect project managers – they don’t have easy jobs. No one likes them catching us out.
• Senior management cannot connect with why they should sit through a line-by-line review of how the project is going. Their lack of patience is palpable. The best CEO I’ve worked with knew every in and out of the project plan and held people accountable.
• The urgency of the moment displaces all the other project activities, and that will make or break a project. Maintaining long-term focus is essential, though hard.

Fifth, user requirements remain unclear. While few people like writing them or ever outlining them in broad strokes, you need precision about expectations and deliverables, whether waterfall or agile.

• Be sure the users know what they exactly want to see. If not, do not begin the project.
• Under pressure to deliver, there’s a lack of discipline because people just want to get something done. ‘Just give me something’ is a lament I often hear.
• Many believe they’ll figure out what they’ll need as they move through the project, thus directing efforts at an ambiguous and moving target.

Sixth, and a point I missed earlier, is that technologists need to develop a higher level of emotional intelligence, which in turn supports relationship building.

• Technologists always need to resolve conflict (e.g., differences of opinion between them and users, between and among users, and with what is promised vs. what will be delivered).
• They are always under pressure as demand for them exceeds their supply. How do they balance demands?
• Technologists rarely receive recognition. Users think technology development is simple – if I can put an app on my phone in no time and it works, how hard is it to implement enterprise software? Not so in the business world where requirements are complex, organizational politics are manifest, and where a heterogenous technology environment means integration is very difficult.

The five steps I outlined before to improving the ‘people-based’ factors affecting IT delivery still hold, and I’ve added a sixth:

1. Solidify the technology/business relationship via governance – talk to each other, build trust, be transparent (if a delivery is going to be missed or won’t provide the expected functionality, let users know right away). Don’t let process overrule relationships.
2. Integrate technology into strategic planning – sounds simple, but it’s not because people see strategy as some distant need. In fact, everyone needs to lace their everyday conversations with questions about ‘what comes next?’ and ‘where are we headed and why?’
3. Set and share a simple, multi-year roadmap for overall business strategy – but make it sufficiently detailed so that users see a picture with sufficient detail about what they’ll be receiving.
4. Establish an open planning process – gather ideas from all corners of the organization; listen to understand, not to be understood (thank you Steve Covey and St. Francis); and don’t rush to a conclusion. More discussions can lead to a better outcome, even if the upfront process is slower.
5. Teach and promote communication and relationship skills – Covid has hurt this step because you can no longer call an impromptu meeting or chew on the idea over lunch. But we’ll figure this out. Pick-up the phone and talk to your technology colleagues, business partners and senior management. Don’t be shy. Don’t defer.
6. When development is ‘in-house,’ be certain the SDLC foundation is repeatable, flexible and delivers fast results. As organizations grow, business and development demands are more frequent, increasing the risk of non-delivery. Accompanying that is inefficiency due to size. Realize too that organizations and business entities are fluid, not static, meaning change is constant. (Mark Katz, Scrum – A High Level Roadmap, 2021)

Please feel free to reach out to me here.