What does “project success” mean?
A lot of people probably assume that identifying success for a project is simple. Well it actually is a bit more complex than you think. This article will outline the conventional way people see it, and then show that it has a lot of problems. And suggest some alternatives.
What is project success?
A few years ago, I did a training course in project management. Right near the beginning, they told us about the dreaded Chaos Report. Ever heard about that?
This was a yearly document produced by the Standish Group (an IT consulting firm) that produced alarming statistics on the vast number of failed projects. We were told that at the end of the course, we would be armed with PMBOK project management techniques that would help us to avert this terrible fate.
Not long after I discovered Agile and found that a lot of the PMBOK material I was taught wasn’t very useful. But recently I’ve decided that the actual problem isn’t PMBOK versus Agile or Prince2 vs Lean or anything like that. We don’t have a problem with too many projects failing (read here for a recent challenge to this paradigm).
The problem is, we don’t even know what it means for a project to succeed or fail.
What counts as project failure?
The traditional definition of project success (and that used by the Standish Group) is that a project succeeds if it delivers agreed scope, on time, and on budget. The project is “challenged” (what the hell?) if it was completed but it didn’t deliver all scope items, and/or went over budget, and/or didn’t complete in time (as per original schedule). The project is “impaired” (what on earth?) if it gets cancelled at some point.
I am going to challenge all of the assumptions behind this ridiculous model, more specifically:
- the fundamental criteria for project success areif it delivers scope on budget and on time
- the corollary of the above, i.e. a project is a failure if it doesn’t do those things
- cancelling a project is a bad thing.
The iron triangle
Traditional project management tells us that we are operating with a fixed “iron triangle”: Scope, Cost, Time. You can get more scope, but it will take more money and/or time. You can spend less money, and deliver quicker, but you’ll have to drop scope. The job of the project manager is to control the triangle.
If you do, then you deliver all the scope on budget and on time, and you therefore “win”. If you fail, and the triangle goes wonky, the project “fails”. The problem with this is obvious: it has nothing to do with whether the project is any good or not. That is, there is nothing here about benefits.
Consider two examples, project A and project B. Each project begins with fixed scope, a six month timeline and a $1 million budget. They are expected to return benefits of $2 million at the end of the year.
Project A completes on time, delivers all scope, and delivers on budget. It provides benefits of $100,000, however. Project B is six months late, at a spend of $2 million, only delivers half the scope, and provides benefits of $5 million.
If you do the financial analysis (let’s leave out discounting time value of money for now), project A provides a net value to the firm of negative $900,000, while project B delivers a net value of $3 million. Yet according to our definition, project A is the success and project B is a failure.
What? How did this madness take hold? There are three problems going on here.
Problem 1: most of your scope is bad
Take a look at any software project. It might be your current one, it might be your competitor’s one, it might be the one you did last year. Most of the scope is probably bad. Maybe even worthless.
I don’t care how good your story mapping is, I don’t care how amazing your business analysts or UX designers are, most of your scope is rubbish and shouldn’t be built.
A bunch of people have done a lot of research here, and the estimate is that around 80% of software that gets built never gets used. Microsoft found that when they made changes to their websites, 20% of the changes worked, 60% of their changes did nothing, and 20% had the opposite effect of what they were trying to do. That is, 80% of the work they did was useless or worse than useless.
People need to wake up to this fact. And if 80% of your scope is worthless, you probably shouldn’t build it in the first place. The job of a software team is to work very fast and hard at identifying that 80% and to not build it.
So just because you delivered all your scope doesn’t mean your project was successful. In fact, some or most of what you delivered will provide zero or negative value for your company.
Recognising and accepting this is one of those important “a-ha” moments in your agile career. If you haven’t made that realisation yet, you really need to. Remember the tenth agile principle, the one about Simplicity? It’s becoming my favourite one: Simplicity–the art of maximizing the amount of work not done–is essential.
Problem 2: It’s not your spend that counts, it’s your return
Financial analysis tells us that the value of a project is the present value of that project. This is found by subtracting the present value of the cost of the project from the present value of the income from the project.
If a project goes over budget, but delivers huge returns (far exceeding what anyone expected), I would call that a massive success, not a failure. If a project comes in on time and on budget and does nothing for the business, I can’t see how you can call that a success. It’s just another sad pointless project in an endless line of sad pointless projects.
Problem 3: the sooner you kill a bad idea, the better
Project Management tells us that the greatest calamity of all is “impairment”, to cancel a project before it is completed. If that happens, then you’re definitely not getting any benefit from it, but you’re not spending more money on it either. If it is going to delivery zero or negative returns to the firm, like most projects do, you have a duty to kill it as soon as you can, and you’re doing everybody a favour if you do.
The likely response from project managers, and my response to them
I’m sure the traditional project managers would quickly respond like this: “This is totally unfair. You’ve shifted the goalposts. What you’re talking about is Benefit Management, which is not what we do. Go look in the PMBOK book. You won’t see Benefit Management listed as one of the eight project management processes. That’s for the business to solve. I’m just a project manager! It’s not my fault if the business comes up with crap ideas for stupid products, it’s just my job to give them whatever they want, when they want it!”.
My response would be that that is a total cop-out. If project management has nothing whatsoever to say about ensuring a project delivers any value for a business, it is a worthless discipline. Decisions about scope, cost and time should be made purely based on how those affect the return to the business.
This isn’t some 90s hippie Agile philosophy, this is simple business acumen. And if project managers don’t have it, they need to get it, or be made obselete.
