When to use Agile vs Waterfall

If you read this blog, you probably think that I really love Agile, and really hate Waterfall. Well, that’s not true. I think Agile (when done well, which it often isn’t) is a good methodology for some contexts. More specifically, agile is a good methodology for building software under conditions of uncertainty.

If you are not under conditions of uncertainty, then Agile might not be best for you. I would actually recommend Waterfall (or something pretty close to it) for some types of work. Agile isn’t a silver bullet, it doesn’t guarantee success, and it isn’t always appropriate. When to use agile vs Waterfall depends on the context of your work. Let’s see what those contexts can be.

Agile is well-suited to (certain types of) uncertainty

Agile is very well suited to projects where there is a lot of uncertainty about what sort of thing you are building, who will want it, how much they might want to pay for it, and how they might want it to look and work. That is, when there is a lot of uncertainty about the customer and the value proposition.

Agile works well here because instead of guessing and betting about many unknowns and building a huge thing and hoping that people like it, you design and build small bits in small chunks. Then you learn and adjust as you go. Waterfall is better suited to projects where there is a strong shared knowledge and understanding of the customer, the value proposition, and how the thing will be built and how it will look and work.

For example, if you run a couple of hardware stores and they work well and you want to open another hardware store, there would be no reason to do that in an “agile” way. You know exactly what a hardware store is and what makes it work and how to make it successful.

However, if you wanted to suddenly branch out and do a website where people buy hardware goods, Agile might be a good choice. Why? You probably don’t already have a hardware store website (if you did why would you make another similar hardware store website? One is surely enough). You don’t know if people want to go there instead of going to a store. And you don’t know what technology to use to build it, you don’t know what products or segments to focus on, and so on. So you could do it in an Agile way. Start with a simple website that just did a couple of things and then add or change it based on new information as it comes in.

What about project uncertainty?

Some people say you should use agile for everything, including bridges or whatever because there is always uncertainty. If you’re building a bridge, you have uncertainty about the weather, about your materials and resources, about government regulations, about cost and time overruns, about random disruptive events like strikes or festivals, and so on. Those are projects risks. They affect everyone. They aren’t the sort of risks I’ve been talking about and they aren’t the sort of risks that Agile is well suited to dealing with.

Types of risks

There are basically three types of risks when you are working on something:

• Project risks. Every piece of work has these: will it be delivered on time, will the resources turn up and cost as much as we thought, will key people quit halfway through the work, etc. Agile and Waterfall projects both have these risks and are both capable of dealing with them, using traditional project risk management techniques.
• Operational risks: will this thing work once we have released it? Will it cause our infrastructure to fall over? Could it have security holes that let people steal our money or leak personal information? Will it get us in trouble with regulators? Every project has these risks and can deal with them. A dedicated operational risk team usually looks after these.
• Value risks. Will this thing be valuable? Will people want it? Might they tell their friends to get it and use it? Will they want to pay for it? Will they want to pay enough for it to justify the cost of building it? Both Agile and Waterfall projects face this risk, but Agile is much better suited to mitigating this type of risk through early and frequent delivery of valuable working software to customers.

The entire body of traditional Waterfall and PMI (Project Management Institute) methodologies have very little to say about the third type of risk. Which is strange, because, for software projects, it is often the biggest risk of all. The reason they don’t is that they consider it to be outside their field: it comes under Benefits Management, which is a problem for “business”, not “technology”. I wrote a lot about that here and why I think it’s a bad way of thinking.

Waterfall is well-suited to projects with high levels of (value) certainty

Waterfall is a methodology based on carefully defining and documenting the thing you are building before you build it. Then you move the whole thing through various big phases, like an assembly line. First you design it, then you build it, then you test it, then you release it. It’s well-suited to work where you have a very good idea:

• of what you are building
• and of the best way to build it
• and for whom you are building it for and how they will want it to work and how much they want to pay for it.

It’s also well-suited to projects where a business (the “client”) pays another company (the “vendor”) to build some software. Agile / Scrum doesn’t work well with these models. That’s because one of the fundamental concepts in Agile software development is that the product owner is on the same team as the developers. They’re not a visitor or a guest or a client or a customer, they work together on the same team. You could try that with a client/vendor model but it is difficult to get it to work.

But waterfall is well-suited to projects where quality is really important, right?

Some people think you should use Waterfall for something where quality is super-important, like flight control systems or medical device systems.

If you’re building something with life-or-death consequences, then obviously quality dominates the iron triangle completely. A lot of those projects do use Waterfall or something similar to it. But that’s not because of quality.

You can build quality into Agile projects just as much as you can in Waterfall projects. You can also have bad quality in Waterfall projects just like you can in Agile projects.

In fact, given Agile projects tend to do more test automation (due to frequent changes and Continuous Integration practices often used), I would say quality can often be higher on Agile than on Waterfall projects.

The reason those types of mission-critical systems are often done in a Waterfall way is the reason I mentioned above. The value criteria are usually very well known and understood up-front.

If I’m designing a medical imaging system, I know exactly what I want it to do. It needs to take medical images, perfectly accurately, every single time, and never anything else. I don’t need to build a minimum viable product and get it in front customers and collect feedback and iterate. I just need to spec it very carefully, design it very carefully, then build and test it very carefully. Then release it and hopefully, it works.

Are there parts of Agile you can use anywhere?

There are some concepts in agile that can be applied to lots of projects, even ones suited to Waterfall. For example, “inspect and adapt”. Inspecting something you’re working on, and using that information to make decisions, is a good idea in any context. “Empowered teams” is a good idea too. People perform better when management trust them and give them autonomy over their work.

But these are basic principles of people and project management and are not specific to agile. If organisations aren’t following them, then that’s a sign of a dysfunctional organisation, rather than an “agile vs waterfall” problem.

Many organisations say they are doing agile and follow the rituals. But then they don’t follow good practices of management, like respect and transparency. Those organisations are not only doomed to sub-optimal performance but give agile a very bad name.