Agile versus traditional project management
You might be wondering about the difference between agile and traditional project management. These are very different approaches and should be used in different situations. This article will explain what these philosophies are, their pros and cons, and when you might want to use one or the other. So let’s find out about the difference between agile and traditional project management!
What is traditional project management
Since around the 1980s, most projects have been managed according to something called the PMBOK, which stands for Project Management Body of Knowledge. This in turn was based on an existing set of practices that had been in the engineering discipline since the 1960s.
These engineering project management ideas had been translated into the software development domain since the 1960s, based on a 1970 paper by Winston Royce, titled “Managing the development of large software systems”.
This paper birthed the famous (or infamous, depending on your perspective) “Waterfall” model of software project management:
Not a lot of people know this, but Royce himself had concerns about how practical it would be to implement this model.
Features of traditional project management
The traditional “waterfall” approach to project management breaks a large project down into a number of sequential phases, each following on from one another. You cannot move onto another phase until you have completed the current phase.
Hence, it is like water spilling down from one level onto another until that level fills and falls down onto the next level. Thus, the name “waterfall”.
The primary goal of this approach is to capture all requirements up-front, so as to minimise change and reduce the risk of time and cost overruns.
It does this by extensive planning of work and careful specification of features, and by implementing change control processes that minimise changes to the baseline plan.
Advantages of traditional project management
There are number of advantages to the traditional Waterfall project management approach, which I’ll explain here.
Firstly, there is reduced chance of changes to the initial project scope. All changes to scope and specification must go through a change control process, which has some form of governance over the top of it (e.g. changes must get approved by a change control board). This can help to reduce “scope creep” and stop people continually changing and adding to the scope of the project.
Secondly, since there is a lot of up-front planning and estimation, it is possible to provide a detailed set of plans and estimates to stakeholders. While estimation on software projects is notoriously difficult (even with up-front planning and specification), this up-front work should provide some level of guidance and comfort around those estimates.
However, it is a grave mistake to think that estimation simply based on plans and designs will always bear out to be accurate. This will be explained in more detail in the agile approach below.
Disadvantages of the project management approach
There are a number of disadvantages of the waterfall approach to project management (when it comes to software, which is the subject of this blog and therefore article).
Firstly, there is a lot of time spent on initial design, planning and specification, which means a lot of time and money is spent before any software is developed. This means that stakeholders will not get to see any working software until well into the project.
Secondly, there is usually little or no testing done until well into the project – not only until all the plans and specification is done, but usually also not until all the code is written. Pushing all the integration and testing to the end like this can be very risky, and means that some nasty bugs might not be discovered until very late in the process. Most of the time, a bug is much more difficult and expensive to fix the later it is found in the project’s lifecycle.
This chart from the NIST shows the increasing cost of defects based on when they are found.
Thirdly, “controlling change” is not necessarily a good thing. Some changes are bad ideas, but many are good ideas that will increase the value of the project. Implementing change control prioritizes controlling the scope (and therefore cost) of the project, over the value of the project.
If adding a feature to a project will cost an extra $10,000 but bring in an additional $30,000 in revenue, it should definitely be included. Of course, the revenue increase number is an estimate (aka wild guess), but so is the cost estimate. The agile approach (which I usually favor) “embraces change” rather than fighting and resisting it.
The alternative agile approach to project management
In the 1990s, many people became concerned with these problems in the waterfall approach to software project management, and the agile community was born. They proposed a very different way of working, based iterating over and over. Each iteration involved a small team collaborating closely to design, build, test and release a small increment of software. The most well-known agile framework is Scrum.
Features of agile project management
The agile approach differs in many ways.
Firstly, it is based on multiple iterations, rather than one big release. If you are just doing one big release, you’re not really doing agile, even if you call yourself agile or give me a scrum-master certification. The idea is to reduce risk by building and releasing software to stakeholders and customers early and frequently.
Secondly, it reduces risk by building in quality from the beginning, rather than testing for it at the very end. Testing is something that everybody does and should be baked into the development process (through automated unit and integration tests) from the very beginning. In fact, if you are doing test driven development, you should be testing even before you start coding! (Yes those tests will of course break – but then you go into the famous “Red Green Refactor” cycle I talked about here).
Thirdly, instead of big component teams that work on separate layers that are integrated at the end, agile is based on small autonomous cross-functional teams. Each team is responsible for building a small end-to-end feature, rather than a layer that cannot deliver value on its own.
Advantages of agile project management
There are a number of advantages to the agile approach over the waterfall approach.
Firstly, you get software shown to stakeholders and delivered to customers much sooner. This can de-risk your project, especially in terms of “value risk”.
Traditional / waterfall project managers are definitely concerned with risks, but they are usually interested in risks around the cost or time or scope of the project. They are not usually interested in value. That is a problem “for the business”.
However, the agile approach prioritises the value of the software over cost or time or scope control. Getting software in front of people early on can really reduce the value risk of the work and can tease out what features are actually going to be important (or difficult) and which are not.
Also, early integration and testing can radically improve the quality of the software. Late and infrequent integration and testing can lead to the dreaded “integration hell” period of waterfall projects, when huge chunks of code that have been shelved on private branches for weeks all gets thrown together at once. This can lead to big merge conflicts and a huge number of defects found in integration testing.
The biggest advantage though is that the flexible approach to scope means that agile teams can move quickly to adapt and find the most valuable things to work on. Remember, it is estimated that 80% of software never gets used. Agile proponents believe it is the job of agile teams to find those useless or unpopular features, avoid them altogether, and try to find the 20% that is really valuable.
Disadvantages of this agile approach
There are also disadvantages to the agile approach. If the stakeholders of a project know exactly what they want up-front, then the loose agile approach may not suit them at all. Agile definitely offers less certainty over scope, time and cost – it embraces change to maximise the value of the work, rather than controlling change to control the cost.
So if scope and cost control are a priority, then agile will not suit very well. If on the other hand you are uncertain about value and what scope you should be prioritising, then an agile approach would actually be advantageous.
An agile approach often produces less documentation, since it prioritizes “working software over comprehensive documentation”. If you are working on a project that requires a lot of documents to be produced (e.g. for regulatory or compliance reasons), then agile might not be a great fit.
Comparison and summary
The two approaches are very different and are really designed to solve different problems.
The agile approach is better suited to “building products under conditions of uncertianty”. If you are creating a product from scratch, and you have a lot of risks and uncertainties about your value proposition and scope, and you don’t really know what to prioritise or what customers want, then agile is a good choice.
If however you have a big block of scope defined and locked in, and want to carefully control the delivery of that scope and make sure that the plan doesn’t change, then Waterfall could be a better choice.
I like to think of the difference between the approaches as between “product” management and “project” management. In fact, if you read the Scrum guide, you will not see the word “project” anywhere! It defines itself as a framework for “product development”.
So waterfall can, under certain conditions, be appropriate for certain projects – but keep in mind the severe risks around late integration and testing.
