The most common reason Agile fails
There are plenty of examples of Agile going wrong. Big government projects trying Agile and crashing and burning. Or “Zombie scrum”. Or “Cargo Cult Agile”. These are all sad stories and all too common. So what goes wrong? There are lots of reasons. Some of them are cultural, some of them are technical, some of them are psychological, some of them are financial. But I think the most common one that ties many of the reasons together, is what I call Scrumfall. This is not the same as Scrum-but, which I talked about earlier. This is worse.
Scrum-but is where an organisation implements Scrum, but leaves out some important bits or messes with some of the concepts in an inappropriate way. Scrumfall is a big, lumbering form of horrible Cargo Cult Agile that calls itself Scrum but really has not much to do with Scrum or Agile at all.
Traditional view of IT in an organisation
The traditional view of IT (development) in an organization sees it as a manufacturing function, whose job it is to turn ideas and money into software. And there is then an “Operations” function, whose job it is to take the output of the development work and put it into production and keep it running. So there are three separate departments, each fulfilling their own function, in a separate way, in a certain sequence. It looks like this:
There are two huge problems with this picture. Firstly, IT is separated (i.e. locally optimised) in time. This is the basic Waterfall vs Agile dichotomy: business comes up with a big idea, then throws it over the wall to IT to build it, who then throw it over the wall to Operations to deploy and run it. Secondly, it is separated (i.e. locally optimised) in space. That is, IT development is seen as a discrete entity in a value chain, entirely separate from “the business” and “operations”.
How Agile is supposed to work
Remember, we’re supposed to be defining and delivering small chunks of software. Going from what the Poppendiecks call “concept to cash“. So rather than the Waterfall model described above, it is supposed to look something like this:
How it often ends up
Most organisations trying to do Agile end up, however, implementing “Scrumfall”, which looks like this:
You have a big up-front “planning” phase, where projects and business cases are defined, budgets are assigned, lots of business analysts and product managers write lots of documents, and architects draw lots of diagrams. Then you go into “development”, which is an Agile thing, right? Because we’re doing Agile, right? And now suddenly we have four months to launch this thing so let’s get moving!
Scrums, sprints, quickly quickly quickly! Everyone do standups and burndowns and VMBs and put post-it notes everywhere! And then the proverbial hits the fan, the release date comes, and there is a big horrible release. Operations don’t really know what’s going on, the deployment scripts sort of worked in Test but production is different so the configurations are all wonky. People freak out and work all weekend but the thing eventually launches and goes live. Then, after a short break, we do it all over again.
Scrumfall is neither Scrum nor Agile
This all-too-common scenario is not Scrum nor is it any type of Agile. It is classic Cargo Cult Agile: use all the names and rituals and procedures, but there is nothing Agile about any of it. Applying “sprints” and “scrum teams” to a big development “phase” is not achieving much at all. The fundamental point of Agile is not just changing the methods by which a big development chunk is done, but to rethink what software development is. It is not a big discrete manufacturing phase in a value stream, but an integral part of how a business builds a new valuable technology product under conditions of uncertainty.
Product Management needs to be Agile too
Remember, big up-front project planning project budgeting are bad. They’re a relic of Waterfall times and command-and-control structures. There is no point doing software development in short sprints if you’ve already locked in your scope and baked in your benefits. We are building products, not delivering projects. You need to learn as you go. And no more of this “us versus them” mentality of business and IT. Product and technology are all one team. Technology is just a particular implementation or manifestation of a product vision, just like marketing and communications are.
Operations needs to be Agile too
Looking at the Operations side of things, there is no point doing software development in short sprints if you build up all your code for months and then throw everything over to Operations for a big bang release. You should be pushing something to production every sprint. Operations should not be a silo waiting to be given a dump of unfamiliar code. You should be working with them continuously, throughout development, as you build iterations of your product. Or better yet, make Operations a function or manifestation of your product vision, just like software development is. Don’t throw it over the wall, make it an activity that your team does and owns. Engineers build it, test it, deploy it, and run it. This is DevOps, and it is closely related to Agile.
Rethink your value stream
Agile requires a rethinking of your value stream. Software Development is not a “manufacturing shop” into which the business throws ideas and money and patiently waits six months while the “nerds” spit out some software. Software development (done right) is not a manufacturing process at all; it is a collaborative, creative and emergent process by which value can be discovered and created. That is the true spirit of Agile: empowered people collaborating to quickly build, learn and modify valuable software. Stop throwing things over the wall. Shorten the feedback loops. And stop thinking you’re doing Agile just because you are building a six-month project in a sequence of two-week sprints.
