Agile is not just about software
We all know the stories about agile failures. About things like Flaccid Scrum, Cargo Cult Agile, and all the others. We all hear the tales of micromanaging PMP-certificate wielding “scrum-masters”, absentee-landlord product owners, and merciless stakeholders. What’s gone wrong? A few things. But mainly, organisations are signing up to Agile for the wrong reasons, and they’re not making the right changes. They’re trying to get away with the least possible disruption: just changing the way software development work is structured. Agile is about software development. But Agile is also about much more than that. It requires changes to the entire organisation.
Companies see IT as a weird separate function
The traditional view of IT is that it is a weird specialist engineering function, separate from the rest of the organisation, and very separate indeed from anyone in the “business”. Those people have jobs that involve understanding customers, and business, and money and making more money. And what do engineers know about that? Nothing! They’re just a bunch of nerds, right? We should just use our business smarts to figure out what to build, and then tell them to build it, and wait until it’s finished, right?
If that is the company’s attitude, they may as well just go all the way. Sack all the IT people, and hire some overpriced vendor or agency. Then we can all pay through the nose, get something nobody wanted, and bring in the lawyers and argue over contracts like the good old days in the 90s.
This view is, of course, a profoundly stupid way of looking at technology. We are now living in the 21st century. IT has become a fundamental part of how all firms operate and interact with customers (and suppliers, partners, and so on). Business and technology need to work together. They can’t be at an arms’ length from each other anymore. In fact, we need to disolve the distinction between business and IT altogether. And we need to start applying Agile principles to not just the way we build software, but the way we run businesses.
Times have changed
We all know times have changed. We live in a completely different world than the one we lived in 30 years ago, let alone the one we lived in 90 years ago. But we still structure organisations, manage projects, choose strategies, hire and reward people, draw up budgets and fund work the same way we did 90 years ago. The days of scientific management need to be resigned to the dustbin of history.
Everyone needs to recognise these (whether they like them or not):
- Almost all of the valuable work people do now (in developed economies) is creative, collaborative knowledge-based work
- Building software is not a repeatable manufacturing process, it is a bespoke creative design process
- We live in a globalised economy with almost no barriers to the movement of ideas, people and money
- Technological change is disrupting not just companies but entire industries, sometimes in just a handful of years
- Time to market and customer value trump all (ALL) other metrics or priorities.
If you don’t get with the times, you will be disrupted and obliterated. It’s that simple.
We need to change the way organisations work
And the values and principles of the Agile manifesto are a great starting point for doing that. Why? Because they focus on time to market and delighting customers. Not stakeholders: customers. The job of a traditional product lead or project manager is to “make the stakeholders happy”. The problem is this.
Stakeholders don’t make or break the company, customers do. And stakeholders rarely know what customers want. Product owners often don’t either, which is why you need to release software as quick as you can and watch what they do and find out what they want.
We need empowered self-organised teams
Contrary to popular belief, the US military is not a shining example of a big hierarchical organisation with strict chains of command. They’re smart and they know that doesn’t work in the chaos of war. They follow the principles of Auftragstaktik or “mission command“. Form many small units of smart empowered cross-functional teams, give them some high-level objectives. Then give them the freedom to adapt to their situation and find the best way to get the job done.
Putting layers of governance and bureaucracy around people means that decisions won’t get made and work won’t get done in time. Which spells death. Usually a slow one, but these days, more of a fast one.
There are simple ways to enable a fast and focused team without layers of management: the XSCALE patterns around Product Squads and Leadership As A Service.
We need to treat people with respect
No more treating technology people as strange nerds to be mistrusted and micro-managed. We need to hire really good people and reward them appropriately. Not by dangling fat bonuses in front of them, tied to stupid KPIs, but by giving them freedom, autonomy, and buy-in to the project and the value they are delivering. Trust people, give them all the tools they need to get their job done, pay them what they’re worth, and get the hell out of their way. That’s the “secret” to management. I just saved you three years and $50,000 so you don’t have to get an MBA.
We need to let developers do their jobs properly, build things properly, and pay down technical debt properly. Otherwise, it smothers us and our products go to hell. It’s called “technical debt” for a reason: it grows by itself if you don’t do something about it. And if you don’t, it gets out of control and becomes technical bankruptcy. I’ve seen it happen and it’s not pretty.
We need to use iterative development
Agile means iterative development, not incremental. Building things breadth-first, to detect and capture customer value in as broad a design space as possible, then simple set-based concurrent engineering to hone down on the delighters that will bowl customers over and maximise throughput. We need to stop being mindless feature factories, pulling random tickets off boards that some stakeholder thinks might look good on a CV or pick up some press on Hacker News. We need to stop doing depth-first incremental development. This type of work is a form of a hill-climbing algorithm that maximises output, locally, instead of optimising globally for outcomes and throughput.
We need to change the way we fund work
There is no point getting people to do “scrums” and “sprints” if you’re giving them a twelve-month project from a pre-canned business plan with fixed features and benefits. That’s not putting a square peg in a round hole, that’s setting the peg on fire and throwing it in a dumpster. The whole point of Agile is frequent discovery and delivery of value. If some budget committee has decided 18 months ago what “value” means for this product, without talking to any customers, or even building a prototype, the exercise is a colossal waste of time and money.
We need to move to independently funded value streams with small-batch discretionary funding, responsible for their own P&Ls. Does that sound scary? Probably not as scary as throwing away tens or hundreds of millions of dollars every year on pointless projects (which a lot of companies do).
We need to change the way we do accounting
The simple truth is that traditional cost-based accounting methods were fairly outdated and inappropriate in the 1980s, and are now laughably bad. Amortising all opex across all business outputs leads to sub-optimal decisions that never maximise throughput. Focusing on cost-reduction instead of revenue maximisation suffers from the same malaise. Big successful companies are:
- made up primarily of people, not physical assets, so traditional balance sheets are largely useless at indicating the long-term value of a company
- focusing fundamentally on customers, not shareholders (and in doing so, ironically delivering large returns to shareholders)
- focusing on maximising throughput rather than minimising costs
- using systems thinking (or ecosystems thinking / whole board thinking), optimising for the whole rather than for a particular product or business unit.
We need to change the way we think about projects
The traditional project mindset worked fine for civil engineering projects in the 1950s, which is where it came from. I’ll leave it to the civil engineers to figure out if it still works for their domain today. It didn’t work well for software in the 1950s and it is despicably bad for software today. The truth is, most valuable software work today is about building products, not delivering projects, and products:
- are (effectively) infinite in duration
- are built for customers, not stakeholders
- have no long-term plan or roadmap since they are continuously evolving to customers’ needs
- are focused on return on investment, not conformance to plan or budget
- fail or succeed based on customer value created, not conformance to specification.
We need to move from a project to a product mindset and fast. I wrote about that here, and Alan Kelly wrote much more about it there.
We need to change the way we run software
It’s not just building it, it’s running it too. No more throwing big piles of buggy code over the wall to poor overworked “Operations teams”. If you’re lucky, they’re in a bunker in the suburbs somewhere – if you’re unlucky, they’re 7000 miles away in a different timezone. Good luck “collaborating”. We need to change all this up.
We need to switch from the horrid half-baked ITIL crap people are doing now to DevOps. Everybody knows it, not enough people are doing it. You build it, you own it, you run it. Developers won’t hate doing this if you let them do their jobs properly, with automated test pipelines running from Dev through Test, Staging and Production. If you give them responsibility for a sensible domain-based fine-grained set of services, instead of some ugly legacy monolith.
We need to change the way we think about risk
When it comes to risk, most organisations treat product or software teams like children. They assume they don’t understand risk, governance or compliance. They force them to run around to risk forums and governance committees and compliance audits. And they force them to “raise risks”, fill out forms, explain themselves over and over to people who frequently don’t understand the product or service they are building. This is condescending and a waste of time and money. So what’s the alternative?
The truth is, using Agile approaches de-risk work from the beginning, instead of frantically throwing mitigants around at the end. They do this by reducing technical risks through spikes, reducing value risk through early prototypes and releases, reducing quality risk through automated testing, and reducing operational risk through small batch size and continuous delivery.
Most governance processes and problems exist because firms have a poor technical and organisational architecture. This means managers have to spend more time playing traffic controller, making people put their toys away, and stopping people crashing into one another, instead of leading and building teams.
When people work in an environment based on small business domain focused cross-functional teams, operating on solid microservice architectures, with security and compliance testing as part of their automated test pipelines most governance problems go away (or don’t turn up in the first place).
We need to change the way we do marketing
Traditional marketing was like most business functions: big, dumb and slow. Fortunately, marketing people have embraced the new digital age with a furious passion and enthusiasm. More power to them! They even have an Agile Marketing Manifesto! Great stuff. Digital marketing, like Agile software development, is all about small and fast: targeting and understanding individual customers, responding quickly to changing environments, maximising returns through late decisions and reduced waste. Fortunately, marketing departments as a whole understand this really well and are leaving the rest of their companies behind.
Conclusion: everyone needs to get on board
It’s quite simple: Agile won’t work if everybody in a company isn’t behind it. It’s not just changing the way software developers do their work. And it’s not even about that plus changing how the testers and business analysts and project managers do their work. It’s about changing the way the whole organisation works: from big slow batches, full of risks and uninformed decisions, to small nimble autonomous and empowered teams. From big clumsy budget cycles and business cases to iterative discretionary funding and continuous delivery.
Some firms understand this and are generating supernormal returns. Others don’t, won’t or can’t, and I don’t need to tell you what will happen to them in the next five years.
