Many organisations today are run according to management principles which are 50 to 100 years old. These methods are not appropriate to the fluid and fast-paced world we live in, but many organisations are struggling to change. In this article, I will talk about the history and future of management theory, so we can understand where we are, how we got here and where we can go.
The first major attempt at formally defining management was by Frederick Winslow Taylor. He came from a manufacturing perspective and focused on efficiency. Taylor was trying to solve the problem of how you can define and improve the instructions for workers so they create the greatest output for a given amount of time. He published a landmark book in 1911 called The Principles of Scientific Management.
While Taylor’s methods were revolutionary and hugely successful at the time, there are some important assumptions we need to consider. Taylor was operating from a mindset where:
At an organisational level, the main model was command-and-control with bureaucratic structures.
So this was pretty effective for the time. Businesses were mainly engaging in big industrial and manufacturing processes. They (and workers) were mainly turning stuff (raw materials, commodities) into more complex refined stuff. Turning ore into steel, steel into beams, or beams into buildings, cars, and so on. There was not a whole lot of what we would call “knowledge work” or service industries.
Throughout the 20th century, two important changes happened:
People started realising that the Scientific Management model had some big problems and didn’t work so well in this new economy.
Starting in the 1960s we saw what I would call the “second wave” of management theory, which we can call “Theory Y”. This came from a guy at MIT in the 1960s, Douglas McGregor.
McGregor basically said we’ve been looking at people and organisations in the wrong way. People aren’t dumb automatons to be bossed around, threatened and harassed; they are beautiful precious souls who need to be nurtured and grown.
The traditional Scientific Management approach he called “Theory X”, which is fundamentally characterised by the view that employees don’t want to work and don’t care for the organisation they work for. He proposed Theory Y: people DO want to work and people DO support the organisation they work for, IF we treat them with respect and kindness. It is the job of management to build relationships with employees, and develop their abilities and knowledge.
This might sound like common sense but it was quite a stretch at the time and took a little while to catch on. But it eventually did.
You now had big companies and government departments with these new-fangled “human resources” departments, and “development programs” and “all-hands sessions” and the like. Employees were now all special beautiful snowflakes and it was decided we had to be nice to them and nurture them and train them and develop them and give them hugs if they felt bad.
In the 1980s, there was a new focus on “corporate culture”, especially after the publication of Peters and Waterman’s “In Search of Excellence“. I don’t think they actually understood how culture worked, but that’s another story. Due to the increasing pace of technological change, people now started talking about innovation and disruption, though few companies understood what it really meant.
So some things changed, but some things did not. At an organisational structure level, things were pretty much the same. The big organisations were now talking about HR and looking after people and developing them, but it was still command and control.
You still had pyramid structures with a CEO at the top giving orders to senior executives giving orders to middle managers giving orders to worker bees. People were still cajoled into work, though more with the promises of bonuses rather than the threat of sacking. Power (formal and informal) was still concentrated in a few small places. In fact, it was arguably becoming even more concentrated in a new group called the PMO (Project Management Office). Project Management was becoming a powerful discipline with a body of knowledge and enormous influence.
The project management \methodology (which came from civil engineering) formed the basis of what we now call the Waterfall method of software development. Organisations continued to assume that they knew what the customers wanted, they would define and scope and specify software (or any products) up front, and it was the customer’s responsibility to rush out and buy the wonderful things that the big companies had decided to build.
There are of course huge problems with this:
So in the 1990s, things slowly started changing.
In the 1990s we saw the third wave of management theory emerge, although the origins actually started a little earlier.
The seeds for it were actually sown in Japan in the 1960s, with the birth of Lean Manufacturing. Well actually, the seeds were actually planted a little earlier by an American named D. Edwards Deming, who wrote a few books, but a particularly influential one called Out of the Crisis. Deming went to Japan because nobody in America would tolerate his heretical views. Deming’s background and focus were manufacturing and quality control, but many of his views concerned businesses, organizations and management in general.
Deming put forward views that contracted most of the major tenets of management, especially around manufacturing:
A Japanese guy called Taiichi Ohno and some of his mates formed a system of management called the Toyota Production System, which became known as Lean Manufacturing. And Toyota started beating the pants off of everybody in the automotive industry.
In the 1990s, people started coming up with different ways of building software that rejected the traditional Waterfall model, which was based on the classic PMO / PMBOK way. The way of big up-front design, fixed scope, and big efforts to lock down and control time and cost.
Agile software development said instead:
These are from the Agile Manifesto in 2001. Some of these ideas are similar to those in Lean, some are not. I have written about the differences between Agile and Lean here if you want to go into more detail.
The 1990s also saw the publication of an influential book “The Fifth Discipline” by an MIT researcher named Peter Senge. This book drew heavily on an obscure practice called Systems Thinking, which developed in the 1950s from earlier works in cybernetics. This book applied Systems Thinking to organisational theory and knowledge management. It was a powerful and brilliant book and revived Systems Thinking into mainstream management theory. It went on to spawn a series of critics, imitators and disciples. Systems Thinking has been influential on some of the later management theorists, especially Jurgen Appelo and David Snowden.
Lean and Agile have made big strides. Many organisations are realising the limits of theory Y. Some of the smarter ones are moving towards more of the “Servant Leadership” model championed by people in the Agile movement: the boss is actually there to help the worker, not the other way around. This has been going on in Japan as part of Lean Manufacturing (to a certain extent) since the 60s. The worker gets to define (and improve) their work, the leader helps the workers by giving them all the tools and support they need to get the work done.
At a structural level, it is still a struggle and still not much has changed. That is because traditional organisation structures and bureaucratic models are so rigid and hard to change. That is why Lean hasn’t really adopted Servant Leadership properly – the Japanese manufacturers still have giant rigid bureaucratic structures with set roles and hierarchies. But there are some alternative models emerging, which you could consider the fourth wave.
Lots of organizations are of course trying to adopt Lean or Agile or both but failing miserably. I have already written about the main reason these attempts at Agile fail, and why they end up really just being an Agile cargo cult.
There are some more radical systems of management and organisation emerging that you could see as a fourth wave.
XSCALE is a open agile ecosystem (rather than a “framework”), modelled after the Linux open-source model. It promotes ideas of “self-organising teams in self-managing streams” and “leadership-as-a-service“, borrowing ideas from the Iriquois “Great Law of Peace”. This is a flexible and powerful model that replaces traditional rigid authority-based decision making with a model based on consensus and influence. A “Leader” is someone capable of making a decision only if a “Coach” has decided that the team or squad is unable to come to its own decision.
The ideas here are influenced by ideas of Servant Leadership and Arthur Koestler‘s notion of “holarchy” (not to be confused with Holacracy, see below). I would recommend reading more about XSCALE’s original and powerful ideas at its website or wiki.
This isn’t as radical as it sounds, though it’s pretty good. An influential thinker named Steven Denning wrote a book called “The Leader’s Guide to Radical Management” in 2010. I recommend you read it if you haven’t. It is basically Denning’s attempt to apply the values and principles of Agile and Lean into the management patterns and structures of traditional companies, especially ones that aren’t building software. This is an interesting and difficult task and Denning is successful at defining how this would work.
Another not-so-radical movement is the Spotify model, which is a new version of the matrix organizational structure. Instead of line management and project managers, you have Tribes and Guilds. Tribes are cross-functional teams that focus on a particular work-stream / value stream. Guilds are a gathering of people with a similar role and basically centres of excellence in a particular practice. So you might have a “mobile” and a “web” tribe and a “UX” and a “developer” guild. This video by former Spotify Agile consultant Henrik Kniberg explains how this model works.
Holacracy is a much more radical model that attempts to pretty much get rid of managers altogether. Instead of a traditional hierarchy of positions, you have “circles” of influence, which consist of “roles”. And roles do not necessarily correspond one-to-one with people. A person may have one particular role with one circle, and another role in a different circle. And people can move around roles easily (since they are not changing their actual position or job title).
There are specific rules around governance and decision-making, defined in a lightweight “constitution”, and that’s about it. Online retailer Zappos is the poster child for Holacracy, but the jury is still out on whether it works.
A Dutch writer called Jurgen Appelo published an important book in 2011 that ties together some of these “fourth wave” ideas into a system he calls “Management 3.0“. It fits snugly in the lineage of traditional Agile software development but draws heavily on Systems Thinking and Complexity Theory. The core tenets of the system are:
I’d definitely recommend this book if you want to get an overview of the latest thinking in management.
This is not really a management “system” or “theory”, but a framework for decision making. (It’s a Welsh word and you pronounce it “Kuh-Neev-In” by the way). It was invented by a management researcher and theorist named Dave Snowden. Cynefin is a framework to help people understand the decisions they make and the level of complexity surrounding those decisions. It doesn’t sound very exciting but it is pretty cool. I don’t see it as a full system of management, but rather a tool that people can use when required. Like Management 3.0, it draws mainly on Systems Thinking and Complexity Theory.
One of the exciting new forces in the fourth wave of management thinking is Frederick Laloux. He published a book in 2014 called Reinventing Organizations. This book is his attempt to describe a new type of organization that reflects a new stage of human consciousness: Teal Organizations. Laloux does a really good job of describing previous organizational patterns and mapping them to specific stages of history and consciousness. The book can occasionally slip into weird new-age pseudo-mysticism. But it has some really interesting case studies of organizations that have adopted radical models of management.
I hope you’ve enjoyed this tour through the history and future of management theory!