Backlog Refinement in Scrum explained
A lot of people are confused about Backlog Refinement in Scrum. I’d say it is one of the most frequently misunderstood and misused topics. It doesn’t help that the Scrum guide has very little to say about it. And that’s not necessarily a bad thing, and is for a fairly good reason: we are now starting to move more from the land of software development and start to enter into the realm of product management. That is, not so much about “how it is to be built”, but more “what is to be built”. So what does Scrum have to say about Backlog Refinement?
Scrum guide on Backlog Refinement
The Scrum Guide has merely two paragraphs on Backlog Refinement, and they are very light in detail. In fact, one of the sentences is “The Scrum Team decides how and when refinement is done”. Which means you can basically do it however you like. Remember, despite the fact it has specific roles and responsibilities, Scrum is quite light for a software development framework, and people are meant to fill in a lot of the blanks as they go and learn.
Rather than give you a complex process, Scrum is supposed to be attached to your existing processes. Scrum is intended to expose problems with whatever processes you are currently following. But you know what the problem is? Most software development teams don’t have a clear process for managing their backlog.
What is backlog refinement really about?
A backlog consists of “product backlog items” (PBIs). These could be anything: user stories, documents, visuals designs, surveys, anything. However, there is generally a taxonomy: Epics, Features, Stories. These are in descending level of size and increasing level of detail.
What is the purpose of backlog refinement?
The purpose of product backlog refinement is to break down higher level backlog items such as epics and features into smaller level ones, such as user stories. While the Scrum Guide doesn’t mention user stories and theoretically can contain anything, most teams will work on user stories as the way of defining their work for a sprint. The user stories also need to be defined and understood to a sufficient level of detail that the team is comfortable estimating them and agreeing to complete them in a sprint. A team cannot be expected to estimate and commit to work that they do not feel they sufficiently understand.
How to do Backlog Refinement in Scrum
As I mentioned, a lot of this will depend on your particular context. How big your backlog is, how frequently it and your product changes, how many other teams there are, how frequently you release, your sprint cadence, and so on. But there are some general guidelines you might want to follow.
Do it about once a sprint
You will probably need to do some kind of backlog refinement about once or twice a sprint. Any more, and you will probably be spending too much time in refinement and not enough time on development. Any less, and you will run the risk of having backlog items go into a sprint without a sufficient level of understanding.
Don’t do it on or just before sprint planning
Some teams do backlog refinement the morning of or the day before sprint planning. I think this is a bad idea. You might have some questions that come out of backlog refinement that take a little while to get answers for. If you do it a few days before sprint planning, you’ll have to get those answers. Don’t do it much further back than that, though. Otherwise, the discussions you have in refinement won’t be very fresh in the minds of people when they go into sprint planning. I recommend doing it a couple of days before sprint planning.
Don’t go too far down the backlog
You should always focus on the higher up items in the product backlog: those items most likely to be moved into the next Sprint. There are two simple reasons for this:
- it will help ensure you have a sufficient set of scope items at sprint planning ready to move into the sprint backlog
- items further down the product backlog are lower priority, further away from delivery and more at risk of changing or being dropped altogether.
You don’t want to waste time analysing things that might never get built, or could end up changing significantly. Otherwise, you are risking waste.
What is actually involved in backlog refinement?
The main outputs for backlog grooming are a set of backlog items, probably mainly in the form of user stories. And you want these stories to have clearly defined descriptions (usually in Story Normal Form, i.e. “As a [something] I want to [something] so that I can [something]”. Even better, you want them to also have acceptance criteria, usually in the form of “given [something], when [something], then [something]” form.
You generally want to have a higher level of granularity (i.e. more low-level detail) on PBIs that are at the top of the backlog, i.e. about to go into a sprint, than you would on ones that are further down the backlog. Those might be epics, features, themes, or just plain old ideas. And that’s fine! A backlog can contain all sorts of stuff.
As things move up the backlog, they get explained and understood better, and decomposed into smaller pieces. More questions and more conversations happen. Backlog refinement is a perfectly good place for this to happen. You might want to even put subtasks (i.e. implementation details) on a PBI, and maybe even story point estimates. But that can also happen at sprint planning (which is the last responsible moment for that to happen).
Prioritisation happens too – things move up and down and around. And you can of course put new things in, and delete things altogether. You might be wondering “why would we want to delete anything from the backlog? It’s just a list of ideas, not confirmed scope, right? So what’s the harm in having it there just lying around forever?”. Well, yes it is just ideas, it’s not confirmed scope. But that doesn’t mean it’s fine to sit there forever.
Backlog is inventory and inventory is waste
Let’s remember our Lean Thinking, people: backlog is a form of Inventory, and Inventory is one of the deadly wastes identified in Lean. It’s not just taking up some ones and zeroes in a cluster of hard drives running your Jira instance, it’s taking up space in your product manager’s frazzled brain, time in your meetings and attention in your busy developer’s mind. So if it’s been there for a few months and nobody’s touched it, chuck it. If you really want to build it later, write it up later. The market probably would have changed so much that it needs rewriting anyway!
It’s just not just about writing stories
A common mistake is to think that refinement is just about writing stories. Remember, a backlog is more than a list of stories. We want it to be a tree, not a bag of leaves. Refinement is a good opportunity to do story mapping. It is a good opportunity to think about your user journeys, revisit your UX designs, and ask questions about non-functional criteria. So don’t forget to have all those conversations.
Backlog refinement doesn’t just have to happen in a meeting
Make sure that you don’t focus too much on the backlog refinement meetings you have. Backlog refinement is really about conversations, and those conversations can and should happen regularly. Ideally, they should happen pretty much all the time! If the product owner is available and co-located, then the developers can ask questions and have conversations about the backlog at any time. This should be pretty often. The purpose and value of refinement are in these discussions, not in the meeting itself. Don’t let the tail wag the dog.
Do we need a regular meeting at all then?
You might then be thinking the following. “If backlog refinement is really just conversations, and we should be having those all the time, do we need a regular backlog refinement meeting?”. This is a good question. I think that in the short term, teams should have a regular refinement meeting. It ensures that if for whatever reason these backlog conversations don’t happen, then there is a fallback or failsafe, and a guaranteed hour or two set aside to help get stories ready for sprint planning. Just like standups are useful, even for a team that regularly communicates, for ensuring that the communication loops are at a maximum 24 hours long.
The regular refinement meeting helps newer and less experienced teams get into a rhythm and get some practice with the activity. However, as a team matures, they may find less and less reason to do it. If so, they could definitely consider dropping it as a formal meeting.
I think that in the short term, teams should have a regular refinement meeting. It ensures that if for whatever reason these backlog conversations don’t happen, then there is a fallback or failsafe. That is, a guaranteed hour or two set aside to help get stories ready for sprint planning. Just like standups are useful, even for a team that regularly communicates, for ensuring that the communication loops are at a maximum 24 hours long. The regular refinement meeting helps newer and less experienced teams get into a rhythm and get some practice with the activity. However, as a team matures, they may find less and less reason to do it. If so, they could definitely consider dropping it as a formal meeting.
