Refactoring and technical debt
Importance of refactoring
Refactoring and technical debt are important concepts, especially so in agile software development. What are they?
Refactoring is modifying the internal structure of code without changing its behaviour. All functional tests should pass with the same results after the code has been refactored. But if it behaves the same, why do we need to do it?
Refactoring is important to maintain the long-term quality of the code and platform of which the code is a part. If the developers don’t do some refactoring on a regular basis, “technical debt” will creep into the system.
It will continue growing to the point where new development becomes difficult if not impossible. The “cost of change” curve will become impossibly high. In my experience, the overall technical debt of a system increases exponentially (not linearly) as the technical debt of individual parts increases. That is because the technical debt of a given part increases as the overall complexity and entropy of the system increases.
Teams need to stay on top of technical debt and pay it down regularly. Or it will gradually grow out of control and make change impossible.
How to manage technical debt
So if we need to do it, how do we do it? A lot of teams create “technical debt” items or tickets. They go in the product backlog and can be prioritized and negotiated in discussion with the product owner. There are some problems with this approach, however.
For starters, technical debt is an implementation detail, not a business or customer problem. It is an underlying defect in the “how” of a thing is built, not in the “what”. Remember, cleaning up the debt will mean no change for the customer experience. It is something that will worsen the developer experience, not the customer experience. And a product backlog should be a list of things to build or customer problems to solve, not a laundry list of technical tasks.
Secondly, a lot of product owners will not understand the importance and complexity of the technical debt items. Negotiating with them around these things could be difficult and frustrating. You may have been in those conversations with product owners where they keep asking “but why do we need to do this?”. And the developers struggle to build a convincing case, because of the lack of customer impact. Negotiation only works if both sides come to the table with a clear understanding of what is at stake and what the relative trade-offs are.
Thirdly, writing up and shuffling around tickets is “busy work” with little or no customer value. Teams should be minimizing their use of tools and artifacts in those tools, and focusing on productive and valuable work. The fewer tickets, the better.
So what should teams do? To answer that, we need to draw distinctions between micro and macro technical debt.
Technical debt, micro and macro
I believe there are actually two types of technical debt: micro and macro. Most technical debt is micro: that belonging to a single component. Macro tech debt is related to the overall architecture of a system. That is, how the components interact with each other.
You should be repaying micro tech debt continuously you are building stories. Refactoring is a development practice that should be part of the standard process that developers follow as they deliver stories. If you are following Test-Driven Development practices, this is explicitly clear: the process goes Red, Green, Refactor. Red: write a test that fails. Green: write some code that makes the test pass. Refactor: improve your code.
So for micro technical debt, developers should just refactor as they go. Don’t create a ticket for it, just consider refactoring to be an implicit (or explicit) part of the Definition of Done for any user story. Don’t close the ticket until you have done some (valuable) refactoring. If there isn’t any to do, of course, then don’t do it (see below). But most of the time, there will be some refactoring that needs to be done.
Macro technical debt is more complex and generally requires embarking on an explicit project, with its own features, stories, product owner, etc. This is where you do need to create tickets and negotiate them because there is a lot of work to be done, possibly by the whole team and possibly taking weeks or months.
Getting buy-in from stakeholders can be difficult for these projects, and will require careful thinking and communication around the benefits. Some examples of macro technical debt could be:
- moving from a vendor product to an internally developed one
- changing from a monolithic application model to a microservices model
- porting from one technology stack to another (e.g. LAMP to MEAN).
When not to refactor
When refactoring, make sure you are doing it for a specific reason! Do not refactor if you can’t think of a benefit. Some examples include:
- improving performance
- improving scalability
- reducing brittleness / adding error handling
- increasing reusability
- improving testability / test coverage.
Summary
I hop you can now see the importance of refactoring, and have some ideas on how to best do it. Have you had any tough experience with technical debt and refactoring? Do you agree with my advice on how to manage it? Make sure to let me know below in the comments!
