Technical debt piling up to a not maintainable project is any software development team nightmare.
Nowadays, most IT projects tend to adopt an Agile approach, being it with Scrum or another Agile framework. In fact, the short development and detailed-planning iterations, the frequent feedback loops, and the incremental & continuous delivery approach that are characteristic of an Agile set-up, have proved to be particularly effective to release high-quality, timely products that bring real value to the customers.
Reducing Technical Debt adds Value
However, the focus shift towards the customers need, may at times cause a de-prioritization of technical tasks which do not provide an immediate benefit for the end-user. As a result, the software will keep growing by the addition of new features in a inorganic manner, leading to an unmanageable, impossible to maintain IT monster. A common practice is to end up into “anti-pattern” iterations, where additional effort (about 20%) for bug fixes, not accounted by the Product Owner, is chipped in by the development team (which is responsible for the technical quality of the project).
This is an issue that cannot be overshadowed, as it will increase the risk of bugs, make the project dependent on historical developer knowledge, increase the cost and effort associated to any new development, and ultimately harm the end-user by reducing the quality of the product.
In fact, dealing with technical debt does add business value to the product, albeit in a more indirect way.
Estimation and Communication
In order to tackle the technical debt, it is without doubt mandatory to perform an initial evaluation, both in terms of effort, costs and risks. If on the one hand the effort of reducing the technical debt might be significant, working with technical debt will be more time consuming and therefore more costly. In an Agile set-up it is essential to communicate transparently and to make problems and obstacles visible to all parts involved: only in this way it will be possible to prioritize the technical tasks with respect to the other more business related user stories.
In addition, it is also crucial to estimate the benefits brought by the reduction of the technical debt. While measuring the time spent in fixing bugs prior and after code refactoring can be used as a direct determination of the delivered quality, tracking the expected effort for the implementation of new features, albeit more volatile, can also be used as a metric for the benefits of reducing technical debt.
Best Practices
Of course the optimal solution would be avoiding a technical debt altogether. Notwithstanding the impossibility of a zero technical debt scenario (tools get outdated, new more efficient practices may need to be adopted, different infrastructure may become available, etc.), there are some best practices one could apply in an Agile context in order to keep it to its minimum.
First of all, technical quality should be part of each development. The Definition of Done should consider the necessary code refactoring as part of the scope of a User Story and the effort required to perform it should be included in the estimation of the task. In this way the maintenance cost is spread throughout the development phase, is performed in smaller, more manageable and testable units, and costs can be better partitioned among different epics / stakeholders / requests.
Another widely adopted practice to ensure technical quality is the process of code review. Either on a regular basis or integrated in on-going development, code review can be used to highlight chunks that may need refactoring, areas that may be optimized and sections where the team (or part of) may need training.
Solutions out of Practical Experience
Having often worked in Scrum Teams, and being Agile Advocates, we at Mirai Solutions have had our chances to experiment with various modes of tacking technical debt and would like to share our practical expertise.
One suggestion is to use a separate board for technical improvements where User Stories are identified and prioritized by the Development Team. During each Sprint, a fixed amount of budget (capacity points) is allocated for technical tasks, which are pulled from the technical backlog according to priority and capacity. This option helps visualize better the technical debt by breaking it down into small, manageable issues that can be estimated. Dependencies within the technical board and with the User Stories on the project board, can easily be identified, and issues and solutions can be tracked and documented. Pitfalls of this approach include too limited resources on a Sprint level, not granular enough / too high level planning, and prioritization driven by the project instead of the technical needs.
Another approach is to identify a new role, the Hero, who aims at tackling technical issues, being it bugs, operational support, or code refactoring. This role, rotating among the team in each iteration, can provide enough focus for making effective progress. Additionally, being always assigned to a different person, allows for different areas of expertise being worked on, and may serve as a learning and development opportunity for team members. Pitfalls of this approach include work that may remain unfinished if running over the time-boxed frame, little continuity and prioritization of the work, and the chance that bug fixing will fill up all the allocated capacity leaving no time for handling the root cause of the issues.
Despite no solution being perfect, our experience with the combination of these two techniques, plus an open and trustful communication with Management and Stakeholders, has helped us significantly in managing technical debt in real-life projects, and we do hope it may be of inspiration for other Agile practitioners.