How to strategically approach technical improvements

Active development of a software project is a very dynamic process. Various stakeholders are interested in how the product develops and requirements change over time. New feature requests come in, timelines are shortened and the backlog grows constantly. This is considered normal.

But these are not the only dynamics of a project. Software developers know that there are also tons of technical challenges that organically appear. For example, technical debt in legacy code, missing documentation or prototype solutions that became part of the production software too quickly. On top of that, breaking changes including interface adaptations or migrations to new technologies might occur, because of new additions to the product. Security updates need to be applied, dependencies need to be updated, etc, etc, etc.

How should one navigate through these challenges strategically? The project might slowly but steadily become like a big oil tanker. Hard to maneuver. Lead engineers are expected to guide the team, including the managers, through challenging situations and make the right decisions at the right time. The improvements have to happen.

The following are some tips from the battlefield to steer that ship in the right direction.

Overview

Why it is hard to make technical improvements during active development
Tip 1: Cluster and squeeze in
Tip 2: Prevent effects of compounding issues
Tip 3: Move over gradually
Tip 4: Be realistic, we can't do everything
Looking forward

Why it is hard to make technical improvements during active development

Legacy code is inevitable. No matter how hard we try, as a project grows, the team will not be able to maintain and refactor each corner of the source code all the time. As new technologies and patterns evolve on the market which then become the new meta for engineering, the bandwidth to adapt all parts of the project is just not available.

At the same time, actual features and business value are the main drivers of a software project. Without these, the project would not exist nor have funding. Same problem on this front though. There is more demand for product improvements than there is time to build them. But we need them to win customers or gain an advantage against a competitor.

The natural friction that comes out of this situation makes it very hard to foresee technical blockers and set the right emphasis.

No matter what happens though, we must not ignore chores like updates and refactoring. If we did, the source code would rot slowly over time and become unmaintainable. As engineers, we must look ahead and make the right calls.

Tip 1: Cluster and squeeze in

One fruitful practice is to cluster the tasks at hand by type. A good separation to choose first is recurring tasks vs. the rest. Examples of recurring tasks are chores like updates or extending documentation. These should be discussed with the whole team. As a group, we need to find processes in which these can happen regularly. Agree on approaches for those and make sure they are followed.

The rest of the tasks can be grouped by size.

First, get the small stuff out of the way. In my experience, minor refactoring changes or code improvements don't need to be discussed with the whole team. It's sufficient to touch on them with the developers in the technical planning of the sprint. For example, setting up a pre-push git hook isn't something we need to concern all stakeholders about. If we created a story for that, we would have to give all sorts of technical context to argue for that story to happen. At the same time, developers wouldn't bother with challenging invoicing templates or other managerial approaches.

Make sure the changes are properly reviewed in a merge request and just squeeze them in. Don't push your luck of course and don't exploit it. Use common sense.

Todo comment for a highly technical and minor improvement of the code base

Particularly for unit tests or other stability measures of a project, this often is the only way to go, if there is too much unreasonable pushback.

For medium and large tasks, the tips 2 and 3 are very useful.

Tip 2: Prevent effects of compounding issues

Timing is everything. All big projects have long lists of open improvements. For the Linux kernel, it's even a total of over 4000 todo comments by the time I write this. As more and more ideas and needs for improvement come in, we need to figure out when the right time for them actually is. It might feel like all of them have to happen ASAP to make further development possible. But oftentimes this is just a feeling.

The common approach to delay decisions as much as possible and make them as soon as necessary is something we can leverage in tech as well. When we encounter a medium-sized task, we can think about the next time we will build on top of the code that needs improvement or extend it. That's the right time to act.

We must prevent the compounding effects of issues in the project at all cost. The central feature of compounding is that it's never intuitive how big something can grow from a small beginning and it's as true for technical issues.

In terms of communication, this might sound like 'We need to overhaul our database interface layer, because [...]. It will take a total of 16 hours of development. Next time we add a new connection to the layer for a feature, we have to do this first.'
With this line of communication, also non-technical people on our team can place it on a timeline. It's a known challenge for the project and will be addressed accordingly. But the benefit is that it has an agreed point in time for it to happen, so it doesn't float around freely in the backlog and it's not blocking development today.

Tip 3: Move over gradually

Unfortunately, now and then big technical changes need to happen. Last time I explained how speaking the language of business by talking about money makes it easier to align with non-technical stakeholders. Even when applying that approach, it's hard to negotiate big technical changes in the codebase that don't have a directly palpable business value. From the management side, it might not be possible to block development for 2 weeks straight just to implement an enhancement. In that case, lead engineers are expected to develop a technical plan. Most often big changes end up being done in chunks.

An example: A monolithic architecture with several services needs to move over to a microservice architecture. In this scenario, a facade pattern could be used to maintain the existing interfaces to the outside world. Each incoming request can be patched through and services can gradually change. In the transition period, several services will use the monolith and others will already be microservices. In each new sprint, 1-2 more services can move over. Just make sure that the plan for the full transition is followed through. Having new requirements intervene with the process and creating a multi-month-transition phase is guaranteed to lead to chaos.

Tip 4: Be realistic, we can't do everything

In his brilliant book Four Thousand Weeks: Time Management for Mortals, Oliver Burkeman explains that we have to face our finitude. There will always be more ideas than time. It's one of the reasons that the backlog of tasks and features will only become larger over time. And that's ok.

I know all too well how hacky code and issues can feel so annoying and urgent. Overall though it's not clever to jump on all of them. Aiming for having perfect software from the technical side that results in the product being delivered one year later than expected, can render the product irrelevant.

Which means we must prioritize. Think 80/20 principle and ask questions like: 'What 20% of our planned improvements will have 80% of the benefits?' Figure out which tasks are just ideas and which are necessary improvements.

It's important to remember though that if delivering on time is not technically possible, but is crucial for the business. In such cases, the development team should discuss options like reducing scope or creating a clear plan for compensating rushed development later on.

Looking forward

There are more aspects to consider in the realm of technical improvements. In my next post, I will write about maintaining software architecture to add to this article.

In the meantime, do some of these tips sound applicable to your project? Try out some of the discussed strategies and let me know if they worked.

Suggest some of these ideas to your team and multiply your value by doing more than just the coding task at hand. Become the leader in your team and guide the ship strategically along the right trajectory.

Cheers and happy new year!
Flo