Understanding the risk profile of your technical debt

Let's get something cleared up right now: "Technical debt" is an imperfect analogy, but it is still useful. In this article, we will explore how to understand the amount of debt you’re taking on and calculate the interest rate you’ll be facing when it’s time to make a change.

"Debt" is a term with a lot of emotional and cultural baggage. As such, many folks instinctively associate tech debt as "bad" and “undesirable.” Some folks find tech debt demoralizing. Some folks think it shouldn't be debt if it was the right thing to do at the time. These positions make sense when you think about accruing tech debt as akin to wearing a badge of shame.

Which brings me to this unpopular opinion: All code is technical debt.

All code is technical debt

Tech debt is often defined as the “cruft” in software that’s created by taking an easy, but suboptimal decision in the software development process. The challenge with this definition is that it opens up a lot of debate on how to quantify that tech debt. Redmonk analyst Rachel Stephens points out that one problem with the technical debt analogy is the lack of "knowns." Actual debt has a known amount, repayment timeline, and interest rate. What if we could quantify some of those attributes of our technical debt?

Given the negative cultural associations with debt, it’s not surprising that folks try to minimize what qualifies as tech debt. Amid confusion and debate of what is or isn’t tech debt, we can’t do anything useful with the analogy. We’re too emotionally wrapped up with the term and our egos.

However, debt itself isn't bad. It can be risky, especially if misunderstood, but debt itself is not inherently good or bad. It's a tool. Financial debt is a financial tool, and technical debt is a technical tool. Debt powers our economy, and technical debt powers your business. If we look at it that way, it's a lot less threatening to label all your code as "tech debt."

But why would you do that? It's the same reason why Honeycomb co-founder and CTO Charity Majors talks about how the best engineers work to minimize the code they write. It's the same reason why Discover Financial retired almost 40,000 lines of code with a Spring rewrite: "Smaller code is easier to maintain, easier to evolve."

“The best senior engineers I’ve worked with are the ones who worked the hardest not to have to write new code” @mipsytipsy #monkigras

— Kristian Glass (@doismellburning) February 1, 2018

By calling all code technical debt, we can quantify the amount of debt on the books. Doing this requires changing your mindset from “debt is bad” to “debt is a tool.” Embrace the reality that if you have code, you have technical debt. It has helped you do something, if at a cost. Whether the cost is high or low, or worth it, requires understanding the interest rate on that debt.

Interest rates and risk

Not all debt is created equal. Indeed, renowned author and software consultant, Martin Fowler suggests this and uses the Technical Debt Quadrant to describe different debt quality. Ignoring this and assuming all debt is the same is a critical flaw in how people use the technical debt analogy. It's the reason DreamScale CEO Janelle Arty Starr finds that using the term "technical debt" distorts the conversation within the business. She observes that the business isn't as bothered by debt as it is about risk.

Interest rates are how risk is expressed for financial debt. Like any bond investor, the business cares about the interest rate of the debt (and the rating of the issuer). But, to Rachel Stephens' point, this information is rarely provided. Indeed, it would seem difficult, if not impossible, to quantify the interest rate. But can we approximate?

This is where we can take some learnings from the derivatives world. Remember those mortgage-backed securities that pooled loans together? They get rated based on their quality, including the likeliness of default on the loan. If we apply that concept back to our technical debt, we can have a different conversation about risks.

What makes a loan at higher risk for default? Many things, but two factors are a loan principal that's too high, or an interest rate that's too high. If we assume the codebase itself represents the principal, then we can judge one risk factor based on the size of the codebase. One million lines of code? That's a big loan.

“Having a discussion about technical debt is a luxury. Being able to talk about technical debt implies success.” – Kelly Sutton, “Quantifying Technical Debt”

Paying down this form of debt requires retiring the code itself. You might love the idea of retiring some of that code, but the idea of actually doing it is scary because it’s so hard to change. The code is tightly coupled to other code and it’s difficult to predict how the application will behave if you make a change, let alone retire it. This is like deciding to pay down the principal on your loan, but most of your payments have just been going to cover the interest. Years in, the principal remains intimidating.

So, what about the interest rate? Can we infer what makes a higher interest rate (and, so, higher risk)? Code change becomes the proxy for interest rates. Seven months and 72 steps to change your code? That’s a high interest rate.

And the two factors are often connected. That million-lines-of-code application is probably also a monolithic piece of software that's hard to change for many reasons. One reason why microservices and serverless are attractive is that they favor smaller units of code that are easier to change.

If we think of the time and effort to make changes to the codebase as the interest rate, we align to the notion of risk. If it takes a lot of time, a lot of effort, manual steps, etc., then it's also riskier and more error-prone. Remember: interest rates are an expression of risk.

In contrast, the less friction that exists between development and production, the lower the interest rate on that code debt. After all, it can be changed or retired more easily. Here are several patterns and practices that are used to reduce deployment friction and, therefore, technical debt interest rates:

Good test coverage. Test-driven development helps with this and I recommend reading Pivotal Labs Head of Engineering Matthew Kane Parker's post on Why TDD.
Continuous integration and continuous delivery (CI/CD) pipelines. Automating test execution keeps feedback about bugs and defects flowing quickly to developers. Automating the steps to deployment reduce the friction to change code. Read the CIO's Guide to CI/CD that I wrote with Brian Roche of Cognizant.
Developer-oriented security vulnerability detection. Security reviews late in deployment cycles are slower and more expensive. Shifting the security detection process earlier reduces this friction. Tools like Snyk and Contrast Security help with this.
Microservices architectures. Properly isolated units of code that do one thing are easier to change than complex monoliths. But simple monoliths may still be the easiest thing to change. I recommend Developer Advocate Nate Schutta's series on Should That Be a Microservice?
Path to production automation. Beyond CI/CD, there are other steps in the process to deploy code to production. Compliance checks and related processes are a big one. This process requires getting different stakeholders to the table to agree on a set of automatable practices. I recommend this podcast with Pivotal’s James Urquhart for an extended discussion.

Each of these practices reduce the time it takes to get code into production. They can also reduce the unpredictability and volatility around release cycles. In fact, one tech leader had the path to production instrumented, complete with a dashboard. He knew exactly how long it took between when code was ready-for-deployment to in-production. Any delays over a certain number of minutes set off alerts that something was wrong. If we think of this in terms of an interest rate, it's like getting an alert that your credit card rate just went up.

By treating all code as technical debt, we can have a useful conversation around interest rates and risk. Instead of developers getting frustrated with tech debt that business and product teams don’t seem phased by, developers can speak the language of business. They can talk about the riskier, “high-interest” rate debt in some areas compared to less risky, “low-interest” rate debt in other areas. It’s like categorizing your tech debt as “prime” versus “subprime.”

That can focus conversations around identifying where it’s critical to pay down debt (i.e., retire the code) or lower the interest rate (i.e., lower the friction to change). It’s a lot like a financial planner guiding you to pay off your high-interest credit card debt first and keep paying your low-interest mortgage as-is.

Next step: Getting to prime tech debt

If it’s hard to change your code, you are exposed to risk in many ways. It’s harder to patch, so it’s riskier from a security perspective. It’s harder to add a new “must-have” feature, so it’s risky from a competitive perspective. And it’s harder to experiment, so it’s risky from an innovation perspective. These are the “high-interest” rates on your tech debt. And the more code you have to manage, the more places you can have toxic debt.

Improving the overall risk profile of your technical debt starts with measuring how much code you have and how much friction exists to change it. Once you have a handle on the amount of “principal” and various “interest rates” you have, consider the different methods listed above to bring down the interest rates on various code bases. The code that needs to change most frequently is good to have low rates on.

Prioritize the code that’s critical to your business and classified as “subprime” today, and see if you can upgrade that code to "prime."