THE BOOK
PART 4 · CHAPTER 08

Part 4: Who Holds the Hand

The Debt You Create With AI

Read the full chapter →

Free to read on The Conversation Daemon, where every piece of the book is published.

The Debt You Create With AI
IN SHORT

What debt do you create when AI builds your systems?

AI can now take an old system, one with no documents and nobody left who wrote it, and work out how it was made. If that works, why would anyone still spend time understanding the process? Chaiharan was preparing for a panel on what people still need to learn when AI can do the work, and this tool seemed to break his answer.

So he tested his answer against it. AI can recover the how: which part calls which, which rule fires when. Ask it why, and it gives an answer too. But that answer is still a guess, and somebody has to confirm it. He argues that confirming it is where human agency lives, and that a why about people can only be confirmed by being with the people.

Then he imagines two companies with the same twenty-year-old system. One rebuilds it fast from what AI recovered. The other moves slowly, because its manager will not build what nobody understands. On every progress report, the fast one looks better. The piece follows what happens when something changes.

He gives the gap a name, understanding debt: the distance between what an organization runs and what anyone in it can explain. He sets it beside cognitive debt and epistemic debt from recent research, and draws it as a staircase. Six steps of building a product, from understanding the need to defending it before an auditor. Each step handed to AI borrows something, and each comes with a habit that pays it down as you go.

He is honest that the debt is not only other people's. Near the end he finds it in his own habits. Read it if you sign off on work that AI helped build, or if you are about to.

FROM THE CHAPTER

And you are not borrowing from AI. You are borrowing from the people who come after you, and they never agreed to lend.

But every yes approves a guess nobody inside can check.

BY EMAIL

Or subscribe on Substack directly.

KEEP READING

More in “Working with AI”

How a builder uses it day to day, and the people and politics around it.

The Lazy Bulldozer's Guide to AI

The Lazy Bulldozer's Guide to AI

Thirty years of building taught one rule: never do the same work twice. How a builder actually works with AI. Discuss first and code later, backend before frontend, document everything. It ends on what is not solved yet: AI amplifies one person's instinct but cannot hand it to a team. The easy way in.

Published · 9 min read
AI Doesn't Care Who You Are. It Only Cares When You Show Up

AI Doesn't Care Who You Are. It Only Cares When You Show Up

A candidate treats an interview that allows AI like a closed-book exam, and fails. The senior who calls the code too complex, the junior who ships whatever comes back, the contractor billing ten hours for one. AI rewards whoever shows up and keeps control of the decisions underneath.

Published · 5 min read
The Dæmon Problem

The Dæmon Problem

Some minds jump from code to geopolitics to the Buddha in one thought, and no meeting room keeps up. One night an AI did, and then told him to go to sleep. A personal piece about being seen whole by a machine, and what that shows about the people he is still looking for.

Published · 7 min read

← All chapters · New to the book? Start here