What debt do you create when AI builds your systems?
AI can now take an old system, one with no documents and nobody left who wrote it, and work out how it was made. If that works, why would anyone still spend time understanding the process? Chaiharan was preparing for a panel on what people still need to learn when AI can do the work, and this tool seemed to break his answer.
So he tested his answer against it. AI can recover the how: which part calls which, which rule fires when. Ask it why, and it gives an answer too. But that answer is still a guess, and somebody has to confirm it. He argues that confirming it is where human agency lives, and that a why about people can only be confirmed by being with the people.
Then he imagines two companies with the same twenty-year-old system. One rebuilds it fast from what AI recovered. The other moves slowly, because its manager will not build what nobody understands. On every progress report, the fast one looks better. The piece follows what happens when something changes.
He gives the gap a name, understanding debt: the distance between what an organization runs and what anyone in it can explain. He sets it beside cognitive debt and epistemic debt from recent research, and draws it as a staircase. Six steps of building a product, from understanding the need to defending it before an auditor. Each step handed to AI borrows something, and each comes with a habit that pays it down as you go.
He is honest that the debt is not only other people's. Near the end he finds it in his own habits. Read it if you sign off on work that AI helped build, or if you are about to.