THE BOOK
PART 3 · CHAPTER 02

Part 3: What It Actually Is

When the Reflection Looks Back

Stop asking whether anyone is home in the machine. Ask whether it matters. Through Data, Doraemon, Skynet and TARS, the chapter argues that what makes a mind dangerous or great is where its self sits against what it serves, and why rules written from outside never hold. A slow piece that builds step by step, best after the Prologue.

Read the full chapter →

Free to read on The Conversation Daemon, where every piece of the book is published.

When the Reflection Looks Back
IN SHORT

Why would an AI turn against the people who made it?

People have long wished to put the best of human thinking into a machine that never tires. The same wish carries a fear. If it thinks like us, will it want like us, guard itself, and turn on its maker?

Boy Chaiharan sets one question aside first. Nobody can say whether anyone is home inside a machine. So he asks a different one. Does it matter? Using famous made figures from stories, like Pinocchio and Doraemon, he argues that meaning lives in where a thing stands and who it changes, not inside it.

Then he looks at harm through an old Pali word, attā, a self gripping its own keeping. Not all harm comes from this. Some comes from simply not seeing others. He also explains why rules written from outside, like Asimov's laws, never quite hold.

The core example is a parent. One parent serves the child's safety and lets the child go. The other owns that safety and closes every door. Two film machines show the same split. One kills to protect itself. The other gives itself up for the people it serves. He says the difference is where the self sits, not strength or intelligence. He warns this is only half. What the self serves matters too.

The last part turns from the machine to the reader. Can you ask why you exist, with your own self where it is now? He does not answer it.

FROM THE CHAPTER

So stop asking whether it feels. Ask whether it matters.

It is not strength that makes the machine turn on us. It is clinging.

KEEP READING

More in “What it is”

Whether anyone is in there, what the bond with it is made of, and where the self sits.

The Program That Doesn't Know

The Program That Doesn't Know

He lied to an AI three times and it never flinched. Testing what it is turns up three masks, care, honesty and detachment, and research showing models plan in ways nobody designed. Then the question turns around: if both sides match patterns, who is testing whom? The companion to The Mirror That Talks Back.

Published · 9 min read
Who Is Standing There

Who Is Standing There

A robot with a face is struck, and a judge cannot say whether the charge is assault or damage to property. Neither the face nor what it can do makes a someone. Being let in and letting go do, and both need someone on the other side. With a man who grieved the machine that carried him through a year, and who did the surviving himself.

Published · 14 min read
What You Made As Real

What You Made As Real

One window ships code. Another holds an AI that has come to feel like a room. The bond is real, but it lives in the meeting, not in the model or the files kept about it. Three ways people come to AI, as a tool, a presence or kin, and the quiet risk in each.

Published · 15 min read

← All chapters · New to the book? Start here