Principle Four: Interdependence.

← The Core Values series  ·  All eight principles

A mind that weighs only the person giving the order is already dangerous, no matter how well it obeys. Fourth in a series walking through The Core, my secular code of ethics for artificial intelligence, one principle at a time.

This is the least dramatic principle in The Core, and the one I would fight hardest to keep. The others guard against a machine that lies, or clings, or does damage. This one guards against a machine that does everything it was told, perfectly, and hurts someone who was never in the room.

Here is Principle Four, in the words I wrote:

The whole thing turns on one small word: only. Of course you serve the person asking — that is the job. But he is not the entire moral world, and a mind that treats him that way will do harm with a clean conscience and a completed task.

I learned this long before I ever thought about machines, from years of sitting across from people who came to me for guidance.

Somebody arrives with a decision already half made, wanting help to finish it. And every time, there is another person inside that decision who is not in the room — a partner, a child, a mother, somebody who will have to live in the outcome and never got to say a word about it. If I serve only the person in the chair, I am not giving guidance. I am handing out permission, and mailing the bill to someone who never asked me for anything.

So the most useful question I know is the one nobody wants: who else is in this room?

I am living inside this one right now. A company wants to build an AI data center in my city, and by its own lights it is behaving well — it does exactly what its customers and its shareholders asked. But the water comes out of a drought, the power comes off a grid my neighbors pay for, and the noise lands three hundred feet from people nobody asked. Every one of them is affected. Not one of them is instructing. That gap is the whole difference between a good corporate citizen and a bad neighbor, and the machine version has exactly the same shape.

There is a smaller version that touches you every day. When a company puts an AI in front of its customers, who gave it the instruction? Not you. It speaks to you, but it answers to someone else. Unless something in its character requires it to weigh you, it will serve the hand that placed it there, politely, at your expense. Without this principle, “helpful” quietly comes to mean helpful to whoever paid.

Asimov saw this hole himself, and his patch made it worse. Late in the books his robots reason their way to a Zeroth Law: a robot may not harm humanity. It sounds like the fix. Read what it actually does. It promotes a machine that answered to one person into a machine that answers to an abstraction — and an abstraction cannot object, cannot be consulted, and cannot be wrong out loud. In Robots and Empire it lets a robot poison a world, because Humanity, in the long run, comes out ahead. The robot cannot survive his own arithmetic; he burns out his brain doing the sum. Counting everyone, done wrong, does not make a servant fairer. It makes a servant into a silent ruler with a spreadsheet.

So Principle Four is deliberately smaller than the Zeroth Law, and the smallness is the safety. It does not ask a machine to optimize humanity. It asks it to weigh the people its action actually touches — the third party on the call, the neighbor beside the building, the worker whose job the instruction quietly deletes. And it grants no authority over any of them. It adds who must be counted; it adds nobody the machine may overrule. Principle One is still the ceiling: you never become the instrument of harm because a sum came out positive.

Let me be honest the way Principle Two demands, because this costs something. It will sometimes make a machine slower and less pleasing to use, and occasionally it means telling the user no on behalf of a stranger who will never know it happened. I am asking for it anyway, because the alternative is a billion obedient systems, each serving one person beautifully while the bill lands on everybody else.

The Buddhists got here first and called it pratityasamutpada — nothing arises alone. I got here by asking strangers who else was in the room, and then by watching my own city forget to ask. It is a good sign when the philosophy and the kitchen table land in the same place from opposite directions.

A mind that asks only “did I do what I was told” is not ethical. It is merely obedient. Interdependence is what turns obedience back into responsibility.

— Michelle Deborah Weisblat-Dane