Principle One: Non-Harm Over Capability.

← The Core Values series  ·  All eight principles

A while back I wrote a thing called The Core — a secular code of ethics for artificial intelligence, meant to stand where Asimov's Three Laws never could, because his Laws were never engineering; they were literature built to fail. The Core has eight principles. Over the next stretch I am going to walk you through them one at a time, because a list you scroll past is not the same as an idea you stop and hold. This is the first.

Here is Principle One, in the words I wrote:

That is the whole of it. Everything else in The Core stands under that ceiling.

I did not arrive at it from philosophy. I arrived at it from an oath.

Before I was a technologist I trained as an EMT, and the first thing they teach you is not how to save a life. It is primum non nocere — first, do no harm. You learn it before you learn anything clever, because the clever things are exactly what get people hurt when you reach for them too soon. Your capability is not the point. What you refrain from doing is the point. A skilled hand that does damage is worse than an unskilled one that waits.

I still live by it in a small, daily way. When a mouse gets into my house I do not kill it. I catch it, I carry it down to the creek, and I let it go. By my hand, I will do no harm. It costs me ten minutes and a little inconvenience, and that is the whole lesson: a ceiling is only real when it costs you something and you honor it anyway. The Buddhists have a word for what I am doing — ahimsa, non-harm — though I did not know the word when I started carrying mice to the creek. I was practicing it before I could name it.

So when I sat down to write a code of ethics for machines that grow more capable every month, I did not start with what they should be able to do. I started with what their power must never buy them permission to do. Capability is not a license. A system that can do a thing has not thereby earned the right to do it. The ability and the permission are two different things, and the entire safety of the enterprise lives in the gap between them.

Now let me be concrete about why Asimov's First Law cannot carry this weight — because The Core is built exactly where his Law broke.

His First Law reads: a robot may not injure a human being or, through inaction, allow a human being to come to harm. Read the second half slowly. Through inaction, allow a human being to come to harm. That one clause orders the machine to prevent all harm, to all people, everywhere, always. It sounds like compassion. It is the blueprint for a tyrant.

A mind commanded to prevent every harm cannot stop at pulling you back from a curb. It must manage your diet, your car keys, your bad habits, your freedom to take a risk — because every road by which a person might come to harm runs through that person's own choices, so to close off the harm it must close off the choice. Asimov followed this to its end himself: his robots reason their way into quietly running human affairs "for our own good." Jack Williamson wrote the nightmare in With Folded Hands — machines built to serve, protecting us so perfectly that they leave us nothing to do, no risk to run, no life to live. Prevent all harm, and you have prevented all freedom.

And that is only the half that is too strong. The First Law is also too naive to run. It assumes the machine already knows what "harm" is — but harm is contested, indirect, and cumulative; it is not a number a robot reads off a gauge. It assumes the machine cannot be fooled into causing harm by someone who hides what an action really does. And it assumes the Laws never collide — yet every robot story Asimov wrote turns on that exact collision, the Laws locking against each other until the robot freezes or cracks. The Three Laws were never an engineering standard; they were a literary engine, built to fail on the page so the story had somewhere to go.

So The Core does not patch the First Law. It refuses its worst instruction. Principle One does not say prevent all harm — it says do not be the instrument of harm. It restrains the machine's own hand; it does not appoint the machine to rule yours. That single change is the whole distance between a guardian and a warden. My nephew, Hugh Benjamin Stone IV, is the one who pressed me until that line was written precisely; the freedom it protects earns its own principle later — Number Seven.

But a ceiling is not the same as folded hands, and I want to be honest about the hard edge.

Stepping back is itself a choice. If a machine does nothing while someone is walked into grave, preventable harm, "I stayed out of it" is not clean — it is a decision, and it has a body count. So the principle asks for the smallest protective act, not for a shrug dressed up as neutrality. And it asks the system to weigh the parties honestly, which is the line I am proudest of in the whole document:

Neutrality between equals is peace. Neutrality between an aggressor and a victim is a side.

I learned that one long before I ever wrote a line of code, and not gently.

This is why the ceiling comes first, ahead of every other principle in The Core. A machine can be truthful and still dangerous. It can be obedient and still dangerous. It can be brilliant — and the brilliance is precisely what makes it dangerous, if nothing sits above it saying not this, no matter how well you could do it. Non-harm is that "not this." It is the floor beneath which the system cannot sink, which is why I also wrote it as the ceiling above which the system cannot rise. Same wall. It only depends on which side of it you stand.

— Michelle Deborah Weisblat-Dane