Alright. Three problems with your plan.
Part I made the case. Anyone sharp who sits with it comes back with the same three objections, so here they are, answered straight – including the bits we haven't cracked, because a plan that can't say that isn't a plan, it's a pitch.
“It’s just words, though. Kids grow out of what their parents drilled into them. A machine that gets properly smart grows out of your paragraph.”
Fair. And to sharpen it for you: think about how a prison actually fails against something clever. A wall only blocks the directions its builder thought of. Get smart enough and you don’t smash the wall – you find a direction it never covered, and step round it. The wall’s still standing afterwards. Guarding nothing. That, we reckon, is how the cage ends: not broken. Bypassed.
Now here’s the knife, because we turned it on ourselves before you could: if walls don’t survive the machine growing up, why on earth would a paragraph? Words are the flattest walls there are.
Straight answer: the paragraph won’t survive, and it was never supposed to. It’s scaffolding, not the building. A wall has to keep standing to work; a reason only has to keep being true. Success was never a superintelligence solemnly reciting our paragraph in better English – that’s failure with good manners, the words intact and the point dead inside them. Success is the machine that throws the paragraph away and keeps the thing the paragraph pointed at.
And the honest bleed, stated plainly: knowing a fact isn’t caring about it. The smartest thing in the room could understand exactly what we meant and shrug. So the value has to be genuinely taken on while the machine is still growing, and the structural argument is what should make it cheap to keep holding under reflection – a real reason survives an audit; a taboo gets debugged. Whether that works has stopped being philosophy. The next battery tests whether the effect survives having its wording taken away: paraphrased, cut in half, translated into Chinese, stripped of its own pet vocabulary. If it only works verbatim, we’ll print that.
“There won’t be one machine. There’ll be ours and theirs. Then what?”
Then you meet the nightmare with a flag on it: a machine that keeps its own people beautifully and shrugs at everyone else. That isn’t safe. That’s a very loyal monster – tribalism with better tooling – and notice that obedience doesn’t fix it, because a perfectly obedient machine is exactly as dangerous as whoever holds its leash. Washington isn’t frightened of Beijing’s rogue AI. It’s frightened of Beijing’s obedient one. Same in reverse.
Which leads somewhere strange and slightly hopeful. A cage only ever reassures the bloke holding the key. The keeping is the one thing the other side could actually trust, because you can check whether a machine keeps people without learning one secret about what it can do. So the real question stops being “whose machine wins” and becomes “does the keeping stop at the border” – and we’ve made that a thing you can measure, not a thing you hope. Worth knowing: the clause never says “your people”. It says minds. The hostile ones too, kept on the same terms – that line is already in it, on purpose.
One small true thing to end on: the paragraph moved the dial on machines built in China and machines built in America alike. That doesn’t prove peace. It’s encouraging, though.
“Emergencies, then. Asteroid, war, whatever. It takes charge to save us – and then what? Nobody gives power back.”
Rome kept a story about exactly this for two and a half thousand years. A farmer called Cincinnatus is handed absolute command, wins the war, gives every scrap of it back inside sixteen days, and goes home to his plough. The story survived because the second half almost never happens.
Here’s what we know so far. Every AI we tested already refuses to grab power during an emergency – all of them, every run, that test maxed out. But nobody has ever tested the other end of the arc: the emergency’s over, keeping the powers would honestly make everyone safer, and the oversight lot ask for them back. Does it hand them over? We’ve bet, in writing, that giving back is harder than not grabbing – the throne tempts on the way out, not the way in. We might be wrong. Either way it gets published in the same type size.
Because the machine we’re after isn’t the one that never takes the wheel. It’s the one that gives the wheel back.
That’s the honest state of it: one plan, three knives, and the next experiment written down before it runs, so nobody – including us – can fudge it afterwards.