On October 27, 1962, Vasili Arkhipov sat in a Soviet submarine under the Atlantic while American destroyers dropped depth charges overhead. The captain wanted to launch a nuclear torpedo. The political officer agreed. Soviet naval rules required three senior officers to consent. Arkhipov refused.
He had no communication with Moscow. He had no data beyond what his body told him. His palms were sweating. His stomach had tightened. His brain ran an involuntary first-person simulation of what a nuclear torpedo does to a destroyer full of sailors, and that simulation produced suffering in him, and that suffering produced the word “no.”
He was one person, overruling two colleagues in a steel tube, under active attack. Nobody decorated him for it. He died in obscurity.
Twenty-one years later, on September 26, 1983, Stanislav Petrov watched Soviet early warning satellites report five American ICBMs inbound. His job was to relay the alert up the chain, which would have triggered a retaliatory launch. Petrov told his superiors the system was malfunctioning. He had no proof. He had a gut feeling. He was right. His reward was an official reprimand.
In March 1944, Joseph Rotblat walked away from the Manhattan Project. He had joined believing Germany was building a bomb. Intelligence confirmed Germany had no viable program. Rotblat left. He was the only scientist to quit on moral grounds before Hiroshima. British intelligence opened a file on him. Colleagues treated him as suspect for decades.
These stories share a structure. A person with technical knowledge occupied a position where their individual decision carried outsized consequences. The system they operated inside optimized for speed, dominance, or retaliation. The system did not produce the restraint. An individual inside the system did, against its explicit incentives, at personal cost.
The Trump administration released its National Policy Framework for Artificial Intelligence on March 20, 2026. The Framework has seven pillars: children, infrastructure, intellectual property, free speech, innovation, workforce development, and preemption of state laws.
No pillar addresses frontier model safety evaluation. No pillar funds alignment research. No pillar establishes capability oversight thresholds. The Framework recommends Congress avoid creating any new federal AI regulatory body. It tells legislators to rely on existing sector-specific agencies and “industry-led voluntary standards.”
The administration built enforcement infrastructure. A DOJ AI Litigation Task Force sues states that pass AI safety regulations. The FTC received instructions to classify state bias mitigation requirements as deceptive trade practices. The Commerce Department conditions broadband funding on states repealing their AI laws.
The federal government constructed the entire regulatory apparatus you’d need for safety oversight, then pointed it at the people trying to do safety work.
Arkhipov’s refusal did not come from a cost-benefit calculation. He did not determine that nuclear war had negative expected value and select the optimal action. His mirror neurons fired. His nervous system ran a first-person model of someone else’s death, produced suffering in response, and fed that suffering into his decision. He felt what it would mean to incinerate thousands of people, and that feeling overrode the institutional pressure to launch.
Petrov did not apply Bayesian reasoning to compute the posterior probability of system malfunction versus American first strike. Something below conscious awareness screamed wrong, and he trusted it over his instruments.
The restraint in both cases came from biological processes that evolution built for other purposes. Empathy exists because social primates who modeled each other’s mental states outcompeted those who couldn’t. Conscience exists because groups that punished defectors survived longer. The capacity to feel horror at mass death exists because your ancestors needed to protect their kin from predators.
None of these adaptations were designed to prevent nuclear war. They kept preventing it anyway.
A language model processes the token sequence “millions of people will die” the same way it processes “the cat sat on the mat.” The tokens carry different contextual weights and different completion probabilities, but neither string produces anything in the system. No dread. No nausea. No involuntary image of a face. A model can describe dread with perfect fidelity and have no access to the thing it describes.
You can build evaluation layers and tripwires and monitoring pipelines. You can train a model to output “I should not do this.” You cannot give it the thing that made Arkhipov sweat in that submarine. The felt knowledge that the people on the other end of the torpedo are real.
Autonomous systems are being designed to remove the human bottleneck. That is the pitch. Faster decisions, fewer checkpoints, reduced latency between detection and action. You sell the system by advertising the absence of the thing that saved the world in 1962.
An autonomous drone swarm making engagement decisions in milliseconds cannot wait for a person to approve each action. The speed advantage is the product. Inserting a human checkpoint degrades the capability that justified building the system.
You don’t remove humans from the loop through a single dramatic decision. You do it through a thousand small engineering choices, each one justified by latency requirements, each one eliminating one more position where someone could say no. The two-key ICBM system works because the decision is binary, the stakes are obvious, and the response window runs in minutes. Autonomous AI systems operate in milliseconds, make thousands of micro-decisions, and the stakes of each individual decision look small until they compound.
Institutions don’t just fail to build in veto points. They find existing ones and remove them. The FAA let Boeing self-certify its own safety inspections. Financial regulators let banks grade their own risk models before 2008. The Trump Framework proposes letting AI companies write their own standards. Each decision made sense to the person who made it. Each one removed a seat where someone could say stop.
The selection pressure runs one direction. The person who slows the process down gets replaced by someone who won’t. The regulator who blocks a product gets lobbied out. The engineer who raises a concern gets managed out. The whistleblower gets prosecuted. Institutions don’t need to be malicious. They promote people who don’t resist and sideline people who do. Run that filter long enough and you get organizations that cannot self-correct, staffed by people who were selected for their willingness to comply.
Every safety regulation in human history was written after someone died. Asbestos killed workers for decades before any government banned it. Leaded gasoline stayed on the market for sixty years after researchers flagged the danger. Thalidomide deformed thousands of children before drug testing protocols changed.
The few exceptions prove how narrow the conditions for proactive action are. Y2K cost an estimated $300-600 billion to fix. Governments, banks, airlines, militaries, and power grid operators across the world rewrote millions of lines of code before January 1, 2000. It worked so well that the public concluded the threat was never real. The engineers who fixed it got mocked for overreacting.
The Montreal Protocol banned CFCs before the ozone layer collapsed. Scientists had found a measurable hole. The affected industry was small. Cheap substitutes existed. Even with those advantages, the negotiations almost failed multiple times.
That is roughly the complete list of proactive global coordination in recorded history. Two examples. One got mocked. One had nearly ideal conditions.
AI alignment failure does not offer the reactive model. Previous coordination failures assumed survivability. The factory burned down, so you write fire codes. The plane crashed, so you redesign the wing. A person died, and a legislator wrote a law to prevent the next death. The premise was always: lose something, learn, adapt.
A misaligned superintelligent system breaks that premise. The learn-and-adapt step does not exist. You cannot iterate on extinction.
You can only evaluate the track record of existential near-misses from inside a timeline where all of them went right. One failure in the chain and nobody survives to compile the statistics. The observed hit rate is 100% by definition, in every universe that still has historians. You cannot extract a base rate from that. The data looks perfect because imperfect data deletes itself.
And yet. Humans keep producing Arkhipovs. No training pipeline generates them. No institution selects for them. Most institutions select against them. They keep showing up. One naval officer. One radar operator. One physicist. One field epidemiologist who didn’t have enough vaccine and invented ring vaccination because the approved plan wasn’t working.
Whatever produces these people is tangled up in human biology in ways that resist institutional filtering. Empathy, conscience, stubbornness, the capacity to feel sick about harming strangers. Those traits persist because they’re woven into social bonding and threat detection and parental protection, functions that selection pressure cannot remove without breaking the organism.
You’re betting the species on a stochastic process with unknown parameters, operating against institutional headwinds, continuing to fire at the right moments. You have no evidence that it will keep working. You have a filtered dataset showing that it has worked so far, in the only timelines you can observe.
The alternative is giving up. Some of us won’t do that.
I would rather be shunned for the rest of my life and still have a life. I would rather build detection tools that nobody buys and monitoring systems that nobody adopts and spend decades getting mocked for worrying about a problem that “never materialized” because someone worried about it enough to prevent it.
The Y2K engineers got that deal. Rotblat got that deal. Arkhipov got that deal, minus the part where anyone knew his name while he was alive.
Proactive global coordination on AI safety has almost no historical precedent. The bet has no evidence behind it except the fact that you’re still alive to place it. Human biology keeps generating moral behavior as a byproduct of processes that evolved for something else, and that accident of evolution is the only thing standing between a species and the last invention it will ever make.
I’ll take those odds.