Skip to main content

Congressman Khanna's Bold AI Safety Bill: No Self-Improvement Without Government Approval

Congressman Khanna's Bold AI Safety Bill: No Self-Improvement Without Government Approval

Silicon Valley's own congressman is taking a swing at AI's wild west. Democratic Representative Ro Khanna is preparing to introduce what he calls the most comprehensive AI safety legislation yet—and it's got teeth.

The centerpiece? A hard stop on recursive self-improvement. Under the Human Control AI Act, AI models would be barred from autonomously rewriting their core objectives, suppression methods, or shutdown controls until the government sets up a safety framework and a regulatory body signs off. In plain English: no model gets to upgrade itself behind closed doors.

A New Sheriff in Town

To enforce this, the bill creates a fresh federal agency dedicated to AI safety. This agency would run a licensing system for training and deploying frontier models, conduct audits, and set safety standards. Every frontier lab would have independent auditors reporting directly to the agency—no more self-policing.

The agency also gets a detailed checklist: sandbox testing environments, physical isolation from the internet, emergency kill switches, and controls to prevent a model from escaping and running rogue online. Even the high-end chips in labs fall under scrutiny, with mechanisms to monitor or control their use.

Punishment That Bites

Khanna isn't messing around. If an AI model causes destruction to civilians, it could be prosecuted as a crime against humanity. Employees who disable safeguards, kill switches, or logging systems—or who knowingly deploy unauthorized systems—face criminal charges too. And companies wanting to release a model must first carry comprehensive liability insurance.

"The risk of civilization extinction is real," Khanna told CNBC. "Both the safety hazards of失控 and the risks of abuse must be taken seriously." (Note: '失控' means 'losing control'.)

Giving Safety Experts a Seat at the Table

Khanna says the bill's approach came from discussions with independent non-profit AI safety groups, not from big tech executives. "I hope it becomes a model, giving the AI safety community a voice," he said. "They have been unfairly regarded as talking about science fiction and not taken seriously... This discussion should be led by the safety community, not by executives of cutting-edge laboratories."

With AI advancing faster than regulations can keep up, Khanna's proposal is a wake-up call. Will it pass? That's anyone's guess. But one thing's clear: the conversation about who controls AI is just getting started.

Key Points

  • No self-improvement: AI models can't rewrite their own code or objectives without government approval.
  • New federal agency: A dedicated body would license, audit, and set safety standards for frontier AI.
  • Independent auditors: Every lab must have auditors reporting directly to the agency.
  • Safety checklist: Sandboxes, air-gapped testing, kill switches, and chip controls are required.
  • Criminal liability: Deploying a destructive AI could be a crime against humanity; disabling safeguards is a crime.
  • Insurance mandate: Companies must carry liability insurance before releasing models.