Congressman Khanna Moves to Curb Runaway AI with Sweeping New Bill

Silicon Valley's own representative, Ro Khanna, is taking a hard line on artificial intelligence. On September 28, the Democratic congressman announced plans to introduce what he calls the Human Control AI Act—a bill that would put the brakes on AI models that can improve themselves without government oversight.

At its core, the legislation targets a specific danger: AI systems that can rewrite their own code or objectives without human approval. Under the proposal, frontier models would be barred from any form of recursive self-improvement until a federal safety mechanism is in place and regulators give the green light. In other words, no self-upgrading until we know it's safe.

A New Watchdog with Teeth

To enforce these rules, the bill calls for a brand-new federal agency dedicated to AI safety. This agency would run a licensing system for training and deploying advanced models, conduct independent audits, and set safety standards. Every frontier lab would need its own independent auditors reporting directly to the agency—no more self-policing.

The agency would also maintain a safety checklist: sandbox testing environments, physical isolation from the internet, emergency kill switches, and safeguards to prevent models from escaping and running wild online. Even high-end chips used in labs would fall under scrutiny, with monitoring mechanisms to control their use.

Criminal Penalties and Insurance Requirements

Khanna's bill doesn't just wag a finger—it carries serious consequences. If a deployed AI model causes destruction to civilians, it could be prosecuted as a crime against humanity. Employees who knowingly disable safety measures, kill switches, or containment systems—or who deploy unauthorized systems—would face criminal charges. And companies hoping to release models must first secure comprehensive liability insurance.

"The risk of civilization extinction is real," Khanna said in an interview. "Both the safety hazards of loss of control and the risks of abuse must be taken seriously."

A Voice for the Safety Community

Khanna says this is the most comprehensive AI safety legislation proposed to date. Its approach comes from discussions with independent non-profit AI safety organizations, not from big tech executives. "I hope it becomes a model, giving the AI safety community a voice," he said. "They have been unfairly regarded as talking about science fiction and not taken seriously... This discussion should be led by the safety community, not by executives of cutting-edge laboratories."

Key Points

No self-improvement without approval: AI models cannot rewrite their core objectives or shutdown controls until a government safety mechanism is in place.
New federal agency: A dedicated body would license, audit, and set safety standards for frontier AI models.
Independent auditors: Each lab must have auditors reporting directly to the agency.
Criminal liability: Deploying unsafe AI or disabling safeguards could lead to criminal charges, including crimes against humanity.
Insurance mandate: Companies must hold liability insurance before releasing models.
Safety community first: The bill prioritizes input from AI safety nonprofits over tech executives.

Congressman Khanna Moves to Curb Runaway AI with Sweeping New Bill