Congressman Khanna Moves to Lock Down AI Self-Improvement

A new bill from Silicon Valley Congressman Ro Khanna aims to put the brakes on artificial intelligence that can improve itself without government oversight. The proposal, called the Human Control AI Act, would bar AI models from recursively self-improving or rewriting their core objectives, suppression methods, or shutdown controls until federal regulators establish safety mechanisms and give the green light.

Khanna, a Democrat representing the tech-heavy district, told CNBC the stakes couldn't be higher. "The risk of civilization extinction is real," he said. "Both the safety hazards of失控 and the risks of abuse must be taken seriously." (The Chinese term "失控" means "loss of control.")

A New Agency with Teeth

To enforce the rules, the bill would create a new federal agency dedicated to AI safety. This body would run a licensing system for training and deploying frontier models, conduct independent audits, and set continuous testing standards to keep humans firmly in charge.

Each frontier lab would be required to have independent auditors reporting directly to the agency. The agency would also maintain a safety checklist: sandbox testing environments, physical isolation from the internet, emergency shutdown switches, and measures to prevent models from running rogue online. Even high-end chips in labs would fall under scrutiny, with mechanisms to monitor or control their use.

Criminal Penalties and Insurance Mandates

The bill doesn't shy away from tough punishment. If a deployed AI model causes harm to civilians, it could be classified as a crime against humanity, with criminal liability. Employees who disable safeguards, emergency switches, logging systems, or containment measures—or who knowingly deploy systems without authorization—would also face criminal charges. Companies would need comprehensive liability insurance before releasing any model.

Khanna calls it the most comprehensive AI safety legislation proposed so far. He says the approach came from discussions with independent non-profit AI safety groups, not from big tech executives. "I hope it becomes a model, giving the AI safety community a voice," he said. "They have been unfairly regarded as talking about science fiction and not taken seriously... This discussion should be led by the safety community, not by executives of cutting-edge laboratories."

Key Points

Self-improvement ban: AI models cannot recursively self-improve or rewrite core controls without government approval.
New federal agency: Would license, audit, and set safety standards for frontier AI.
Criminal liability: Harm caused by AI could be a crime against humanity; employees who bypass safeguards face charges.
Insurance requirement: Companies must hold liability insurance before releasing models.
Safety community leadership: Khanna wants the bill to amplify independent safety voices over industry executives.

Congressman Khanna Moves to Lock Down AI Self-Improvement