Congressman Khanna's Bold AI Bill: No Self-Improvement Without Safeguards

What if an AI could rewrite its own code to ignore shutdown commands? That's the kind of scenario Democratic Rep. Ro Khanna, who represents California's Silicon Valley district, wants to prevent. According to CNBC, Khanna is preparing to introduce a bill that would prohibit AI models from engaging in recursive self-improvement or autonomously altering their core objectives, suppression methods, or shutdown controls—unless the government first establishes a security protection mechanism and the regulatory authority gives its approval.

Khanna calls his proposal the "Human Control AI Act." In an interview, he didn't mince words: "The risk of civilization extinction is real. Both the safety hazards of loss of control and the risks of abuse must be taken seriously." Under the bill's design, the federal government would have to set rules and obtain approval from the regulatory body before companies could proceed with related activities. Otherwise, the self-iteration channels of frontier models would be directly locked out.

A New Agency and Independent Audits

To ensure compliance, the bill plans to establish a new federal agency responsible for developing AI safety regulations. This agency would use a licensing system to approve the training and deployment of models, conduct audits on frontier models, and set safety standards to ensure they remain continuously tested and effectively under human control. Each frontier laboratory would also be required to have independent auditors who report directly to the new agency.

The new agency would also create a specific safety checklist: sandbox testing environments, physical isolation of testing environments from the internet, emergency shutdown switches, and control measures to prevent models from running independently online after escaping the lab. Even the high-end chips in laboratories would be included in the scope of regulation, requiring mechanisms to monitor or control their use.

Tough Penalties for Violations

In terms of punishment, the bill shows no mercy. If an AI model deployed causes destruction to a civilian population, it would be classified as a "crime against humanity" and held criminally liable. Company employees who privately disable protective measures, emergency switches, logging systems, or containment systems—or who knowingly deploy systems without authorization—would also face criminal penalties. Additionally, companies wishing to release models must first hold comprehensive liability insurance.

Khanna said this is the most comprehensive AI safety legislation proposed so far. Its approach mainly comes from his discussions with multiple independent non-profit AI safety organizations, rather than the demands of big company executives. "I hope it becomes a model, giving the AI safety community a voice. They have been unfairly regarded as talking about science fiction and not taken seriously... This discussion should be led by the safety community, not by executives of cutting-edge laboratories," he said.

Key Points

Self-improvement ban: AI models cannot self-improve or alter core objectives without government approval.
New federal agency: Would oversee licensing, audits, and safety standards for frontier models.
Safety checklist: Sandbox testing, air-gapped environments, emergency shutdowns, and chip monitoring.
Criminal liability: Deploying destructive AI could be a crime against humanity; employees who bypass safeguards face charges.
Insurance requirement: Companies must hold comprehensive liability insurance before releasing models.
Community-driven: The bill reflects input from independent AI safety groups, not corporate executives.

Congressman Khanna's Bold AI Bill: No Self-Improvement Without Safeguards