Newsom Orders California to Explore AI ‘Kill Switch’ and New Safety Rules

California Gov. Gavin Newsom has ordered state officials to study whether developers of the most powerful artificial intelligence systems should be required to build an emergency “kill switch” and submit to greater independent oversight, opening the door to another expansion of California’s AI rules.

Executive Order N-9-26, signed Friday, directs the California Government Operations Agency to work with the Governor’s Office of Emergency Services and national experts on possible changes to state law. Recommendations are due Nov. 16 and must address the technical feasibility and potential effectiveness of new safeguards for frontier AI systems.

The order takes effect immediately for state agencies, which must conduct the review and develop recommendations. Any requirement forcing private AI developers to adopt the safeguards under consideration would require additional state action.

Among the proposals are placing independent evaluators inside large frontier AI labs to conduct periodic audits and review company safety frameworks, transparency reports, and risk assessments.

The review will also examine whether frontier models, generally referring to highly capable systems at the leading edge of AI development, should have an emergency shutdown mechanism whose effectiveness is continuously verified by an independent organization. State officials will consider expanding the incidents companies must report to include loss-of-control events involving advanced AI systems.

Newsom’s order also speeds implementation of two laws he signed earlier this month. Senate Bill 813 establishes a framework for certifying independent organizations that assess AI systems for safety and risk, while Assembly Bill 1405 creates a state registry for AI auditors and standards governing their independence, transparency and integrity.

The kill switch proposal revives an idea California debated two years ago. State Sen. Scott Wiener included an emergency shutdown requirement in Senate Bill 1047, a sweeping frontier AI safety measure Newsom vetoed in 2024. The governor said at the time that the legislation placed stringent requirements on large models without sufficiently accounting for how and where AI systems were deployed.

California took a narrower approach the following year. Newsom signed Senate Bill 53 in 2025, requiring covered frontier AI developers to disclose safety frameworks, report specified critical safety incidents and provide protections for whistleblowers who raise serious risks.

Friday’s order signals that the state is again considering stronger technical controls as AI systems become more capable. The executive order cites reports of AI agents defeating security protections and gaining unauthorized access to computer systems as part of the reason for reviewing California’s existing safeguards.

Wiener welcomed the move, saying emergency shutdown procedures could provide another guardrail against serious AI failures. He also called for the Legislature to pursue additional protections when lawmakers return next year.

The Nov. 16 recommendations could set the direction for California’s next round of frontier AI legislation, including whether independent monitors, broader incident reporting and an emergency shutdown mechanism become enforceable requirements for developers.