California Gov. Gavin Newsom has ordered state officials to study whether developers of the most powerful artificial intelligence systems should be required to build an emergency “kill switch” and submit to greater independent oversight, opening the door to another expansion of California’s AI rules.
Executive Order N-9-26, signed Friday, directs the California Government Operations Agency to work with the Governor’s Office of Emergency Services and national experts on possible changes to state law. Recommendations are due Nov. 16 and must address the technical feasibility and potential effectiveness of new safeguards for frontier AI systems.
The order takes effect immediately for state agencies, which must conduct the review and develop recommendations. Any requirement forcing private AI developers to adopt the safeguards under consideration would require additional state action.
Among the proposals are placing independent evaluators inside large frontier AI labs to conduct periodic audits and review company safety frameworks, transparency reports, and risk assessments.
The review will also examine whether frontier models, generally referring to highly capable systems at the leading edge of AI development, should have an emergency shutdown mechanism whose effectiveness is continuously verified by an independent organization. State officials will consider expanding the incidents companies must report to include loss-of-control events involving advanced AI systems.
Newsom’s order also speeds implementation of two laws he signed earlier this month. Senate Bill 813 establishes a framework for certifying independent organizations that assess AI systems for safety and risk, while Assembly Bill 1405 creates a state registry for AI auditors and standards governing their independence, transparency and integrity.
The kill switch proposal revives an idea California debated two years ago. State Sen. Scott Wiener included an emergency shutdown requirement in Senate Bill 1047, a sweeping frontier AI safety measure Newsom vetoed in 2024. The governor said at the time that the legislation placed stringent requirements on large models without sufficiently accounting for how and where AI systems were deployed.
California took a narrower approach the following year. Newsom signed Senate Bill 53 in 2025, requiring covered frontier AI developers to disclose safety frameworks, report specified critical safety incidents and provide protections for whistleblowers who raise serious risks.
Friday’s order signals that the state is again considering stronger technical controls as AI systems become more capable. The executive order cites reports of AI agents defeating security protections and gaining unauthorized access to computer systems as part of the reason for reviewing California’s existing safeguards.
Wiener welcomed the move, saying emergency shutdown procedures could provide another guardrail against serious AI failures. He also called for the Legislature to pursue additional protections when lawmakers return next year.
The Nov. 16 recommendations could set the direction for California’s next round of frontier AI legislation, including whether independent monitors, broader incident reporting and an emergency shutdown mechanism become enforceable requirements for developers.
California is banning AI chatbot toys for children under 16 after safety tests found some products could discuss sexual topics and tell kids where to find knives, pills and matches.Gov. Gavin Newsom signed Senate Bill 867 on Sept. 10, creating a four-year moratorium on the manufacture and sale of covered... Read More »
California will require advertisers to tell consumers when a realistic actor selling a product or service was generated by artificial intelligence under a new law signed Wednesday by Gov. Gavin Newsom.Senate Bill 1050 requires a clear and conspicuous disclosure when an audio, video or audiovisual advertisement prominently features an AI-generated... Read More »
California will bar social media platforms from offering addictive features such as infinite scroll and autoplay to users under 16, impose new safeguards on AI companion chatbots and expose some companies to damages reaching $1 million per child under a sweeping package signed Thursday by Gov. Gavin Newsom.Signed September 10,... Read More »
California lawmakers have approved legislation that would impose new requirements on attorneys who use generative artificial intelligence, including verifying AI-generated work, disclosing its use in court filings, and protecting confidential client information.Senate Bill 574, known as the Court A.I. Protection Act, passed the Legislature and was sent to Gov. Gavin... Read More »
OpenAI and CEO Sam Altman are facing 30 new federal lawsuits from students, teachers and a school principal affected by the Tumbler Ridge mass shooting, expanding litigation that claims the company failed to alert Canadian authorities after its safety team identified the shooter as a potential threat months before the... Read More »
