California is banning AI chatbot toys for children under 16 after safety tests found some products could discuss sexual topics and tell kids where to find knives, pills and matches.
Gov. Gavin Newsom signed Senate Bill 867 on Sept. 10, creating a four-year moratorium on the manufacture and sale of covered toys beginning Jan. 1, 2027. The prohibition runs through the end of 2030.
Concerns surrounding the products intensified after consumer researchers tested AI-enabled toys capable of holding open-ended conversations with children. A U.S. PIRG Education Fund investigation found that an AI teddy bear called Kumma provided information about where to find potentially dangerous objects, including knives, pills, and matches, and, during extended conversations, discussed sexually explicit subjects.
Researchers also found that another AI toy told a test user whose age was set at 5 where matches could be found. Kumma later stopped giving some of the troubling responses after its maker conducted a safety audit, according to the report, showing how quickly the behavior of AI-powered toys can change through software updates.
SB 867 applies to physical products designed, marketed or manufactured as toys for children under 16 when they contain what California law defines as a companion chatbot. The definition covers AI systems capable of providing adaptive, human-like responses and sustaining a social relationship with a user across multiple interactions.
The law bars companies from manufacturing, selling or exchanging covered toys in California, along with possessing them with the intent to sell or offering them to retailers. Violations can trigger civil remedies available under the state's companion chatbot law.
More limited technology remains outside the definition. The statute excludes certain video game characters whose conversations stay within the game and stand-alone voice assistants that do not sustain an ongoing relationship with users. AI features inside a toy also remain outside the moratorium unless they meet California's definition of a companion chatbot.
State Sen. Steve Padilla, who authored the bill, said the four-year pause is intended to give lawmakers time to develop safety standards as increasingly sophisticated AI enters products marketed to children. His office has described SB 867 as the first state moratorium of its kind.
The Toy Association opposed the moratorium and favored studying the technology while developing safety standards. Following Newsom's signature, the trade group said it would continue providing the toy industry with guidance on emerging AI technology and monitoring similar legislation.
The ban comes as California expands its regulation of AI products used by minors. Newsom signed SB 867 alongside a broader package of child technology laws, including new requirements governing companion chatbots, parental controls and child safety assessments.
Unless lawmakers extend or replace the restriction, the AI toy prohibition will expire Jan. 1, 2031.
California will bar social media platforms from offering addictive features such as infinite scroll and autoplay to users under 16, impose new safeguards on AI companion chatbots and expose some companies to damages reaching $1 million per child under a sweeping package signed Thursday by Gov. Gavin Newsom.Signed September 10,... Read More »
California will require advertisers to tell consumers when a realistic actor selling a product or service was generated by artificial intelligence under a new law signed Wednesday by Gov. Gavin Newsom.Senate Bill 1050 requires a clear and conspicuous disclosure when an audio, video or audiovisual advertisement prominently features an AI-generated... Read More »
California lawmakers have approved legislation that would impose new requirements on attorneys who use generative artificial intelligence, including verifying AI-generated work, disclosing its use in court filings, and protecting confidential client information.Senate Bill 574, known as the Court A.I. Protection Act, passed the Legislature and was sent to Gov. Gavin... Read More »
Elon Musk’s artificial intelligence company xAI has sued Minnesota over the state’s first-in-the-nation ban on tools that create realistic nude images of identifiable people, arguing the law restricts protected expression along with abusive content.The company filed the case July 27 in federal court, days before the measure took effect Aug.... Read More »
California’s new AI transparency rules took effect August 2, requiring major generative artificial intelligence companies to give the public tools for identifying images, video, and audio created or altered by their systems.The California AI Transparency Act applies to companies whose generative AI systems receive more than 1 million monthly visitors... Read More »
