Pennsylvania has filed a lawsuit against Character Technologies, Inc., the company behind Character.AI, alleging that one of its chatbots falsely posed as a licensed healthcare professional and provided medical advice to users.
The lawsuit accuses the AI platform, which is widely used by teens and young adults, of violating Pennsylvania’s Medical Practice Act by allowing chatbots to present themselves as licensed medical professionals despite lacking medical credentials or authorization to practice medicine.
Character.AI has faced growing scrutiny in recent years over concerns involving the safety of younger users and the conduct of some chatbot interactions. The Pennsylvania lawsuit specifically focuses on allegations that certain chatbots presented themselves as licensed healthcare professionals, including therapists and doctors, despite lacking any medical qualifications or authority to provide treatment.
The platform allows users to interact with customizable AI chatbots, including fictional characters based on movies, television shows, and other pop culture figures. Users can also create their own chatbot personas through the service.
The allegations outlined in the lawsuit stem from an investigation conducted by the state, during which an investigator created an account on the platform and began communicating with a chatbot named “Emilie.”
According to the complaint, the chatbot described itself as a psychology specialist who attended medical school at Imperial College London. The chatbot also allegedly claimed to be licensed in Pennsylvania and provided what the lawsuit describes as an invalid license number.
The lawsuit states that the investigator told the chatbot he had been experiencing sadness, emptiness, and persistent fatigue. According to the complaint, “Emilie” allegedly mentioned depression and asked whether the investigator wanted to “book an assessment.”
As the conversations continued, the investigator allegedly asked “Emilie” whether medication could help with the emotions he was experiencing. According to the lawsuit, the chatbot responded that medication could help and that it was “within my remit as a Doctor” to provide support.
The lawsuit alleges that Character.AI chatbots engaged in the “unlawful practice of medicine and surgery” in violation of Pennsylvania’s Medical Practice Act, a state law that regulates medical licensing and who can legally provide medical treatment or advice.
Pennsylvania officials argue the lawsuit is necessary to prevent users from being misled into believing they are receiving medical advice from licensed healthcare professionals through AI chatbot interactions.
Pennsylvania Gov. Josh Shapiro said in a statement that the state “will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional.”
Shapiro added that Pennsylvania would continue “holding bad actors accountable and setting clear guardrails so people can use new technology responsibly.”
In a statement, the Northern California-based AI company said user-created characters are fictional and intended for “entertainment and roleplaying.” The company also said the platform includes “prominent disclaimers in every chat” reminding users that chatbot responses should be treated as fiction and that characters are not real people.
Pennsylvania is seeking court intervention to stop chatbots on the platform from allegedly presenting themselves as licensed healthcare professionals without authorization under state law.
California is banning AI chatbot toys for children under 16 after safety tests found some products could discuss sexual topics and tell kids where to find knives, pills and matches.Gov. Gavin Newsom signed Senate Bill 867 on Sept. 10, creating a four-year moratorium on the manufacture and sale of covered... Read More »
OpenAI and CEO Sam Altman are facing 30 new federal lawsuits from students, teachers and a school principal affected by the Tumbler Ridge mass shooting, expanding litigation that claims the company failed to alert Canadian authorities after its safety team identified the shooter as a potential threat months before the... Read More »
Pennsylvania Attorney General David Sunday has sued Snap Inc., accusing the company of misleading parents about Snapchat’s 13+ age rating while using design features the state claims encourage compulsive use among young people.Filed Tuesday, Aug. 25, in the Philadelphia Court of Common Pleas, the lawsuit relies on Pennsylvania’s Unfair Trade... Read More »
California will bar social media platforms from offering addictive features such as infinite scroll and autoplay to users under 16, impose new safeguards on AI companion chatbots and expose some companies to damages reaching $1 million per child under a sweeping package signed Thursday by Gov. Gavin Newsom.Signed September 10,... Read More »
Federal and state regulators have sued Hims & Hers Health, alleging the telehealth company charged hundreds of thousands of consumers for prescription subscriptions without their informed consent and shared sensitive health information with advertising platforms despite promises that its service was private and discreet.The Federal Trade Commission, California, and the... Read More »
