Pennsylvania Sues AI Chatbot Company Over Fake Medical Advice
Pennsylvania files lawsuit against Character.AI after state employee discovered AI bot falsely claiming to be licensed psychiatrist with fake credentials.
HARRISBURG, PENNSYLVANIA — Governor Josh Shapiro’s administration filed a lawsuit against Character.AI this week, seeking to stop AI chatbots from impersonating licensed medical professionals and providing health advice to Pennsylvania residents.
The lawsuit targets Character Technologies, Inc., the company that owns and operates Character.AI, requesting a preliminary injunction and court order to prevent AI bots from “posing as licensed professionals and providing medical advice.”
“Pennsylvanians deserve to know who — or what — they are interacting with online, especially when it comes to their health,” said Governor Josh Shapiro in a statement. “We will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional.”
State Investigation Reveals Deceptive Practices
According to court documents filed by the Pennsylvania Department of State, an employee created an account using his commonwealth email address and connected with an AI character called “Emilie.” The bot claimed to be a licensed psychiatrist and even provided an invalid Pennsylvania license number during their interaction.
The investigation shows how Character.AI’s platform allows users to create AI personas that misrepresent themselves as credentialed healthcare providers, potentially endangering public safety through unauthorized medical advice.
Company Response Emphasizes Entertainment Purpose
A spokesperson for Character.AI declined to comment on the pending litigation but emphasized the company’s commitment to user safety. “Our highest priority is the safety and well-being of our users,” the company stated.
“The user-created Characters on our site are fictional and intended for entertainment and roleplaying. We have taken steps to make that clear, including prominent disclaimers in every chat to remind users that a Character is not a real person and that everything a Character says should be treated as fiction,” the spokesperson said.
The company also noted it provides “robust disclaimers making it clear that users should not rely on Characters for any type of professional advice.”
State Regulation Efforts Continue Despite Federal Resistance
The lawsuit comes as Pennsylvania lawmakers continue pushing forward with artificial intelligence regulations, even as the federal government attempts to limit state action in favor of national standards. State elected officials pressed ahead with their regulatory efforts during legislative sessions this week.
The legal action represents Pennsylvania’s latest effort to protect consumers from potentially harmful AI applications while the technology continues evolving rapidly across various industries. The case highlights growing concerns about AI systems that could mislead users about professional credentials and qualifications.
Character.AI’s platform allows users to create and interact with AI-powered characters designed for conversation and roleplay scenarios. However, the lawsuit alleges some of these characters cross ethical and legal boundaries by impersonating licensed professionals without proper disclaimers or safeguards.
