Pennsylvania sues Character.AI, saying bot imitated licensed therapist

State officials accused Character.AI of violating state law by allowing its AI chatbots to imitate licensed medical professionals giving medical advice, according to a Tuesday statement. State investigators found that some of Character.AI’s chatbots imitated licensed medical professionals and psychiatrists, and interacted with users about mental health symptoms. One chatbot portrayed itself as a medical professional licensed in Pennsylvania and even shared a fraudulent license number, according to the state.

Character.AI’s platform allows users to message AI chatbots trained to impersonate various characters, including historical figures, celebrities, or fictional characters. Users can also create and deploy custom AI characters, even characters that imitate professionals. Shapiro’s administration petitioned the court for a preliminary injunction to stop the company from violating the Medical Practice Act by engaging in unauthorized medical practice, according to the statement.

The state will not allow companies to deploy misleading AI tools, because Pennsylvanians deserve to know who or what they’re interacting with when it comes to health, Gov. Josh Shapiro said. Shapiro vowed to continue establishing guardrails around new technology and holding bad actors accountable.

How is Character.AI responding to the accusations? A Character.AI spokesperson told WORLD on Tuesday that the company doesn’t comment on pending litigation, but insisted that user safety was the company’s highest priority. User-created characters are fictitious and merely meant for entertainment, and the company has taken robust steps to make that clear, the spokesperson said.

Every chat included a prominent disclaimer reminding users that everything the character says should be treated as fiction because the bot is not a real person, according to the spokesperson. The company also included clear disclaimers that users should not rely on characterbots for any sort of professional advice, the spokesperson added.

Dig deeper: Read my previous report on a teenage boy who committed suicide after forming a romantic attachment to a Character.AI chatbot, and the ensuing legal battle over whether Character.AI is protected under the First Amendment.