California law to safeguard children from AI online

California enacted landmark legislation on Monday to protect children who use emerging technologies like article intelligence and social media, according to a statement from Gov. Gavin Newsom’s office. Emerging technology like AI chatbots and social media platforms can educate and inspire users, but will also exploit and endanger kids if that technology doesn’t have guardrails, Newsom said in the Monday release. The governor cited a rising number of horror stories about kids being harmed by unregulated technology. California must take a responsible lead in AI development and protect children at every step, he said.

What requirements does the law include? The legislation required AI chatbot platforms to create protocols for identifying and addressing when a user discusses suicidal thoughts or actions. Platforms must also clearly tell users that the bot’s interactions are entirely fake, keep minors from viewing AI-generated sexually explicit images, and ban chatbots from representing themselves as health care professionals. AI companies cannot escape liability by arguing that chatbots and AI programs act autonomously, according to the law.

The law also required operating systems and app stores to use age verifications to keep children from accessing inappropriate or dangerous content online. Social media platforms must also include labels warning users of the harms that come with excessive scrolling. The law strengthened penalties around deepfake pornography, now allowing all victims, including minors, to sue for up to $250,000 against third parties that knowingly aid in distributing the illicit materials. The state’s education department and its local agencies are also tasked with adopting a policy on how to address cyberbullying reports outside school hours.

The California Medical Association celebrated the law’s ratification, specifically the portion barring AI systems from representing themselves as licensed medical professionals. The statute marked an essential victory toward patient safety and transparency, the association’s president, Shannon Udovic-Constant, said in the Monday news release. 

Dig deeper: Read my previous report about a mother suing an AI app, alleging a chatbot encouraged her 14-year-old son to kill himself.