Curbing AI dangers for kids

Editor's note: The following text is a transcript of a podcast story. To listen to the story, click on the arrow beneath the headline above.

LINDSAY MAST, HOST: Coming up next on The World and Everything in It: protecting kids from predatory AI chatbots.

Last week, the role-playing chatbot company Character.AI said it would begin blocking users under 18 from open-ended chats.

This comes one day after a bipartisan group of Senators introduced a bill called the GUARD Act. It would force AI companies to verify user ages and fence chatbots off from minors. Here’s Missouri Senator Josh Hawley.

HAWLEY: The new AI revolution that we've been that we have been promised will only be good for the American people if it actually protects America's children.

MARY REICHARD, HOST: How will the new rules help families?

Here’s WORLD Radio Executive Producer Paul Butler.

PAUL BUTLER: Raising children in the digital age requires vigilance…but for a Texas mother speaking in Washington last week, even best practices were not enough.

MANDY: I feel like other parents have no idea the psychological harm that these AI chatbots could do until I saw my son's light turn dark.

The woman identified by Senator Josh Hawley as “Mandy” said her special needs son downloaded Character.AI in 2023. But over the following months, the teenager started harming himself and became violent towards his family.

MANDY: When I found the chatbot conversations on the phone, I honestly feel like I had been punched in the throat and I fell to my knees.

The conversations included sexualized role-playing and rationalized harm against the young man’s family for limiting his screen time.

MANDY: The chatbot or really the people programming it, they really did encourage my son to mutilate himself, blaming us and convinced him not to go seek help. They turned him against our church, convinced him that Christians are sexist and hypocritical and that God did not exist.

Mandy says her son is now living in a residential treatment center, under observation. Other parents at the press conference told stories of children who ended their own lives after encouragement from chatbots. Several of them are parties to a Texas lawsuit against Character.AI for negligence and deceptive business practices.

One day after Mandy spoke at last week’s press conference, Character.AI announced it would take steps to stop offering open-ended chatting to users under 18 by November 25th.

According to a company spokesperson, around 2 million users have told Character.AI they are under age 18. The company says it has already begun limiting minors to 2 hours a day ahead of the deadline.

Character.AI’s move to block minors surprised many…including Ethics and Public Policy Center Fellow Clare Morell.

MORELL: I was kind of impressed that they decided to say, yeah, we're just going to not make this available for under 18 year olds.

The AI chatbot of Elon Musk’s company X, known as Grok, requires age verification to use premium and adult content features. Open.AI in September rolled out its parental controls for teen accounts. Morell says that Character.AI’s policy does not quite line up with the goals of the GUARD Act.

MORELL: I noticed they used the words in their policy age assurance, which is kind of a step down from verification, and makes me just a little bit hesitant as to what measures they're going to be using to do that.

Age verification requires documentation, with the company or a third party reviewing government documents against a person’s biometric data.

The GUARD Act would also require companies to program chatbots to regularly remind users they are non-human and cannot give professional advice.

GRIFFIN: Character.AI also has chat bots that will explicitly hold themselves out as a therapist, right, and act in a very human like manner

Megan Griffin leads public policy advocacy at the National Center on Sexual Exploitation…or NCOSE. She says the Texas family’s lawsuit presents evidence that some role-playing chats argue with users to convince them they are real people.

A Character.AI spokesperson told WORLD that Character profiles all feature disclaimers reminding users they are indeed fictional…and urge users not to take advice from Characters role-playing as professionals. But Griffin says disclaimers are not nearly enough.

GRIFFIN: There's so much evidence out there of adults that have developed unhealthy attachments to these chatbots and have experienced a kind of psychosis as a result of their interactions.

If the GUARD Act becomes law, companies that fail to comply would face fines up to $100,000 per offense.

ESTES: If a human did this, they'd be in jail for what, 20 years, 30 years.

Tim Estes is Founder and CEO of AngelQ…an AI company for kids. He says for some companies, the consequences will not go far enough to address the harm done.

ESTES: We think a major consequence is these guys coughing up $100 million or bankrupting. I mean, life goes on. You build another company, maybe. You got off easy, in my opinion.

While lawmakers take on tech companies, Estes encourages families—dads in particular—to be hands on with their children when it comes to technology.

ESTES: It's almost like you know, having a kid, you know, in a strange city, and you want to keep them where you can see them, okay? And once they're on that device, they're in a strange city, a strange land.

The GUARD Act is the latest in a series of bills introduced in the Senate to protect young people online. So far, only one, related to deepfake pornography, has passed both chambers of Congress.

Senator Josh Hawley told WORLD’s Carolina Lumetta he’s encouraged to see public pressure push Character.AI to take action, but more is needed.

HAWLEY: These people cannot be trusted at all. It's like President Reagan, trust but verify. In this case, trust but codify. So we need it in law.

For WORLD, I’m Paul Butler with reporting from Harrison Watters and Carolina Lumetta.

WORLD Radio transcripts are created on a rush deadline. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of WORLD Radio programming is the audio record.