Regulating in the dark

Editor's note: The following text is a transcript of a podcast story. To listen to the story, click on the arrow beneath the headline above.

MARY REICHARD, HOST: Fifteen years, nearly 35-hundred editions—and still something new each weekday morning.

It’s Tuesday September 15th. Good morning, I’m Mary Reichard.

NICK EICHER, HOST: And I’m Nick Eicher.

First up on The World and Everything in It: The speed of AI and coming change.

As you heard earlier in the program, President Trump chimed in yesterday on a growing political fight over how fast AI is advancing. It started back in July, an incident known as the Hugging Face Hack, when seven hundred autonomous AI agents hacked the site in a coordinated swarm.

REICHARD: Then last week, a researcher who worked for Anthropic and OpenAI…said in a resignation post that “people building AI earnestly believe that it could kill us all by the end of the decade.”

This has gotten more than one hundred seventy million views.

EICHER: Some AI executives, including Sam Altman, Dario Amade, and Elon Musk, now say that they are open to slowing down. Senator Bernie Sanders has introduced legislation to ban superintelligence. But critics say a slowdown hands the race to China. And President Trump says existing laws are sufficient.

REICHARD: Well joining us now to talk about it is Neil Chilson. He is head of AI policy at the Abundance Institute. He's a lawyer, computer scientist, and former chief technologist at the Federal Trade Commission. Neil, good morning.

CHILSON: Thanks for having me on. It's great to be here.

REICHARD: Really glad you're here. Well, Anthropic’s Dario Amodei said a swarm of autonomous agents could hold the Internet hostage within 6 to 12 months. We've even heard fears that AI will be smart enough and evil enough to kill us all by 2030. So do you think he's wrong about the risk?

CHILSON: Dario has been raising concerns about AI since as long as he's been in the industry. And so he has a long track record of saying this about the very technologies that he's building. Obviously, AI is a powerful tool. It raises some risks. I think the hugging face incident has gotten a lot of attention. But overall, I think there are risks here, but they're much more mundane risks of misuse by people than they are of superintelligence taking over and killing us all.

EICHER: You know, I wonder about the kinds of provocative statements that get people's attention. The one that got my attention was killing us all by 2030. How is that even possible?

CHILSON: There is a common thing among very, very smart people that they think intelligence is the primary thing that shapes the world and I think all of us with more common sense know that it's it's more than just being very smart that makes somebody an effective communicator, that makes somebody an effective leader, that makes them, uh, you know, maybe an effective bad guy even.

EICHER: Right. And mankind does not exactly have the cleanest reputation.

But, you know, I don't want to just ridicule the idea. I do want to talk about the remedy that he's proposed. And you've argued for years against top-down control of complex systems. This is a call for voluntary, industry-led pace, which seems like a bottom-up alternative versus what we heard about Bernie Sanders' proposal of just let's ban the whole thing. What's the problem to be solved here?

CHILSON: So I think there's a distinction between the idea that companies might want to slow down themselves. I have no problem with Anthropic or OpenAI choosing to make this decision themselves. But it is not a bottom-up solution if the government is telling everybody, hey, we're going to set the pace of this race. We're going to set the milestones that you have to meet. That's the very definition of top-down control. And I think that raises a bunch of concerns for innovation, for the benefits to consumers, and for the international competition that we have going with China.

REICHARD: Well, I want to follow up on this a bit. You've proposed a federal push for frontier AI companies to reasonably mitigate catastrophic risk.

I'm going to ask you to put that in concrete terms, whether we're talking about open AI or Anthropic or anybody else for that matter. What would AI companies have to do under your plan that they don't have to do now?

CHILSON: Some people want a checklist. They want a boxed checklist of all the things that they need to do in order to comply with the law, with the idea that would make AI safe. I have a big concern with that type of approach because nobody fully understands the risk that's here because this is a very unusual new technology. There's another type of law called tort law that says you have a duty. We're not going to tell you how to execute that duty, but if something goes wrong, it's going to be your responsibility. And so the goal is to provide the incentive to the company.

And so my proposal would be that duty is very clear, that there be a way for them to ensure that the government can verify that they are performing the duties that they are claiming and that would provide the proper incentive without being micromanaging, without trying to pretend like we know how to govern this stuff when we don't. I think that would be a much better approach. And there is some active legislation in the Senate right now that takes a very similar approach.

EICHER: So speaking of incentives and you're speaking the language of economics, The Wall Street Journal is worried about another economic concept, and that's regulatory capture. The idea that regulation written in the name of safety could lock in the dominant market position of today's biggest AI companies. You're talking about a federal standard that would preempt state AI safety laws, and there's patchwork of that. But how do you build a national regime without turning the leaders of today into the protected incumbents of tomorrow?

CHILSON: Always a big challenge. And I worry about that.

The way you would set up a regime like this is to avoid those checklists. Those types of expensive checklists of compliance are the exact sort of things that the biggest companies can afford and the new upstarts can't necessarily afford. And I would say, here are the basic principles you have to comply with. And then make sure that there's not much discretion in the process. The other way that companies capture the government is If there's a lot of slop, if there's a lot of discretion that the government can exercise when it's making decisions about who gets to go ahead and who doesn't, then it's all about who has the best relationship with the regulators. And that's always going to be the incumbents, not necessarily for nefarious reasons. Often it's just because they've been around the longest and they've had to engage with the regulator. And the best way to avoid that general principle. Tort law, common law style types of regulation, consumer protection regulation, antitrust that are very general purpose, not targeted to a specific industry.

REICHARD: Well, Neil, speaking of relationships, Amodei called China the toughest dilemma and admitted that he does not know if coordination is even possible. So if Beijing doesn't slow down, at what point then does American caution itself become a national security risk, you think?

CHILSON: China feels very differently about AI. Even the population feels very differently about AI than the U.S. They're focused on delivering tools to the population. And AI over in China has like an 80% positive. Rate, right? It's almost the exact opposite here in the US. People are very skeptical of this technology, in part because they haven't seen the direct impacts on their life yet. And so China has some advantages here, both in the ability to build new energy, the ability to do manufacturing. They don't have the best models. They don't have the best scientists. We do in the US. But they're hungry for anything that would let them catch up.

REICHARD: Neil Chilson is the head of AI policy at the Abundance Institute. Neil, thank you.

CHILSON: Thank you for having me.

WORLD Radio transcripts are created on a rush deadline. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of WORLD Radio programming is the audio record.