How to stay in the AI game: Scan millions of books. Then shred them.
Tech companies are using print books to build AI models, but critics decry the literary destruction and what some call intellectual property theft
Getty Images / Photo by Stéphane Ouzounoff / Hans Lucas / AFP
Behind today’s major artificial intelligence models lie printed books—millions of them. Books are a key component that AI companies use to train the language models powering chatbots, enabling them to be accurate and sound natural.
But to obtain the contents of those books quickly, at least some tech companies have reportedly been destroying the volumes in the process—cutting out the pages and feeding them into scanners before ultimately shredding them—leading to a public outcry.
Internal documents unveiled in a court case earlier this year showed that Anthropic, the artificial intelligence company behind AI model Claude, was scanning and destroying books in a large-scale effort to train its models. Documents suggested the San Francisco–based company had destroyed over 2 million books so far. Anthropic referred to the setup as Project Panama and kept it secret, acknowledging in internal documents that it didn’t want the public to know about the project. It instructed employees not to mention it publicly.
A recent 404 Media investigation tracked a single order of nearly 1,000 books to an Amazon warehouse in Las Vegas, where the outlet reported an operation called VGT3 scans books for AI training. An employee at the warehouse reportedly told 404 Media the workers’ job was to receive massive shipments of printed books and quickly disassemble them for scanning. Hydraulic cutters removed the spines from the books, cutting the pages free, and other machines scanned the loose pages before employees disposed of them, the employee said.
When I asked Amazon about the report, the company declined to answer my questions about how many books it had purchased and what it was doing with them. Amazon told me the company purchased books through commercial channels to help develop and improve the products and services its customers used.
In the wake of the revelations about AI book training, used booksellers have expressed worry about rare books going through Anthropic’s pipeline. One bookseller told 404 Media that AI companies are destroying rare books with almost no surviving copies, causing irreversible damage to the literary community. Some public commentators have described the mass book destruction as scenes out of Ray Bradbury’s Fahrenheit 451 and George Orwell’s 1984 or made historical connections to the Nazis’ book burnings in World War II. While others say the tech companies’ practice is more practical than nefarious, it raises questions about the limits of fair use of intellectual property and the loss of hard-to-find books.
“It certainly shows a fundamental disrespect for the written word,” award-winning children’s author Polly Holyoke told me. “Maybe the good side of it is that they’re saving something. But I find it disturbing, the lack of respect for the written word, the lack of respect for the law and for authors’ and songwriters’ and musicians’ rights.”
AI companies are hunting for books written before the era of large language models, aiming to ensure they avoid feeding into their AI models books that have themselves been written by AI. As a result, used book dealers have seen orders placed for thousands of niche print copies. Dutch antiquarian bookseller Pieter de Vries received an order for over 3,000 titles, including some that were hard to find, he told Dutch news outlet BNR. European booksellers, many of whom sell antique or rare copies, have seen orders placed for large quantities of older, hard-to-find European books, according to BNR.
The vice president of the International League of Antiquarian Booksellers, Mats Petersson, said the group is closely monitoring the situation. “ILAB represents around 1,600 rare booksellers in 39 countries worldwide who share a deep love for books and a commitment to the preservation of our shared cultural heritage,” Petersson said. “Should evidence emerge that books are being unnecessarily destroyed, ILAB would strongly oppose such practices. At the same time, ILAB cannot prescribe where its affiliated members sell books, nor can it tell customers what they may do with a legally purchased item.”
To obtain single copies of millions of different books, AI companies appear to be using third-party companies to place bulk orders. Booksellers in Europe and Canada that have seen large orders of books come in have sometimes cited the buyer as Zoom Books, a Canadian company that offers shipments by the truckload of up to nearly 40,000 books at a time. In response to media queries, Zoom Books has denied it digitizes or destroys books, although it would not reveal its clients, citing confidentiality agreements.
Some companies have realized that the business of destroying books doesn’t sit well with the public. One company, ISBNdb, specializes in cataloging books and boasted on its website earlier this year that the world’s best AI training data is sitting on a shelf. But it also acknowledged that the optics of an AI company destroying 2 million books is not a sympathy-generating headline.
“Responsible physical sourcing is not book burning,” ISBNdb said in the web article. “It is the completion of a book’s lifecycle: from tree to knowledge to tree again.”
ISBNdb recently deleted the article from its website, and it also removed a landing page offering bulk book purchase services to AI companies. When I reached out to ISBNdb, the company responded with a statement saying it has never purchased, scanned, or destroyed a book for AI training.
“The quotes from our site are marketing copy from the landing page and related posts for the service we were exploring market interest,” the company said. “We put the page up after the Project Panama coverage, to see whether there was demand. ISBNdb never bought or supplied printed books for AI training—no orders, no purchases, no books.”
Ironically, tech companies may be destroying the books used for AI training in part to avoid breaking U.S. copyright law. A judge in a 2024 class-action copyright case, which several authors brought against Anthropic for illegally downloading over 7 million books for AI training, ruled in Anthropic’s favor when it came to destroying books. The judge reasoned that as long as Anthropic legally acquired print books and then only retained one digital copy through its destructive scanning, it wasn’t violating copyright law.
Although Anthropic has agreed to pay out $1.5 billion in damages to thousands of authors affected by its illegal book downloads, the company walked away victorious on the matter of training its AI model on legally acquired books. The judge deemed that practice to be fair use.
David Sacks, co-chair of the President’s Council of Advisors on Science and Technology, accused Anthropic of hypocrisy. He said on social media that while the AI company maintains it is entitled to train for free on all the world’s output, in spite of objections by the content’s authors, the company deems any similar training on its own output to be intellectual property theft.
Meanwhile, book authors continue to object to the use of their works for AI training.
“They stole from us,” said Stuart Gibbs, a New York Times bestselling author who took part in the suit against Anthropic. “They took my work without my permission and they didn’t negotiate anything. They did it first and then afterwards negotiated the settlement, but it’s not nearly what I feel my work, or anybody’s work, is worth.”
Gibbs added: “The reason someone can write a book or tell a joke or write a song is because they’ve spent thousands of hours honing that craft, and for somebody to come along and push a button and say ‘Now I can do what you do’ is incredibly offensive.”
Polly Holyoke agreed. “I think it’s really remarkable to live in an age where knowledge is accessible to so many people,” she said. “But what Anthropic and the other big AI companies have done is not just about making knowledge available. They’ve literally demolished the protections that copyright law afforded to creative folks, and that is truly a crime and a terrible shame.”
Holyoke said that the settlement payout she expects to receive—$3,000 per book—is insufficient compensation for the “thievery” of her creative output. “Anthropic decided to totally disregard copyright law and 300 years of precedent. … They’re doing the minimum amount to redress the situation.”
“They can study and scan 7 million books and pretend to be us, but are they going to be able to make us cry?” she said of AI attempts to mimic human authorship. “They have not lived a lifetime and had all of the experiences that we have.”
I appreciate your honest film reviews. —Jeff
Sign up to receive Muse, WORLD’s free weekly email newsletter on arts and culture.
Please wait while we load the latest comments...
Comments
Please register, subscribe, or log in to comment on this article.