AI companies gain a lot by training on data they don’t own, and little stops them inside their own walls. Penalties arrive years later and cost less than the advantage. Penalties also need proof, and nobody outside sees an internal model.
Court records show they have done it
Meta trained its Llama models on pirated books from a site called LibGen after license talks stalled. Anthropic also kept a library of pirated books. Last year it agreed to pay authors $1.5 billion, about $3,000 per book.
The labs accuse each other too. Anthropic said this year that labs in China used tens of thousands of fake accounts to copy what Claude knows.
Courts have mostly allowed the training
Two federal judges ruled last year that training AI on books can count as fair use. Anthropic paid $1.5 billion for keeping pirated copies, not for training. The biggest case, the New York Times against OpenAI, still has no ruling almost three years after filing.
Each company gains by taking first
A company that trains on everything pulls ahead of rivals who hold back. A company that holds back falls behind rivals who don’t. No company trusts the others to hold back, so each takes what it can. Game theorists call this a prisoner’s dilemma: each player acts in its own interest, and everyone ends up worse off.
The penalty doesn’t change that math. The $1.5 billion settlement set a US copyright record, but Anthropic was worth more than $180 billion when it agreed to pay. A company in a race treats that fine as a cost of business.
Nobody can check internal models
A lab can train models only for its own use, to test ideas or to make training material for the models it sells. Outsiders never see those models or their data.
Copyright law still covers them, but proof requires the training data. Outsiders see that data only when a lawsuit forces it out. Europe’s new AI law makes companies publish a summary of training data, but only for models they release. Research and internal models are exempt.
We can’t see inside these labs, so we argue from incentives, not against any one company. When the gain is large, the penalty small, and nobody checks, assume someone does it.
Your chats feed the pile
OpenAI and Anthropic both train on consumer chats unless you turn the setting off. Your refined pitch, tested pricing, or solved client problem can train a model your competitors also use.
So AI companies pull ahead while everyone else’s work gets more alike, as we covered in Asymmetric AI.
| Consumer AI apps | Business plans | Open models you run | |
|---|---|---|---|
| Your chats stay out of training by default | — | ||
| You control where your data goes | — | Partly | |
| The terms stay under your control | — | — |
Four steps keep your know-how out
- Turn off training in every AI app. ChatGPT and Claude put the setting under privacy.
- Use business accounts for work. Business and API plans don’t train on your data by default.
- Keep your best know-how private. Your notes, your own systems, and people you trust hold it more safely than a chat box.
- Prefer AI you control. An open model you run sends your work to nobody.
Bottom line: AI companies gain a lot from training on data they don’t own, the penalties are small, and nobody can check internal models. Assume a consumer AI app can train on anything you give it. PageLines runs on open models, and each member gets an AI that works only for them. See how it works.
Common questions
Do AI companies train on copyrighted data?
Yes. AI models were trained on books, articles, and art, often without permission. Courts have mostly ruled that training can be fair use, but keeping pirated copies cost Anthropic $1.5 billion.
Does ChatGPT or Claude train on my chats?
On consumer plans, both train on your chats unless you turn the setting off. Business and API plans don't train on your data by default.
Can anyone check what an AI company trains on?
Not for models a company keeps internal. Outsiders see training data only when a lawsuit forces it into the open, and Europe's disclosure rule covers only models that are released.

