Anthropic vs Alibaba: Why AI Distillation Isn't a Crime

🚨 This post contains opinions of the author in hypothetical scenarios and should be taken with a grain of salt.

When AI labs clash in public, it’s easy to lose sight of what the disagreement is actually about. Stripping away the rhetoric helps clarify the underlying technical and legal questions.

The Anthropic vs Alibaba accusation

Anthropic has alleged that Alibaba distilled Claude to train one of its models. The BBC covered the dispute. The short version: Anthropic believes Alibaba's team used outputs from Claude to help train its own models, which would break Anthropic's terms of service.

Under Section 3 of Anthropic's terms of service is clearly states that the user is forbidden to:

To develop any products or services that compete with our Services, including to develop or train any artificial intelligence or machine learning algorithms or models or resell the Services.

While Anthropic may have a case on the grounds of ToS violation, there are some topics that are worth discussing.

Distillation is not an attack

When AI companies report that a rival distilled their model, the narrative often sounds like an intellectual property heist or a technical breach. But distillation is neither a cyberattack nor an exploit. It is one of the oldest and most practical techniques in modern machine learning.

Check out this token chaser video compare qwen vanilla with a opus distilled version.

According to Professor Gustavo Galegale (coordenador do MBA em Gestão de TI da FIAP), current AI regulations (including the European Union’s AI Act) do not categorically identify model distillation as a prohibited practice. The EU AI Act prohibits specific uses of AI considered to pose unacceptable risks, including certain forms of manipulation, social scoring, biometric categorization, emotion recognition, and real-time biometric identification. It does not prohibit distillation simply because one model is trained to approximate the behavior or outputs of another.

From a governance perspective, Galegale argues that organizations should establish clearer rules for the use of model outputs. "AI providers need to state precisely what customers may and may not do with generated outputs". At the same time, companies developing models should implement technical controls, monitoring mechanisms, and auditable processes instead of relying exclusively on broad contractual clause.

What distillation actually means

The core idea is simple. You have a large, capable model and a smaller model that you want to improve.

The big model is the teacher. The small model is the student. You feed the same inputs to both, and you train the student to reproduce what the teacher produces. Over many examples, the student starts to pick up the teacher's behavior and style.

The concept predates modern LLMs, it came from a 2015 paper by Geoffrey Hinton et al., who called it "knowledge distillation." The name makes sense: you're boiling a large model down to its useful essence.

⚠️ It is important to point out that distillation is not guaranteed to reproduce the teacher's capabilities. It can transfer behavior surprisingly well, but there is a wall related to the student's model inherent capabilities.

Two flavors that get confused

There are two versions of this, and mixing them up causes half the arguments online.

• Classic distillation uses the teacher's internal probability distribution, sometimes called soft labels or logits. The student sees not just the final answer but how confident the teacher was across all options. This is richer signal, and it requires real access to the model's internals.

• Output distillation, sometimes called data distillation, just uses the teacher's text outputs. You prompt the big model, collect its answers, and fine-tune your smaller model on those question-answer pairs. No internal access needed. Anyone with an API key can do this.

When companies accuse rivals of distillation, they almost always mean the second kind. Someone hit their API, harvested a pile of responses, and trained on them.

Google was going to sell you distillation

Here's the part that makes the outrage look inconsistent. Google was reportedly preparing a Gemini distillation service, a managed product to help you distill a large Gemini model into a smaller one.

A Google Cloud page describing the service was spotted, and the community discussed it on r/LocalLLaMA. As of today, that page appears to be archived and its release status is unclear.

Sit with that for a second. One of the biggest labs in the world was building a productized, sanctioned way to distill its own flagship model. That demonstrates that even frontier labs view distillation as a valuable engineering technique when performed under authorized conditions.

ToS violation is not a crime

If Anthropic's allegations are true, the dispute is best understood as a breach of contract, not a criminal offense. When a customer breaches a ToS, they may expose themselves to penalties estabilshed by contract laws.

But that does not, by itself, amount to crime. That's a commercial dispute for a civil court, not evidence of criminal wrongdoing.

Ultimately, Galegale cautions against treating a commercial disagreement as proof of criminal wrongdoing. “These allegations may raise important questions involving contracts, competition, intellectual property, and cybersecurity. But each of those questions must be evaluated according to its own legal requirements. A Terms-of-Service violation cannot, on its own, substitute for evidence of a crime.”

The irony is that Anthropic itself is defending multiple copyright lawsuits over the data used to train Claude. In one of those cases, the company agreed to pay $1.5 billion to settle a class-action lawsuit brought by authors whose books had been copied into Anthropic's training corpus. A federal judge approved the settlement in July 2026, making it the largest known copyright recovery in U.S. history. The plaintiffs argue that Claude was trained on copyrighted books without permission and copying copyrighted works into a training corpus constitutes copyright infringement, regardless of whether the model reproduces those works verbatim (source)

A dangerous precedent

But there is another aspect of the Anthropics ToS that I find far more troubling.

To develop any products or services that compete with our Services, including to develop or train any artificial intelligence or machine learning algorithms or models or resell the Services.

Let's let that sink for a minute and image these hypothetical scenarios:

• What if I use claude to implement a algorithm that make local inference radically faster?
• What if run a company that rank images based on some criteria and anthropic starts providing that service?
• What if I build a better tokenizer after experimenting with Claude?
• What if I use Claude to write code for a vector database, a compiler, or an inference engine that later becomes part of an AI product?
• What if I ask Claude to explain a machine learning paper, and that knowledge ultimately helps me build a competing model?

Where does legitimate use end and "developing a competing AI service" begin? Is the ownership of the code claude outputs really mine or it can shift depending on how broadly those terms are interpreted.

Sure, mostly people might say we should expect me to be reasonable. That intent and scale matters: running a prompt that helps you build a tool isn't the same as automatically scraping of millions of synthetic QA pairs specifically to fine-tune a direct clone model is. This is true. But would you realy count on that subjectivity when a multi trillion dollar industry is at stake ?

As AI becomes a general-purpose tool for software engineering and becomes more embedded within the software itself, drawing a clean line between "using AI to build software" and "using AI to build a competing AI product" will become increasingly difficult. That may end up being one of the most important legal questions of the next decade.

FAQ

What is model distillation, and why is it so controversial?

Distillation is a machine learning technique where a smaller model (the "student") is trained to mimic the outputs or internal behaviors of a larger, more capable model (the "teacher"). While it is a standard engineering method for making AI models faster and cheaper to run, it becomes controversial when companies use outputs from a competitor’s proprietary API to train their own commercial models, often violating the provider's Terms of Service.

Is distillation illegal?

No. Distillation is neither a cyberattack nor a criminal act: it uses standard API requests to gather text outputs. When done without permission, it represents a civil breach of contract (a ToS violation) rather than a criminal offense or a technical security breach.

Why are frontier AI labs accused of double standards regarding training data?

Frontier labs often invoke "fair use" to justify scraping vast amounts of copyrighted material from the web to train their flagship models, while simultaneously using restrictive Terms of Service to forbid others from using their model outputs for training. Critics point out the irony in labs defending web-scraping in court while aggressive about output-harvesting by competitors.

How could broad "Do Not Compete" clauses affect ordinary developers?

As AI tools become deeply integrated into software development, terms that forbid using an AI to "build a competing product or service" create significant legal ambiguity. If an engineer uses an AI assistant to help write a compiler, database, or optimization algorithm, and the AI provider later releases a similar tool, broad ToS language leaves it unclear whether the developer's work constitutes a contract violation.