2026-05-11T04:37:37.051000+00:00 · user_written
how many tokens is annas archive ? What would an llm trained on it look like ?
Based on a real day; names and selected personal details have been changed or omitted.
Anna's Archive LLM Estimate
Earlier conversation
2026-05-11T04:37:37.461304+00:00 · assistant_claim
I’ll treat this as two questions: rough corpus-scale math, then what capabilities/risks a model trained on that mix would have. The exact token count is not publicly knowable, so I’ll bound it from available size/count signals.
2026-05-11T04:37:41.537636+00:00 · assistant_claim
Anna’s Archive is roughly single-digit to low-double-digit trillions of text tokens, not hundreds of trillions, unless you count raw compressed PDFs/images naively.
Public counts put it around ~63.6M books + ~95.7M papers, with torrent listings around ~1.1 PB total. Those counts include many PDFs/scans/duplicates/metadata records, so storage size is a bad proxy for LLM tokens. citeturn709325search1
My rough estimate:
| Component | Count | Avg tokens/item assumption | Token estimate |
|---|---|---|---|
| Books | ~64M | 50k–120k | 3.2T–7.7T |
| Papers | ~96M | 4k–9k | 0.4T–0.9T |
| OCR/scanned/long-tail weirdness | — | — | +0.5T–5T |
| Total, messy raw extracted text | — | — | ~4T–14T tokens |
| After dedupe/filtering | — | — | maybe ~2T–8T useful tokens |
If you trained an LLM only on Anna’s Archive, it would probably feel like:
Extremely bookish and academic. It would know a lot of textbooks, scholarly phrasing, humanities, science papers, old fiction, reference works, manuals, monographs, translations, and obscure long-tail subjects.
Less internet-native. Compared with a web-trained model, it would be weaker at forum slang, recent memes, product docs, GitHub issues, UI patterns, StackOverflow-style debugging, up-to-date cultural/current events, and everyday conversational examples.
Strong long-form prose. It would likely write essays, summaries, historical explanations, literary analysis, academic arguments, and “chapter-like” text well.
Weak instruction-following unless fine-tuned. Raw pretraining on books/papers does not automatically make a ChatGPT-style assistant. You’d still need instruction tuning, preference tuning, tool-use data, safety training, etc.
Potentially good at deep domain knowledge, but not necessarily factual retrieval. It might absorb lots of scientific and scholarly patterns, but it would still hallucinate unless paired with retrieval. “Trained on papers” does not mean “can cite the exact paper correctly.”
Noisy and legally radioactive. The corpus reportedly indexes/links large amounts of copyrighted books and papers from shadow-library sources, so training on it would create major copyright, provenance, memorization, and dataset-governance problems. citeturn709325search1turn709325search8
The most interesting version is not “train a model only on Anna’s Archive.” It is: use a legally clean, deduped, book-heavy corpus as a knowledge/prose layer inside a broader web+code+conversation+tool-use training mix. That would probably produce a model that feels more like “well-read graduate student” than “Reddit/GitHub-native agent.”