FineBooks scores 14 open OCR models on historical books

Hugging Face and EleutherAI have put a concrete number on whether today's open OCR models can reliably re-read historical books at scale. Their new FineBooks leaderboard scores 14 open-weight models against 2,165 expert-transcribed pages drawn from the Biodiversity Heritage Library, an archive of over 300,000 digitized natural-history items totalling more than 64 million pages.

The top score belongs to dots.mocr, a 3B model, at 97.6% reading accuracy. Behind it, OvisOCR2 at 0.9B hit 96.9% and PaddleOCR-VL-1.6 at 1B hit 96.1%, both at under half a dollar per thousand pages. olmOCR-2 at 8.3B scored 95.7%. The pattern is the striking part: several sub-2B models are effectively matching the bigger ones on this workload while costing a fraction to run.

The team splits the "is it good enough" question into three concrete audiences instead of giving one verdict. For LLM training corpora, the answer is yes: leading models beat legacy library OCR pipelines and re-processing whole collections is economically feasible, with the write-up citing a Talkie experiment in which models trained on OCR-derived text learned at 30% of the efficiency of models trained on hand transcriptions. For libraries considering ripping out legacy pipelines, it is a maybe, because the new models output Markdown or plain text rather than the ALTO XML that library systems rely on for word-level coordinates. For scholarly transcription, it is a no: the models silently modernize archaic letterforms such as the long s and various ligatures, which is fine for a language model but wrong for work that requires faithful character preservation.

The scope is narrow and the post says so. All 2,165 ground-truth pages come from six book volumes in English, French, German, and Latin, all in Antiqua-family typefaces. Fraktur, non-Latin scripts, handwriting, multi-column layouts, and newspapers, magazines, and archival materials are all outside the benchmark, so top-line accuracy may not carry over to those. AI Weekly logged the leaderboard as shared by two of the researchers we track, which fits its natural audience: builders of open training corpora rather than scanning-shop operators.

Next up, according to the post, is running the winning models across the roughly 200,000 public-domain BHL items and releasing the resulting text, datasets, page classifiers, and processing pipelines openly. If the accuracy holds outside the small benchmark set, a large chunk of public-domain scientific literature stops being a pile of low-quality legacy OCR and starts being useful training and search data for anyone building open models.