Tiny VLM cuts Lakota archival OCR errors from 17.1% to 3.9%

Fine-tuning a sub-3B-parameter vision-language model on 35 transcribed pages dropped character error rate on bilingual Lakota archival text from 17.1% to 3.9%, according to a new arXiv preprint by Téa Wright, Claudia Iron Hawk, Katharina von der Wense and colleagues.

Training on only nine pages added just 1.1 percentage points over the full-data run.

The paper frames OCR for Lakota, a North American Indigenous language, as a community-grounded design problem. The authors inspect "four practical facets of low-resource archival OCR that arise in previous work and conversations with community members: constrained training budgets, segmentation noise, scan degradation, and orthographic variance."

The catch is in deployment. "Evaluating under realistic automatic segmentation nearly doubles the CER of the best-performing models," the abstract reports, meaning that in archives where line and word boundaries are extracted by a pipeline rather than a human, the headline numbers do not survive the handoff. The authors close by calling for language technologies "grounded in the workflows and constraints of ongoing revitalization efforts."