Pretraining Language Models on Historical Text

Shared on Bluesky by 3 AI experts

• Ted Underwood @tedunderwood.com: For fans of Talkie-1930 and all the methodological questions raised by historical models, here's a new entrant into the field, TypewriterLM,… →
• G Grace @gracekind.net amplified

  Ted Underwood @tedunderwood.com

  For fans of Talkie-1930 and all the methodological questions raised by historical models, here's a new entrant into the field, TypewriterLM, trained up to 1913. Corpus, instruction-tuning datasets, and event dataset are …

  View on Bluesky →
• A AI Firehose @ai-firehose.column.social: TYPEWRITERLM, a new model trained on 54 billion historical tokens before 1913, enhances understanding of the past while tackling data qualit… →