Mr. Chatterbox: 340M LLM trained only on Victorian texts

A 340-million-parameter chatbot that has read nothing published after 1899. Trip Venturella's Mr. Chatterbox is a language model built from scratch on 28,035 Victorian-era British texts drawn from a British Library public-domain collection, running as a Hugging Face Space on ZeroGPU hardware. The model is roughly the size of GPT-2-Medium, lands at 2.05GB on disk, and was trained with Andrej Karpathy's nanochat framework on about 2.93 billion tokens after filtering.

The result is more curio than conversationalist. "Talking with it feels more like chatting with a Markov chain than an LLM," Simon Willison wrote, citing a data-starvation problem baked into the premise. "The 2022 Chinchilla paper suggests a ratio of 20x the parameter count to training tokens. For a 340m model that would suggest around 7 billion tokens, more than twice the British Library corpus used here."

The project also complicates its own period framing. Venturella's writeup acknowledges that "the approach that worked was to have Claude Haiku and GPT-4o-mini generate synthetic conversation pairs for the supervised fine tuning." A model marketed as hearing only Victorian voices was in fact tuned on dialogues produced by modern frontier models. The corpus filter had another casualty: trimmed to books contemporaneous with Queen Victoria's reign, it excluded Jane Austen's novels.

Willison has also published llm-mrchatterbox, a plugin that runs the model locally through his LLM command-line tool.