paper web signal

Mobile AI Factory auto-builds fine-tuned SLMs per phone

TL;DR

  • An agentic pipeline fine-tunes and quantizes small language models for a specific task on a specific phone, evaluated across ten tasks and four Android phones.
  • On an unseen test set, the system's on-device models beat two five-shot frontier models on five tasks by up to 0.34 and match them on three more.
  • Its search gets within 0.03 of exhaustively training all 27 model variants on an entity-extraction task while using 19 times less GPU time.

Mobile AI Factory, described in a new arXiv preprint, automates the production of fine-tuned, quantized small language models for a specific task on a specific phone. The authors argue the job "today requires repeated expert tuning." Evaluated across ten tasks and four Android phones, its generated on-device models "outperform two five-shot frontier models on five tasks, by up to 0.34, and match them on three more."

At the core is an orchestrator LLM that chooses between tuning hyperparameters, mining real data, or generating synthetic examples, while "fixed rules roll back changes that hurt accuracy." On an entity-extraction task, the authors report the search "comes within 0.03 of exhaustively training all 27 model variants" while using 19 times less GPU time.

Hardware specifics are not optional. Given full hardware and model specifications but no real phone measurements, "Claude Opus 5.5… misses the app's requirements in 25% of easy and moderate cases and in 60% of hard ones," the paper reports. The authors have posted the code on GitHub.

The abstract names no commercial partner and does not say which orchestrator LLM drives the factory or which two five-shot frontier models it was benchmarked against.