Concept Catalyst tests scrutable AI interfaces for K-12 teachers

A new preprint proposes "scrutable interfaces" — interfaces that link a manipulable knowledge representation to an underlying AI model, so users can change what it produces without understanding how it works — as a way to structure how K-12 teachers use generative AI to build classroom content.

The arXiv paper by Gennie Mansi, Sunni Newton, Roxanne Moore, Meltem Alemdar and Mark Riedl introduces a tool called Concept Catalyst and reports on an exploratory Wizard-of-Oz study with ten middle and high school engineering teachers, whose sessions ran 55 minutes on average. Screen and audio recordings were collected alongside the classroom material the teachers produced.

The authors write that their findings "provide empirical insights about how scrutable interfaces can positively structure teachers' interactions with generative AI models when creating classroom content," and that such interfaces "can help teachers reflect on their teaching practices while improving efficacy, efficiency, and motivation when using AI." They frame the contribution as extending scrutable interfaces in two ways: "to support teachers as users (not just students) and to structure interactions with generative AI models."

The study is exploratory. The interface was tested via Wizard-of-Oz simulation rather than a deployed system, and the sample is ten interviews with engineering teachers.