The anatomy of an AI-supported academic article is becoming more apparent. I desk reviewed ~70 manuscripts this week for our journal. Reckon 20ish had significant AI assistance. On top of hundreds of others recently I'm starting to see patterns. Are other editors are seeing the same or similar? 1/
What's most upsetting me as a journal editor is that many papers we receive where AI use is disclosed for editing and language refinement actually appear to have been ruined rather than improved by AI. We get what seem like promising underpinning studies, but totally obscured by over-complex text.
AI-assisted text is deadly for academic journal editors and reviewers. I'm detecting a lot more of it recently as an editor. It's still there in the "Frankencitations" and those little linguistic "tells" like "AI is not the cause. It's the symptom of an underlying reality." But it's more, too. 1/
The evidence about AI in education remains terrible as a basis for policy or practice. As schools in some places come back from summer next week (oh Scotland it's too early), here's a wee roundup on just how truly woefully AIED research has stood up and why educators shouldn't call on it 1/
I keep hearing that AI is going to transform research and publishing. Then in my journal editing role I come across the below - an instruction from AI to an academic author to add in missing scholarly references. Claude is telling the named author how to slip this past the journal editor 🤬
Good god the really tepid pauses on AI in primary schools in NY and LA have wound up the LinkedInerati so badly they're already dropping a million Claude-written complaints and counter-proposals. Small children do not have to be your tech test subjects, guys. Get. A. Grip.
Wee bit of advice for anyone writing up research about AI in education. Journals like the one I edit are drowning in it. Reviewers aren't accepting invitations. Most manuscripts offer *nothing* new. Yes it's a big current topic BUT it's not inevitably important as a research subject...
I do think access to LLMs has sent some academic writers a bit doolally. At our journal I desk rejected a paper recently because it was just *impossible* to read. There seemed to be an underpinning study hiding in it, but I reckoned the (disclosed) use of AI for "editing only" wrecked it...
AI is a huge problem for education policymakers. There's no good evidence to go by, all countries feel like they must act on it, and it's a source of huge uncertainty. From a research perspective, we're just getting a view of how different national policy centres are dealing with AI.
Maybe if they had actually read what my work says about education, data and the edtech industry, instead of relying on some kind of AI summary of it, they would recognize that I - as an academic "who lives in the literature" - am about the last person to want to convert it into autosummary podcasts
