David Manheim

In some discussions, sure, I agree. (And not coincidently, most of the systems don't have robust control or oversight: arxiv.org/abs/2507.03525) But yet again, the motives of speakers don't make their claims true or false.

Limits of Safe AI Deployment: Differentiating Oversight and Control arxiv.org

That's one of several varieties, yes. (And it's not at all coincidental that the formalization came from people who were working with Eliezer.) arxiv.org/abs/1803.04585

Categorizing Variants of Goodhart's Law arxiv.org

The complacency is not working out well, and costs for pranks / hacks / ruining lives with hacks will keep dropping at least in the intermediate term. As I wrote last year:

Security Complacency Meets Frontier AI: The Coming Collapse of ‘Secure by Apathy’ — LessWrong lesswrong.com

Yeah, I strongly agree about the need for agreed-upon standards for evaluations, hence my work with ISO, and working on projects like this: evals-consensus.ai, as well as clearer rules for audits, hence my paper here: link.springer.com/article/10.1...

AI Evaluation Consensus Statement evals-consensus.ai

Yeah, I strongly agree about the need for agreed-upon standards for evaluations, hence my work with ISO, and working on projects like this: evals-consensus.ai, as well as clearer rules for audits, hence my paper here: link.springer.com/article/10.1...

The necessity of AI audit standards boards - AI & SOCIETY link.springer.com

I was really happy to appear on the @futureoflife.org podcast this week to talk about @evals-consensus.ai and the challenges of evaluations of AI systems, along with discussions of Goodhart's law, AIxBiosecurity, AI persuasion, forecasting, and human oversight of AI.

Why AI Evaluations Are Broken and How to Fix Them (with David Manheim) youtu.be

I keep seeing claims AI companies will get rich, or won't make any money, or that open source models will be cheaper and undercut prices, and similar claims that ignore the simple fact that the economic pricing power is all about compute and token generation costs. davidmanhei…

The Future Economics of LLMs — Interactive Companion davidmanheim.com

New post: Huge technological revolutions aren't usually positive for those living through them - and this bodes poorly for AI even if it is a normal technology. davidmanheim.substack.com/p/if-ai-is-n...

If AI is normal technology, history is not reassuring. davidmanheim.substack.com

Over on the bird site, Anthropic employees are using their slop machine that 'can't think' to solve longstanding mathematical open questions which have eluded humans for close to a century.

The other side of the anthropomorphic fallacy discussing LLMs is the utensilomorphic fallacy, uncritically claiming AI agents are like tools, e.g. that they only do what users ask them to or allow, and that consequences must be misuse or errors.

Public opposition to data centers somewhat matches the vibes of the people worried about AI safety, but actually stopping their construction isn't meaningfully helpful for AI safety, it's at best irrelevant.

GPT-5.5 seems to do more 'hacking per dollar' ...Mythos does more 'hacking per token'" - @peterwildeford.bsky.social Thinking about risks from bad actor's access to scaled models, the costs involved here are still absolutely tiny compared to other attack modes.

"Propensity to evade monitoring" - ? It sounds like AI companies are insisting that their models are just tools, to evade responsibility for creating something legitimately anthropomorphic that has goals the companies don't want.

Fantastic story from Stefan Larsson @lund-university.bsky.social giving the keynote today at #AIES2026 He thought he was done working on copyright law, and had moved on to AI issues - then found out that *his book on copyright* had been pirated to train LLMs via the Anthropic class-action lawsuit.