defeating LLM nondeterminism essay
Defeating Nondeterminism in LLM Inference
4 experts are actively discussing the implications.
2 experts
1 community
1 sources clustered
“And since this thread is likely to attract some "well, actually..." I will also park, here, this article from Murati's "Thinky Machines" group on (while deterministic in theory) what it actually takes to get deterministic output, in practice, involving cont…”
“thinkingmachines.ai/blog/defeati... from Horace He”
4 experts discussed this · 12 posts
Adverb: Yeah, he's a fraud. Could pick apart predictions, arguments, technical knowledge but his repping for the companies he supposedly hates is the cleanest of giveaways.
Adverb: He thinks LLMs can't be deterministic which like, listening to someone who knows their subject -- even if not with terms like greedy sampling & batch invariant kernels -- could save people's time.
Adverb: I actually took like 5 minutes to look over one of his posts & stopped after the first embarrassing technical inaccuracy.
Open the full discussion →