Leo Boytsov

🧵There is a fundamental issue with reference-based LLM-judges. People implicitly assume a reference-based judge behaves like: score=f(candidate,reference)score=f(candidate,reference) However, the actual behavior is closer to: score=f(candidate,reference,parametric knowledge,pr…

Judging Against the Reference: Uncovering Knowledge-Driven Failures in LLM-Judges on QA Evaluation arxiv.org

On the negative side, it is often done incorrectly and only adds more technical debt. Instead of solving systemic issues first, people are just slapping an AI band-aid in the hope to stop a gushing artery. 🟦 searchivarius.org/blog/llm-ski...

searchivarius.org

This meticulous study delves into the intricate tapestry of lexical biases in LLM-assisted academic writing. It underscores a nuanced interplay of preferred words, unveiling how they have dramatically enhanced and reshaped the realm of science. www.linkedin.com/feed/update/...

LLMs Favorite Words in Academic Literature Dramatically Increase Since GPT Era | Alex Glynn posted on the topic | LinkedIn linkedin.com

🔹Now long-running asynchronous agents expose a weakness in GRPO, so this paper brings the critic back, but with several engineering fixes. 🟦 www.linkedin.com/posts/ravid-...

Just read the new paper from Tsinghua/Z.AI on async RL for agents (arXiv:2607.07508). It comes several weeks after the release of GLM-5.2, in which they mentioned that they use a critic instead of… | Ravid Shwartz Ziv linkedin.com

🧵LLM Skills: A New Source of Technical Debt. I have some thoughts related to the recent craze to automate everything with agentic skills. On a plus side, I think it can be very useful when done correctly.↩️

Interestingly we went full-circle in RL for LLMs and LLM agents: 🔹Initially, OpenAI (and some others) used RLHF with PPO, which requires training a critic (reward) model. 🔹Then, researchers moved from PPO with a critic to critic-free GRPO because critics were expensive and unstable. ↩️

Fun fact: Two some of the most influential statistical NLP papers were authored by Brown et al in 1990 & 2020 1990 Statistical Approach to Machine Translation 2020 Language Models are Few-Shot Learners *) It is not the same Brown **) I believe author names in IBM papers were ordered alphabetically