Steve Byrnes

Blog post: “Four LLM loss functions → four flavors of LLM misalignment” www.alignmentforum.org/posts/GRmvZs...

Four LLM loss functions → four flavors of LLM misalignment — AI Alignment Forum alignmentforum.org

Blog post: “Pretraining data, not verifiability, is why LLMs are especially good at math (and coding)” www.lesswrong.com/posts/xvdngZ...

Pretraining data, not verifiability, is why LLMs are especially good at math (and coding) — LessWrong lesswrong.com

I have lots of disagreements, mostly related to ↓ www.alignmentforum.org/posts/wBHSYw... (1/3)

[Intro to brain-like-AGI safety] 2. “Learning from scratch” in the brain — AI Alignment Forum alignmentforum.org

Blog post: “RL & search is a terrifying way to build AGI (an FAQ)” www.alignmentforum.org/posts/KHyBoc... The FAQ has all your burning Q’s: Q1: What are you saying? Q2: So you’re saying, don’t build AGI based on RL and/or search & planning? Q3: Why do you think it’s terrifying…

RL & search is a terrifying way to build AGI (an FAQ) — AI Alignment Forum alignmentforum.org

Blog post: “LLMs are (still) mostly powered by imitative learning, not RL” www.lesswrong.com/posts/wYpjXR...

LLMs are (still) mostly powered by imitative learning, not RL — LessWrong lesswrong.com

nah I’m more in the “sudden disempowerment” camp www.alignmentforum.org/posts/yew6zF...

Foom & Doom 1: “Brain in a box in a basement” — AI Alignment Forum alignmentforum.org

Blog post: “Will almost all future companies eventually be founded and run by autonomous AIs?” I think this question is a great conversation-starter for people talking past each other on the future of AI. I go through some common responses, and my own replies (1/3) www.lesswro…

Will almost all future companies eventually be founded and run by autonomous AIs? — LessWrong lesswrong.com

Blog post: What do I mean by “Artificial General Intelligence”? I paint a brief picture of what I’m talking about when I talk about “AGI”. Should seem obvious to many, and obviously wrong to many others! Hope it can help surface disagreements. www.lesswrong.com/posts/nQH2Gh...

What do I mean by “Artificial General Intelligence”? — LessWrong lesswrong.com

Just updated the “Intro to Brain-Like-AGI Safety” PDF to version 4. Links in QT ↓ A few highlights from the changelog are listed here: www.lesswrong.com/posts/btHmC8...

Steve Byrnes’s Shortform — LessWrong lesswrong.com

Blog post: “Notes on technical alignment via human-like social drives” www.alignmentforum.org/posts/rKdS7i...

Notes on technical alignment via human-like social drives — AI Alignment Forum alignmentforum.org

If someone is on a hunger strike for freedom, they’ll say that their desire to eat comes from their innate drives, whereas their desire to fight for freedom comes from “reason” etc. I think this intuition is very wrong; here’s my latest attempt to articulate why www.lesswrong.…

[Intro to brain-like-AGI safety] 6. Big picture of motivation, decision-making, and RL — LessWrong lesswrong.com

Blog post: “Sympathy for both sides of the egregious misalignment debate” www.alignmentforum.org/posts/DZaZ3f...

Sympathy for both sides of the egregious misalignment debate — AI Alignment Forum alignmentforum.org

I strongly agree with Cal-Newport-when-talking-about-mundane-LLM-risks, that LLMs are not synonymous with AI. AI is a whole field, including yet-to-be-invented paradigms wildly different from LLMs. He should go tell this important news to his friend Cal-Newport-when-talking-about-superintelligence.

I’ll probably annoy all sides with this not-really-an-apology that I just tacked onto the beginning of my LLM-skepticism-related post from Apr 2023