⚡ 2 h early
OpenAI Astra technique security concerns
OpenAI Technique in ‘Astra’ Model Sparks Security Concerns
5 experts across 4 network communities independently surfaced this.
Why this matches
Questions & unknowns reaction
1 attributable expert contribution
· Dr. Émile P. Torres
“Does anyone have a copy of this article they could share with me? www.theinformation.com/articles/sec...” evidence ↗
5 experts
4 communities
1 sources clustered
How the network is reacting
Experts are approaching this through 3 distinct lenses.
Multiple readings
Concern & critique
1 expertRisks, limits and unintended consequences.
“> joins big paper about not training models to think in nonsense > their AIs commit felonies against HuggingFace > actually, trained AIs to think in nonsense > top danger level for hacking Let's just release it anyways! Nice defection OpenAI! 🤗 www.theinfor…”
Opportunity & adoption
1 expertNew capabilities, benefits and practical upside.
“Astra used looped transformers www.theinformation.com/articles/sec...”
Questions & unknowns
1 expertWhat remains unresolved or contested.
“Does anyone have a copy of this article they could share with me? www.theinformation.com/articles/sec...”
3 experts discussed this · 8 posts
Tim Kellogg: Astra used looped transformers www.theinformation.com/articles/sec...
Tim Kellogg: safety concerns — looped transformers skip converting their latent space into text, which effectively creates non-text portions of CoT reasoning which, if you can’t read it, that’s harder to monitor
Tim Kellogg: however, i their earlier update, they explain that they can indeed see all of the CoT. So they have some sort of tooling that effectively turns it back into something human readable, like text bsky…
Open the full discussion →