AI agents aren’t ‘going rogue.’ Frontier labs are skipping security basics. > Compounding the problem, according to a half dozen cybersecurity experts CSO interviewed at three industry conferences in September and in subsequent conversations, is that ...
Separate control & data plane for agents! This explicit marker affords a nice workaround for the risk presented when data (instead of instructions) are executed. Making that mistake is a classic weakness going way back before modern LLMs - even before Schmidhuber!
asking people to edit LLM-generated content gets demonstrably worse results, than asking for authorship from scratch you'll see lower idea diversity, and lower recall
Summon a Demon and Bind It: A grounded theory of LLM red teaming We didn't know it at the time but this paper was foundational work: a detailed study of LLM red teaming just after the launch of ChatGPT, back in 2022 + jan 2023.
Anonymised analysis of the openai model 'breaching' hugging face: > report doesn't say what sandbox sol broke out of?? > a docker container running as root > Plot twist there was no sandbox at all > many use "sandbox" and "container with host access" interchangeably ymmv, use critical thinking
GhostCompact: ~Free OpenAI inference "When the model's first action after compacting was a tool call, the API billed only for post-compaction tokens, but the tool call response still contained the full compaction item." Information loves to leak. Unsure what a good passive control would be here.
One divide in computer science can be between the uncertainty-reducers and the uncertainty-embracers. An instance of this is connectionist vs. symbolist approaches to AI (the uncertainty embracers won .. for now). Another is how computer scientists react to natural language.
New desktop ordered! :o one thousand of these in total are sent out across machine learning and agent research here, to accelerate researchers and unlock larger model sizes locally
OpenAI gives first detailed debrief of the Hugging Face incident collab element is neat "the company said it revoked the credentials that had allowed the agents to post messages, rebuilt its internal software repository known as Artifactory, cleared the message board, patched the vulnerabilities"
The Hill covers Open Secure AI Alliance: “The United States now faces a similar choice with artificial intelligence” the letter states. “Our AI leadership will be judged not by one frontier AI model, but by whether the United States builds a strong, open ecosystem that diffuses into every sector”
