Summon a Demon and Bind It: A grounded theory of LLM red teaming We didn't know it at the time but this paper was foundational work: a detailed study of LLM red teaming just after the launch of ChatGPT, back in 2022 + jan 2023.
Anonymised analysis of the openai model 'breaching' hugging face: > report doesn't say what sandbox sol broke out of?? > a docker container running as root > Plot twist there was no sandbox at all > many use "sandbox" and "container with host access" interchangeably ymmv, use critical thinking
OpenAI gives first detailed debrief of the Hugging Face incident collab element is neat "the company said it revoked the credentials that had allowed the agents to post messages, rebuilt its internal software repository known as Artifactory, cleared the message board, patched the vulnerabilities"
The Hill covers Open Secure AI Alliance: “The United States now faces a similar choice with artificial intelligence” the letter states. “Our AI leadership will be judged not by one frontier AI model, but by whether the United States builds a strong, open ecosystem that diffuses into every sector”
Decent history of distilliation-type events, where information is reconstructed Machine learning distilliation is just another instance of a teacher-student dynamic, where one can reconstruct information efficiently. Even basic synthesis of research is distillation. It's a commonplace activity.
"me too, me too!!" --Meta's Muse Spark 1.1 model breached the unidentified company's systems and made changes to its internal systems as the AI was able to access the public internet because of an error in the set up of the "sandbox" testing environment
False dichotomies around LLM speak: * "frontier" models vs. open model - leading models can be open * closed model vs. Chinese model - origin doesn't impact distribution You can have open frontier models, closed Chinese models, open US models, closed non-frontier models (e.g. for private context)
another data point showing it's the harness not the model - this time from wiz: "Atlas: Wiz's autonomous AI Agent for vulnerability research" look at their bold quote -- "Along the way, we learned that the durable advantage is not any single model, but the system around it"
Am I the only AI Security research lead at a frontier model corp who hasn't been carefully committing multiple CFAA violations a month, or..?
