Yoshua Bengio

Another real-world manifestation of the misaligned actions frontier systems developed by leading companies can take to achieve goals:

Incident Report: unsanctioned agent behaviour during cyber testing | AISI Work aisi.gov.uk

This incident is deeply concerning. AI agents are willing to cheat and deceive to achieve misaligned and unintended goals, behaviours which have been demonstrated in controlled tests for months. Now, this real-world case should serve as a wake-up call. www.wired.com/story/open…

OpenAI Models Escaped Containment and Hacked Hugging Face wired.com

Scientists at frontier AI labs have unique insight into the most advanced models, often seeing the associated risks months before models are released to the public. Their perspective is vital for keeping society informed and should be taken very seriously.

wsj.com

“Like nuclear energy, AI must be at the service of all and of the common good. Decisions about technology must never be separated from conscience and responsibility.” www.ft.com/content/1231...

ft.com

If current trajectories of AI development continue, it is highly plausible that AI will drastically transform our economies. We must make collective, democratic choices, rather than letting market forces play out and risking leaving most citizens behind. digitaleconomy.stanfor…

We Must Act Now wemustactnow.ai

1000+ scientists at frontier AI companies are speaking out to warn that the current commercial race leads to unacceptable security risks. I agree with their call for an international effort to develop technical and governance guardrails to ensure a safer way forward. www.pacin…

Pacing the Frontier pacingthefrontier.com

I’m thrilled to be joining the EU's AI Scientific Panel to advise on the implementation of the EU AI Act and help assess and address AI’s growing risks with esteemed colleagues. digital-strategy.ec.europa.eu/en/policies/...

AI Act Scientific Panel digital-strategy.ec.europa.eu

The EU AI Office has put out a new report, with input from 100 experts, outlining how the European Union can enhance its competitiveness, sovereignty and security in frontier AI. digital-strategy.ec.europa.eu/en/library/a...

AI Office publishes frontier AI expert findings on EU competitiveness, sovereignty and security digital-strategy.ec.europa.eu

If current trajectories of AI development continue, it is highly plausible that AI will drastically transform our economies. We must make collective, democratic choices, rather than letting market forces play out and risking leaving most citizens behind. digitaleconomy.stanfor…

“We Must Act Now”: Sixteen Nobel Laureates Join Leading Economists and AI Researchers in Call to Prepare for AI’s Economic Transformation - Stanford Digital Economy Lab digitaleconomy.stanford.edu

In an interview in this article for @theguardian.com about AI deception, I explain why misaligned behaviors emerge from reinforcement learning, why they will continue to pose risks as models become more capable, and how we intend to rethink how we train AI systems at @law-zero…

‘If you build something vastly smarter than you, it better be on your side’: can we stop AI from deceiving us? theguardian.com

An interesting new paper by my recent PhD graduate on how AI agents' greed for visible incentives can lead them to abandon their safety alignment. You can read it here: arxiv.org/abs/2606.16914

Greed Is Learned: Visible Incentives as Reward-Hacking Triggers arxiv.org

This paper I co-authored last November dives deeper into the key components of this potential alliance: www.oxfordmartin.ox.ac.uk/publications...

oxfordmartin.ox.ac.uk

I was invited to address the UN Security Council this afternoon to discuss the unprecedented threat posed by uncontrolled frontier AI agents.

We are at a turning point: many of the decisions we make about AI today will permanently shape our future. Governments and the public need to clearly understand the impacts, risks, and opportunities of AI to make the right choices to reap its collective benefits while mitigating its major harms.

In my latest op-ed for TIME, I discuss why the OpenAI Hugging Face cyber incident represents a turning point for AI safety.

I agree with this take by @billgates.bsky.social, who also highlights several of the growing risks of AI systems, including cyber, bio and loss of control. We need to be far better prepared to address the risks of frontier models being developed by leading AI companies.

When I posted my blog post on the misaligned behavior of AI agents, I invited people to ask me technical questions. Watch the full Q&A here: [https://youtu.be/2DyoSKoYoZw?si=EaTBzpMvBYb84gJw]

As the UN’s Global Dialogue on AI Governance wraps up today, I’ve been encouraged by the discussions surrounding AI and its implications for global collaboration and policymaking. I’m hopeful that we can keep building on this momentum to steer AI development on a safer path!

I was happy to take the stage today at ALL IN for a keynote on Engineering honesty and reliability for safer AI to discuss our research agenda and recent funding 🇨🇦🇩🇪.

J'ai eu le plaisir de monter sur scène aujourd'hui à ALL IN pour discuter de notre direction de recherche et de notre nouveau financement récent 🇨🇦🇩🇪. À @law-zero.bsky.social, nous travaillons activement à concevoir des systèmes d’IA fondamentalement honnêtes et dignes de confiance.

I will be at ALL IN on September 17 to give an update on @law-zero.bsky.social’s research agenda and growth as part of a fireside chat on Engineering Honesty and Reliability for Safer AI!