1/5 The case for diffusion language models: "A lot of the [early] gains in language modeling performance have come from scaling pre-training...[the training algorithm] was designed to be very parallelizable across GPUs": Volodymyr Kuleshov of @cornelluniversity.bsky.social at the Simons Institute
1/3 AI and Its Discontents: @ucberkeleyofficial.bsky.social's Jitendra Malik talked of the "rumblings of dissent,", when it comes to the current approach of scaling up frontier multimodal models, at the Simons Institute workshop on Topics in Intelligence: World Models and Social Reasoning
1/2 The Return of RNNs. "AI is still lacking a good memory mechanism...We are in this era of 'attention is all you need' but I don't think that's going to cut it," said @phillipisola.bsky.social of @mit.edu at the Simons Institute workshop on Topics in Intelligence: World Models and Social Reasoning
1/4 Are diffusion language models ready for the real world? Not quite. Today's diffusion language models are missing some key ingredients, said Volodymyr Kuleshov of @cornelluniversity.bsky.social, at the Simons Institute workshop on Diffusion Generative Modeling: Progress and Next Steps
1/4 "AI is not when computer can write poetry. AI is when computer will *want* to write poetry." @ucberkeleyofficial.bsky.social's Alyosha Efros quoted a friend, when talking about the nature of "True AI," at the Simons Institute workshop on Topics in Intelligence: World Models and Social Reasoning
1/4 Remember Sydney, the chatbot that urged a @nytimes.com columnist to break up with his wife? Or how an LLM fine-tuned to output insecure code became misaligned on tasks unrelated to coding? What do these have in common? — Roger Grosse of @anthropic.com and U of Toronto at the Simons Institute.
1/3 Besides "robotic, embodied systems, AI systems never see the real world. They only see these artifacts humans create & feed into them...From these [snapshots of reality] our AI needs to infer the world that produced these snapshots": @shiryginosar.bsky.social at the Simons Institute
1/3 "Our conclusion is that AI consciousness is inevitable." In back-to-back talks, Manuel Blum and @lenoreblum.bsky.social of @cmu.edu discuss the Conscious Turing Machine and AI consciousness at the Simons Institute workshop on The Role of TCS in Modern Machine Learning