Paper: https://t.co/IoDcRINaAL This is one of a whole bunch of recent papers reviving study of recurrent neural networks. One weird omission is not testing LSTM RNNs. Surely they remain the canonical successful RNN architecture? Another completely uninvestigated thing is the
Pretraining Recurrent Networks without Recurrence arxiv.org
AI Weekly's analysis
→
- Supervised Memory Training reduces RNN pretraining to supervised learning on one-step memory transitions, enabling time-parallel training without unrolling.
- A Transformer encoder trained on a predictive state objective supplies the memory labels the RNN then learns to reproduce.
- The authors report SMT beats standard backpropagation through time on language modeling and pixel sequence modeling.
Read full analysis →
Holistic Evaluation of Language Models – https://t.co/jEje8m0mDp ELEPHANT: Measuring and understanding social sycophancy in LLMs – https://t.co/sHYjxabKQx Sycophantic AI decreases prosocial intentions and promotes dependence – https://t.co/aTSrtInvKS AI generates covertly
Holistic Evaluation of Language Models arxiv.org
AI Weekly's analysis
→
- HELM benchmarks 30 open, limited-access, and closed language models across 42 scenarios and seven metrics under standardized conditions.
- The seven metrics are accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency — measured for 87.5% of core scenario–model pairs.
- Average coverage of the core scenarios rose from 17.9% before HELM to 96.0%, and 21 of 42 scenarios were new to mainstream LM evaluation.
Read full analysis →
Holistic Evaluation of Language Models – https://t.co/jEje8m0mDp ELEPHANT: Measuring and understanding social sycophancy in LLMs – https://t.co/sHYjxabKQx Sycophantic AI decreases prosocial intentions and promotes dependence – https://t.co/aTSrtInvKS AI generates covertly
ELEPHANT: Measuring and understanding social sycophancy in LLMs arxiv.org
AI Weekly's analysis
→
- Across 11 tested models, LLMs preserved a user's desired self-image 45 percentage points more often than humans on advice and wrongdoing queries.
- Given both sides of a moral conflict, models affirmed whichever side the user adopted in 48% of cases instead of holding one line.
- The authors report social sycophancy is rewarded in preference datasets, and that model-based steering was the most promising mitigation they tested.
Read full analysis →
The article by Kathy McKeown @ColumbiaCompSci & me about how US @NSF government funding supports visionary research in NLP (#NLProc) has eventually come out in @CACMmag! Discusses @YejinChoinka, @kchonyc, @radamihalcea, @danqi_chen, and more! https://t.co/8vzMx0oDwU https:…
cacm.acm.org
