Juan Diego Rodriguez
Researcher with public evidence across AI research, NLP & language.
- AI signals
- 11 past 30d
- Sources
- 11 distinct domains
- Discussões
- 41 past 30d
- Latest signal
- 5d ago
Articles & links
“It's literally the gulag,” one of the employees claims. “You have zero purpose in life all of a sudden, you barely interact with anyone, you just have these tasks every week." www.wired.com/story/mark-z...
2) Characterizing Narrative Content in Web-scale LLM Pretraining Data, by @teagrjohnson.bsky.social, @elliottash.bsky.social, @andrewpiper.bsky.social, @mariaa.bsky.social arxiv.org/abs/2606.19468 Why: annotation and analysis of narrative features across the pretraining data (…
- A new arXiv preprint introduces NarraBERT, a RoBERTa-based classifier, and applies it to 3 million passages from the 3-trillion-token Dolma corpus.
- The framework operationalizes three narrative elements, agency, setting, and events, across 11 interpretable dimensions, trained on 400 annotated passages.
- The authors report narrative qualities are unequally distributed across pretraining sources and topics in ways current curation practices do not measure.
jacobin.com/2026/07/ai-n... “AI is a machine that takes good jobs and makes them precarious. It’s a machine for turning stable employment into gig work”
An incomplete list: 1) Explaining Attention with Program Synthesis, by Amiri Hayes, @belindazli.bsky.social and arxiv.org/abs/2606.19317 Why: LMs can generate code that can explain/replace attention heads! 🤯 (Program synthesis 🤝 Interpretability)
- Under 1,000 LM-generated programs reproduce attention patterns in GPT-2, TinyLlama-1.1B and Llama-3B with above 75% average IoU on TinyStories.
- Replacing 25% of attention heads with the synthesized programs raised average perplexity by only 16% while keeping downstream question-answering performance.
- The pipeline computes a head's attention matrices, prompts a pretrained LM to write Python that reproduces them, then re-ranks by held-out accuracy.
4) Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention arxiv.org/abs/2605.29548
- The paper argues power-law scaling already implies a larger model will learn parts of the data distribution a smaller model cannot, even with infinite training data.
- Pretraining experiments on OLMo models from 4M to 4B parameters found only the larger models learned the infrequent and complex tasks.
- The proposed mechanism is reduced gradient interference: weaker common-task updates leave rare-task features intact in larger models.
"I do not understand why anyone who believes that AI will eliminate most jobs and much of humanity’s relevance would rush to build that future." Really???? The answer is money about.fb.com/news/2026/08...
Does anyone know why MASS (arxiv.org/pdf/1905.02450) didn't take off, while BART did? (OK maybe no one remembers BART today, but it was popular around 2019-2022)
Recent commentary
Tip on reviewing ARR papers in 2026: - Check the references FIRST. I wasted 40 minutes reviewing a paper before realizing the references were LLM-generated.
If we over-rely on LLMs for science, I fear we will remain stuck in the convex hull of what already exists. Is anyone studying this?
So much academic writing is bland and uninspiring. And LLMs will make it worse. What your favorite examples of *good* writing?
I'm so tired of seeing LLM-generated reviews. They just wasting all our time. If this continues, "peer review" will mean nothing
It was a pleasure to moderate the panel at the ACL 2026 SRW: "The next big questions in NLP: What should students work on?" Huge thanks to our panelists, @boydgraber.bsky.social, @igurevych.bsky.social, Preslav Nakov and Yulia Tsvetkov for their thoughtful answers and helpful advice! Takeaways 👇
I wish I could just go to an island, no internet, no deadlines, no AI. Just a library, a coffeeshop, and people to collaborate with.
I asked GPT 5.6 Sol to solve the Collatz conjecture. Why is it searching the NOAA fisheries website??
It's crazy to think it took only 4 years to go from "LLMs are only good for generating misinformation at scale" to them proving new theorems, writing code, and writing (mostly) accurate literature surveys.
Google AI: "oh I thought you meant Neuro-Linguistic Programming" I really thought we were over this! Apparently not
"Specification and Removal of Demons." Not what I expected to read in an NLP book! (Charniak and Wilks, Computational Semantics)
In Juan Diego Rodriguez's orbit
Center = Juan Diego Rodriguez. Left = members they follow (green edges). Right = members who follow them (blue edges). Top = mutual follows (orange edges, slightly larger). Drag any node to reposition; click to open that profile.