The history of sampling techniques is certainly interesting. And I loved Ettinger and others showing BERT liked "A robin is not a [bird]" since bird was salient in context. arxiv.org/pdf/1907.13528. But models got better as things like post-training steer them out of dumb repe…
Who's Who of AI
Kyle Mahowald
UT Austin linguist http://mahowak.github.io/. computational linguistics, cognition, psycholinguistics, NLP, crosswords. occasionally hockey?
What they're sharing
arxiv.org