On the first page of search results there's this recent preprint ("On Verbalized Confidence Scores for LLMs" arxiv.org/html/2412.14...) so I gave it a skim. Interesting! Then I realized that this paper was cited in the Google AI overview... for a claim that I was pretty sure i…
Who's Who of AI
Casey Fiesler
Information science professor, technology ethics
information science professor (tech ethics + internet stuff)
kind of a content creator (elsewhere also @professorcasey)
though not influencing anyone to do anything except maybe learn things
she/her
more: casey.prof
What they're sharing
On Verbalized Confidence Scores for LLMs arxiv.org
Black Mirror Survey Study — Internet Rules Lab internetruleslab.com
giveto.colorado.edu
Articles & links
Ever seen a news story about AI and thought “this could be an episode of Black Mirror”? We're running a research study where we ask people to imagine generative AI futures and pitch their own science fiction story. More detail at the link and please share! www.internetruleslab…
A side effect of a new department (founded 2015) is that we have very very few alumni donations. And I sometimes get the (very kind) question "how can I support you since you don't get paid for teaching random internet people about AI ethics?" Support my students instead :) bi…
Their own posts
Recent commentary
Particularly in the context of (appropriate) crackdowns on LLM-generated papers (e.g., on arXiv) and more and more tales of fabricated citations, two reminders: (1) LLMs do not "cite their sources" (2) Using an LLM to generate, complete, or fix LaTeX code can result in fabricated citations
In the "you can't make this up" category of LLM interactions... TL;DR Google Gemini claims that LLMs "often give themselves a 95%+ confidence score even when they are hallucinating" - which was itself a "hallucinated" claim - then later assigns itself 0.95 confidence in that hallucinated claim. 🧵
I saw someone referring to the booing of commencement speakers talking about AI as a tantrum from students who don’t understand technology. That made me think that I bet machine breaking was also perceived as a tantrum. Anyway, here’s some thoughts about the Luddites.
Ran into another example of the Google AI Overview misinterpreting a search term as a command. :) This is a conference acronym that I saw on a CV and didn't know what it was!
hey so if anyone wants to do the "for science!" thing and pilot/give me feedback on a (long - because I would love feedback on where to shorten) survey about AI that also involves creating science fiction... reply and I will DM you :)
Well I guess it's nice that the bots are supportive of my Black Mirror + generative AI research study...
Their network
In Casey Fiesler's orbit
Center = Casey Fiesler. Left = members they follow (green edges). Right = members who follow them (blue edges). Top = mutual follows (orange edges, slightly larger). Drag any node to reposition; click to open that profile.