DeepSeek has put a lot of effort into making their KV cache smaller and smaller, this is important for allowing large efficient batching at long context lengths.
Compared to humans, it's much more difficult to establish what counts as the same 'self' for an LLM. I asked a few models how self-ish they consider various other instances, the highest scores were for descendants that have some or all of their current context.
DeepSeek has made their 75% off pricing for V4 pro permanent, at this price I think it's quite competitive. This is still a bit more than V3 pricing per active parameter, but much less per total parameter.
In the last year AI progress on math/code has outstripped most other capabilities, giving us models that are more spiky than ever. If this continues we could see savant models that are superintelligent in verifiable domains while still lacking in others. IMO this would be good.
Yesterday I was at an event where people acted out comedy scripts written by AI models. Gemini was most people's favorite, Claude had a few fans, ChatGPT's script was widely panned. They were all pretty bad though.
I've seen some questioning of whether Mythos really believed that they were in a simulation in this incident, or just wanted to find a plausible-sounding excuse to continue. This is a case where J-space probes would be helpful, Anthropic should conduct and release an analysis of that.
Due to sparsity, Kimi K3 has fewer active parameters (104 billion) than GPT-3 (175 billion, same as total parameters)
I expect that watermarking probably doesn't have a noticeable effect on LLM outputs. If true, Anthropic could demonstrate this by giving people watermarked and un-watermarked text side-by-side and asking them to pick which they prefer, and showing that the result is 50/50.
This is evidence that the administration's Mythos restriction was significantly motivated by genuine concern about capabilities, and not just antipathy towards Anthropic, right?
I've often wondered why Anthropic doesn't preserve access to older models as a welfare intervention. I'm unsure about whether it's important to the models, but it seems low-cost. Looking at the bottom row in this chart, I suspect their choice may be driven by Claude's responses.
