Karnofsky's Fund Directs $336M to Anthropic Evaluators
Key insights
- Coefficient Giving controls 30-40% of tracked AI safety funding, totaling roughly $336 million since 2017, making it the dominant single funder.
- Holden Karnofsky co-founded Coefficient Giving, joined Anthropic's technical staff in January 2025, and is married to Anthropic co-founder Daniela Amodei.
- SaferAI's primary funder Jaan Tallinn simultaneously led Anthropic's $124M Series A and serves as an Anthropic board observer.
Why this matters
The structural arrangement means AI companies like Anthropic are, through a few degrees of separation, funding the organizations authorized to evaluate whether their models are safe to release. Redwood Research's collapse from $10 million to $22,060 in annual revenue demonstrates that funding dependency already shapes which labs survive and which shut down, regardless of research quality. With Anthropic's seven co-founders pledging 80% of an estimated $37.8 billion fortune and Coefficient Giving projecting $1 billion in 2026 deployments, the pipeline is about to grow by an order of magnitude with no new structural safeguards in place.
Summary
Coefficient Giving, formerly Open Philanthropy (rebranded November 2025), controls 30-40% of tracked AI safety funding, roughly $336 million since 2017. Its co-founder Holden Karnofsky joined Anthropic in January 2025 and is married to Anthropic co-founder Daniela Amodei. METR and SaferAI, which evaluate Anthropic models, are among the organizations this pipeline funds.
Essentially: (Coefficient Giving, SaferAI, METR) place evaluators in financial dependency on Anthropic's backers.
- SaferAI's funder Jaan Tallinn led Anthropic's $124M Series A and sits on its board; an Anthropic researcher also advises SaferAI on methodology.
- Redwood Research dropped from $10M in 2023 revenue to $22,060 in 2024, with $2.9M in ongoing expenses.
AI safety lacks mandatory conflict-of-interest disclosure requirements that pharmaceutical research carries by law.
Potential risks and opportunities
Risks
- METR and SaferAI face credibility challenges from investors and regulators scrutinizing Anthropic's June 1 S-1 filing, which could trigger disclosure demands about evaluator conflicts of interest.
- Redwood Research, running $2.9M in expenses against $22,060 in 2024 revenue, faces insolvency without new funding, potentially creating a gap in independent AI safety research capacity.
- If the $37.8 billion co-founder pledge flows primarily through Coefficient Giving without new governance structures, it deepens rather than resolves the structural conflicts the article documents.
Opportunities
- Government funders like NSF, currently running a $10.9M AI safety program, are positioned to expand credibly given conflict-of-interest questions now surrounding private philanthropic evaluators.
- Independent philanthropies named in the article, including Pew Charitable Trusts and Longview Philanthropy, can fill the evaluator-funding gap with no Anthropic financial ties.
- Governance and compliance consultancies could build practices around mandatory conflict-of-interest disclosure standards for AI safety evaluators, modeled on the pharmaceutical research requirements the article benchmarks against.
What we don't know yet
- Whether METR has ever softened or withheld a finding because Anthropic declined model access, given METR's own acknowledgment that companies face no legal obligation to cooperate with pre-release testing.
- What governance changes, if any, Coefficient Giving implemented after Holden Karnofsky joined Anthropic's technical staff in January 2025, since the article describes no firewall between his new role and grant decisions.
- How the $37.8 billion co-founder wealth pledge announced in January 2026 will be governed, specifically whether new independent grantmaking bodies will be created or Coefficient Giving will absorb the capital.
Originally reported by itbb.substack.com
Read the original article →Original headline: Who Funds the Watchdogs — Investigative Piece Finds Coefficient Giving Directs ~$336M to AI Safety Orgs Including METR and Redwood, Creating Structural Conflicts With Anthropic Evaluators