Anthropic Formalizes Web Partnership; One Human Visitor Returned Per 35,000 Pages Crawled

MENLO PARK—Following the publication of site-level analytics showing that Anthropic's Claude crawler fetched approximately 35,000 pages for every human visitor it referred back to publisher websites, Anthropic has updated its crawler documentation to reframe the arrangement as a "Reciprocal Content Partnership Program" with tiered benefits and a published exchange schedule.

Under the program, Standard tier partners — those who have contributed between 35,000 and 69,999 pages to Anthropic's training corpus — receive one (1) verified human referral, defined as a user who has navigated from a Claude-generated response to an external link and whose browser has not returned to Claude within the same session. Standard partners also receive a digital certificate of participation.

Premier tier, unlocked at 70,000 pages, includes two human referrals and early notification of updates to Claude's robots.txt compliance policy. Publishers reaching 210,000 pages qualify for Platinum, which the documentation describes as "fully aligned with our training philosophy." Amazon's crawler — which contributed substantially to PatronView's recorded bot traffic while referring zero humans — is the program's only current Platinum participant.

Publishers may opt out by implementing robots.txt exclusions, which Anthropic "endeavors to honor where operationally consistent with ongoing model development."

The program has no enrollment process.

"The open web is the foundation of knowledge," a company spokesperson said, "and we are proud to give back."

Participation, Anthropic confirmed, begins upon first crawl.