cbc.ca via Hacker News

Marmot library consortium logs 2B AI-bot hits in 30 days: CBC

AI Ethics Generative AI ai-business

TL;DR

  • Marmot, a nonprofit Colorado library consortium, logged more than 2 billion bot connection attempts in a single 30-day period.
  • UNC Chapel Hill recorded over 11,000 Finnish-music searches from thousands of IPs in a day, periodically locking out human users.
  • OpenAI, Google, Anthropic and Microsoft back robots.txt opt-outs, but compliance is voluntary and third-party scrapers ignore it.

Marmot, a nonprofit consortium of academic, public and school libraries across Colorado, logged more than 2 billion bot connection attempts in a single 30-day window, CBC News reported. Earlier in 2026 the same system was seeing around 6 million attempts every 24 hours.

The volume is showing up on individual campus catalogues. At the University of North Carolina at Chapel Hill, library staff clocked more than 11,000 searches for Finnish music from thousands of different IP addresses in a single day. A comparable month before the surge brought something like 15 searches combining 'Finnish' and 'music.' As simultaneous searches climbed from a normal busy-period peak of 100 to 500 or more, the UNC catalogue periodically shut students, faculty and staff out.

"They're just out there on the internet, just bombarding us with trying to grab up as much material as possible," Jill Strykowski, cataloguing and government documents lead at San Jose State University, told CBC. Eric Hellman, executive director of Project Gutenberg, said poorly written scrapers are "destroying open access" on the internet.

The bots routinely ignore robots.txt. OpenAI, Google, Anthropic and Microsoft back opt-out conventions, but "compliance is voluntary, and the agreement does not cover problematic bots run by third-party companies or smaller AI startups." Scrapers have been recorded requesting thousands of documents per second across rotating IP farms, which makes address-level blocking impractical without catching legitimate users. Michael Weinberg, co-director of GLAM-E at NYU Law School and the University of Warwick, flagged that same trade-off.

UNC eventually installed an AI-based firewall to sort bot traffic from human readers. Other libraries have fallen back on CAPTCHA challenges or login walls, each of which narrows public access as the price of staying online.

Shared on Bluesky by 2 AI experts