Major AI Labs Adopt Unified Standard for Web Publishers to Block Model Training
OpenAI, Google, Microsoft, and Anthropic have committed to a unified web standard allowing publishers to opt out of AI scraping through a single policy token.
All stories tagged web scraping on The Silicon Valley News.
OpenAI, Google, Microsoft, and Anthropic have committed to a unified web standard allowing publishers to opt out of AI scraping through a single policy token.
A coalition of civil liberties groups is urging New York Governor Kathy Hochul to veto Senate Bill 9934A, warning that its ban on 'stealth crawlers' will dismantle open-web scraping, search indexing, and security research.
Cloudflare has introduced Kitesurf, a cloud-hosted browser designed specifically for AI agents rather than human users, aiming to bypass the resource-heavy overhead of Chromium.
A federal appeals court has clarified the scope of the Computer Fraud and Abuse Act, ruling that building a web browser with AI capabilities does not constitute unauthorized access, a decision with significant implications for tech competition and innovation.