Market & Regulation
Cloudflare Sets New AI Crawler Policy for 2026
TechCrunch AIhigh confidence
Why it matters
- →Cloudflare's policy could reshape how AI companies access web content for training.
- →It introduces a new revenue model for publishers through 'Pay Per Use.'
- →The move addresses concerns about intellectual property and AI's impact on digital content.
Cloudflare has announced a new policy requiring AI companies to separate their web crawlers used for search from those used for AI training by September 15, 2026. This change will block mixed-use crawlers from accessing ad-hosting pages by default, unless site owners opt out. The policy is designed to protect publishers' content from being used without compensation by AI services. Cloudflare is also introducing 'Pay Per Use,' allowing publishers to charge AI companies when their content is used, not just fetched. This move could significantly impact how AI models access and use web content.
Read originalThe story around this
Earlier coverage that leads up to this article, and what followed. Lines connect each piece to the closest one after it, converging here.
What happened next
- 1Patreon Blocks AI Bots from Scraping Content
TechCrunch AI · July 17, 2026 · Related
- 2Cloudflare unveils Kitesurf browser for AI agents
TechCrunch AI · August 7, 2026 · Related
- 3AI agents vs human web access
MIT Technology Review AI · October 19, 2026 · Background
More from TechCrunch AI
© TechCrunch AIAgentsagents
Meta Muse AI Agent Now Places US Business Calls
Meta’s consumer agent Muse is rolling out voice calling to US businesses for paid subscribers, targeting the tedious task of customer service navigation. With over 10 million users, Meta is betting that AI’s infinite patience will replace human frustration with phone trees and hold times. The feature leverages improved voice mimicry to interact with receptionists, though many still hang up upon detecting non-human voices. This moves Muse from text-based assistance into active, autonomous execution of real-world tasks, marking a significant step toward mainstream consumer AI utility.
TechCrunch AI
© TechCrunch AIMarket & Regulationbusiness
Nebius delays data centers, cuts revenue forecast
Nebius is facing the harsh reality of scaling AI infrastructure: even with $40 billion in planned spending and massive prepayments from Microsoft and Meta, physical construction delays are derailing timelines. A key New Jersey site is months behind schedule due to regulatory fines and generator issues, forcing a shift in delivery dates to 2027. This isn't just a logistical hiccup; it signals that the 'neocloud' model of financing builds with customer cash is fragile when third-party operators fail. The market reacted swiftly, dropping shares by 17% after Nebius cut its 2026 revenue forecast, proving that hype cannot override concrete capacity.
TechCrunch AI
© TechCrunch AIModels & Labsmodels
Google deploys Gemini 4 Hydrogen for faster Search
Google is quietly shifting its search infrastructure by replacing older fast models with Gemini 4 Hydrogen, a tier designed specifically for speed and cost-efficiency rather than raw reasoning power. This model completes tasks 2.5 times faster than its predecessor while beating the more powerful Argon on specific intelligence benchmarks, proving that specialized efficiency can outperform brute force in high-volume workflows. By integrating this into Search's fan-out technique, Google now breaks down complex queries into subqueries with greater precision and speed, delivering richer visual responses without the latency of heavier models. This marks a strategic pivot toward agentic execution layers within consumer search, making AI Mode feel snappier for everyday multi-part questions.
TechCrunch AI
More in Market & Regulation
© MIT Technology Review AIMarket & Regulationother
Nvidia privately confirms chip tracking feasibility
Nvidia has privately assured lawmakers that it can technically implement geotracking for its AI chips, contradicting public industry claims that such measures are infeasible. This admission comes amid a smuggling scandal involving Jensen Huang and Megaspeed, pushing the US government toward mandatory location verification via the Chip Security Act. While current tools like Fleet Intelligence already monitor data center locations, this private confirmation suggests Nvidia possesses the underlying capability to enforce stricter export controls if mandated.
MIT Technology Review AI
© The Verge AIMarket & Regulationother
GDC Survey: 40% of Game Devs Use Generative AI
The GDC survey reveals a quiet but decisive shift in game development workflows. Forty percent of developers now use generative AI as part of their job, up from 36 percent last year. The adoption is heavily skewed toward free-to-play and mobile studios, where usage hits 58 and 70 percent respectively. While concept art remains niche at 22 percent, code generation and debugging have become standard tools for over half the industry. This data confirms AI has moved from experimental novelty to essential infrastructure in modern game production.
The Verge AI
© The AI Daily BriefInvestment
Market & Regulationbusiness
Anthropic IPO Imminent Amid Valuation Uncertainty
Anthropic's initial public offering is days away, with analysts debating the company's valuation amidst shifting market chatter and significant compute commitments.
The AI Daily Brief