Perplexity is repeatedly modifying their user agent and changing IPs and ASNs to hide their crawling activity, in direct conflict with explicit no-crawl preferences expressed by websites.
Ignoring the self-promo part of the article, Cloudflare is in big part responsible for all of this. Their home page promotes mostly “AI” tools, they are even offering “pay-per-crawl” to make it a viable business model.
So while you correctly shit on Perplexity, don’t forget that Cloudflare is even worse enemy and not a friend.
Ignoring the self-promo part of the article, Cloudflare is in big part responsible for all of this. Their home page promotes mostly “AI” tools, they are even offering “pay-per-crawl” to make it a viable business model.
So while you correctly shit on Perplexity, don’t forget that Cloudflare is even worse enemy and not a friend.