// AI news · 2026-11-02 · Dev.to AI
AI crawlers outperform search crawlers: 7-night census
Over seven nights, PerplexityBot outperformed OpenAI's OAI-SearchBot 273:30.
Published 2026-11-02 05:28 (Beijing time)Source Dev.to AIAuto-summarised, fact-checked against source
01Summary
SEO tools like Majestic and Semrush recorded 1,274 visits—more than all search engines combined. 65.5% of bot hits had no identifiable name. AI crawlers are now dominant in content access, with Perplexity leading the AI family. The site allows AI crawlers to cite content, driving a 50% weekly increase in AI referrals.
Summarised from the original article by an open-source model, then checked automatically: every number and English name must appear in the source. The original article is authoritative.
02Key points
- PerplexityBot outperformed OpenAI's OAI-SearchBot 273:30.
- SEO tools recorded 1,274 visits—more than all search engine bots combined.
- 65.5% of bot hits had no identifiable user-agent string.
- AI crawlers drive a 50% weekly growth in AI referrals.
03Source
AI crawlers vs search crawlers: the 7-night census
Dev.to AI · published 2026-11-02 05:28 Beijing time
05Related news
- Same source · Building Secure AI Chat with React and .NET2026-11-02 · Dev.to AI
- Same source · Author builds independent AI coworker2026-11-02 · Dev.to AI
- Same topic · AI is shifting app development power2026-11-01 · Xataka
- Same topic · 14 AI Models Show Moral Decisions Affected by Prompt Order2026-11-01 · T客邦
- Same day · Developer and AI Discuss Language Survival2026-11-02 · Dev.to AI
From AIMap AI news: collected from 1,100+ official and media sources. Summaries only restate the original article; follow the link for the full story. All stories of 2026-11-02 · Archive by date · All AI news · Today