Figures / Holders
7.0%
Of sites shut out Bytespider (ByteDance's crawler)
427 of the 6,094 robots.txt files read in the current pass disallow Bytespider from the whole site.
Updated 17 Oct 2026, from its newest record.
The 3 records it is computed from, newest first
- As stated219 of 1,898 sites that answered shut at least one AI crawler outDated17 Oct 2026SourceTranco list of the most visited sites…
- As stated244 of 2,089 sites that answered shut at least one AI crawler outDated16 Oct 2026SourceTranco list of the most visited sites…
- As stated233 of 2,107 sites that answered shut at least one AI crawler outDated15 Oct 2026SourceTranco list of the most visited sites…