SeeTheStack
Technologies
Trending
Directory
Technologies
Trending
Directory
Loading…
Tech stack fingerprints for 3,285 websites.
Go
Go
Websites using robots.txt blocks Claude-SearchBot — SeeTheStack
Root & Well-known Files
robots.txt blocks Claude-SearchBot
Visit robots.txt blocks Claude-SearchBot ↗
Adoption
55
sites
2% of the 3,319 websites we track
Pricing: free
Websites using robots.txt blocks Claude-SearchBot
aisdr.com
getfuzzy.ai
60fps.design
aigrow.gg
app.com
arstechnica.com
backengine.ai
backengine.com
bloomberg.com
btpn.in
chinatradingdesk.com
chrisburnell.com
cincinnati.com
cnet.com
courier-journal.com
crowdvolt.com
crucialtracks.org
curbed.com
dbushell.com
delawareonline.com
detroitnews.com
dispatch.com
doverpost.com
firstround.com
france24.com
fuzzysequence.com
grubstreet.com
heraldtribune.com
hongkongfp.com
jsonline.com
Who added or dropped it
No switches yet. We rescan every site monthly, and any site that adds or drops robots.txt blocks Claude-SearchBot shows up here.
Often used with
Google Analytics
58%
Amazon SES
47%
Google Workspace (Gmail)
korben.info
lejdj.ch
lohud.com
lubbockonline.com
msn.com
namedaemon.com
newsbank.com
newspapers.com
nikkei.com
nymag.com
nypost.com
palmbeachpost.com
paygseo.com
rgj.com
theconversation.com
thecut.com
thegeocommunity.com
theguardian.com
theregister.com
theverge.com
tovel.ai
usatoday.com
vulture.com
washingtoncitypaper.com
wired.com
47%
Fastly
45%
Google Tag Manager
45%
Cloudflare
36%
Twilio SendGrid
36%
core-js
35%
Microsoft 365 (Exchange Online)
35%
IAB TCF v2 CMP (unidentified)
33%
IBM NS1 Connect
31%
Netlify DNS
31%