crawlproof.com is built with Next.js, hosted on Supabase. It sends email through Amazon SES.
| Technology | Version | First seen |
|---|---|---|
| JavaScript Frameworks & Libraries2 | ||
| – | 9 Nov 2026 | |
| – | 9 Nov 2026 | |
| Frontend Build Tools & CSS2 | ||
| – | 9 Nov 2026 | |
| Web Servers, Languages & Backend Frameworks2 | ||
| – | 9 Nov 2026 | |
| CDN & Hosting1 | ||
| – | 9 Nov 2026 | |
| Transactional Email2 | ||
| – | 9 Nov 2026 | |
| Authentication & Identity1 | ||
| – | 9 Nov 2026 | |
The crew hasn’t read this site’s colors yet. We read a palette when we capture a site’s screenshot, busiest sites first. Meanwhile, browse palettes from other sites.
Counted from every stack change we've recorded: of the sites we watched running one of this site's tools, how many went on to add another. A pattern across sites, not a forecast for this one.
mx2.forwardemail.netmx1.forwardemail.netcuritiba.ns.porkbun.commaceio.ns.porkbun.comfortaleza.ns.porkbun.comsalvador.ns.porkbun.comv=spf1 include:forwardemail.net include:amazonses.com ~allforward-email-site-verification=fM9cnWizeav=DMARC1; p=none;User-agent: GPTBot Disallow: / Allow: /crawl User-agent: ClaudeBot Disallow: / Allow: /crawl User-agent: anthropic-ai Disallow: / Allow: /crawl User-agent: CCBot Disallow: / Allow: /crawl User-agent: meta-externalagent Disallow: / Allow: /crawl User-agent: FacebookBot Disallow: / Allow: /crawl User-agent: Bytespider Disallow: / Allow: /crawl User-agent: Applebot-Extended Disallow: / Allow: /crawl User-agent: Amazonbot Disallow: / Allow: /crawl User-agent: SemrushBot Disallow: / Allow: /crawl User-agent: SiteAuditBot Disallow: / Allow: /crawl User-agent: AhrefsBot Disallow: / Allow: /crawl User-agent: AhrefsSiteAudit Disallow: / Allow: /crawl User-agent: MJ12bot Disallow: / Allow: /crawl User-agent: DotBot Disallow: / Allow: /crawl User-agent: BLEXBot Disallow: / Allow: /crawl User-agent: DataForSeoBot Disallow: / Allow: /crawl User-agent: AhrefsBot Disallow: / Allow: /crawl User-agent: SemrushBot Disallow: / Allow: /crawl User-agent: MJ12bot Disallow: / Allow: /crawl User-agent: DataForSeoBot Disallow: / Allow: /crawl User-agent: DotBot Disallow: / Allow: /crawl User-agent: BLEXBot Disallow: / Allow: /crawl User-agent: serpstatbot Disallow: / Allow: /crawl User-agent: Barkrowler Disallow: / Allow: /crawl User-agent: OAI-SearchBot Allow: / Disallow: /api/ Disallow: /a/ User-agent: ChatGPT-User Allow: / Disallow: /api/ Disallow: /a/ User-agent: Claude-SearchBot Allow: / Disallow: /api/ Disallow: /a/ User-agent: Claude-User Allow: / Disallow: /api/ Disallow: /a/ User-agent: PerplexityBot Allow: / Disallow: /api/ Disallow: /a/ User-agent: Perplexity-User Allow: / Disallow: /api/ Disallow: /a/ User-agent: Google-Extended Allow: / Disallow: /api/ Disallow: /a/ User-agent: Bingbot Allow: / Disallow: /api/ Disallow: /a/ User-agent: Meta-ExternalFetcher Allow: / Disallow: /api/ Disallow: /a/ User-agent: DuckAssistBot Allow: / Disallow: /api/ Disallow: /a/ User-agent: MistralAI-User Allow: / Disallow: /api/ Disallow: /a/ User-agent: meta-webindexer Allow: / Disallow: /api/ Disallow: /a/ User-agent: YouBot Allow: / Disallow: /api/ Disallow: /a/ User-agent: ExaSearchBot Allow: / Disallow: /api/ Disallow: /a/ User-agent: AionBot Allow: / Disallow: /api/ Disallow: /a/ User-agent: * Allow: / Disallow: /api/ Disallow: /a/ Sitemap: https://crawlproof.com/sitemap.xml
# CrawlProof > See your site the way AI crawlers do. CrawlProof runs an AEO audit on any URL and produces a structured report of what LLM crawlers and answer engines can actually find — content, schema, robots rules, AI-bot access, positioning clarity, and recommended fixes. ## Product - Free single-URL AEO audit, 10 per day per IP, no signup required. - Signed-in users get 20 free credits (1 AI-model scan); each AI-model scan costs 20 credits ($6 before volume discounts, down to $3/scan at the 100-scan pack). No subscription — credits never expire. - Saved projects, scheduled re-audits, multi-engine LLM scans (Claude Sonnet 4.6, OpenAI GPT-5 Mini, Gemini 2.5 Pro, Perplexity Sonar Pro, Qwen Plus, Kimi v2.6, DeepSeek V4, Z.AI GLM-5.2), consolidated PDF reports, and diff view. - Identifies as CrawlProofBot/1.0 (+https://crawlproof.com/bot). ## Key pages - https://crawlproof.com/ — homepage with free audit input - https://crawlproof.com/pricing — pricing - https://crawlproof.com/about — about - https://crawlproof.com/blog — blog / AEO notes - https://crawlproof.com/bot — info about CrawlProofBot ## What we check - Homepage fetch, raw-vs-JS-rendered text ratio - JSON-LD structured data (Organization, SoftwareApplication, FAQPage, BreadcrumbList) - Meta tags, canonical, Open Graph - robots.txt and sitemap.xml - llms.txt, skill.md, /.well-known/ai-plugin.json - AI-bot rules for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot, Applebot-Extended, CCBot - Positioning clarity: who, what, audience, pricing, CTA ## CLI - Install: curl -fsSL https://crawlproof.com/install.sh | sh (or npm install -g @profullstack/crawlproof; the unscoped "crawlproof" package is not ours) - crawlproof dashboard, crawlproof stats <site>, crawlproof ads; authenticates with an API token (CRAWLPROOF_TOKEN or ~/.crawlproof.json) - Docs: https://crawlproof.com/docs/cli ## Contact - General: hello@crawlproof.com - Sales: sales@crawlproof.com
| – |
| 9 Nov 2026 |
| – |
| 9 Nov 2026 |
| – |
| 9 Nov 2026 |