robots.txt
See which AI crawlers other sites block →# As a condition of accessing this website, you agree to abide by the following content signals:
# (a) If a content-signal = yes, you may collect content for the corresponding use.
# (b) If a content-signal = no, you may not collect content for the corresponding use.
# (c) If the website operator does not include a content signal for a corresponding use, the website operator neither grants nor restricts permission via content signal with respect to the corresponding use.
# The content signals and their meanings are:
# search: building a search index and providing search results. Search does not include providing AI-generated search summaries.
# ai-input: inputting content into one or more AI models, such as retrieval augmented generation, grounding, or other real-time taking of content for generative AI search answers.
# ai-train: training or fine-tuning AI models.
# ANY RESTRICTIONS EXPRESSED VIA CONTENT SIGNALS ARE EXPRESS RESERVATIONS OF RIGHTS UNDER ARTICLE 4 OF THE EUROPEAN UNION DIRECTIVE 2019/790 ON COPYRIGHT AND RELATED RIGHTS IN THE DIGITAL SINGLE MARKET.
User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=no
Allow: /
Disallow: /dashboard
Disallow: /api
User-agent: GPTBot
Allow: /
Disallow: /dashboard
Disallow: /api
User-agent: ClaudeBot
Allow: /
Disallow: /dashboard
Disallow: /api
User-agent: CCBot
Allow: /
Disallow: /dashboard
Disallow: /api
User-agent: PerplexityBot
Allow: /
Disallow: /dashboard
Disallow: /api
User-agent: Google-Extended
Disallow: /
Sitemap: https://www.agentseo.dev/sitemap.xml
Host: www.agentseo.dev
llms.txt
Compare with other sites' llms.txt files →# AgentSEO
> AgentSEO is an SEO API for AI agents and developers. Use this file as the compact reading list for understanding the product, docs, integrations, and agent-facing API surface.
## Agent Resources
- [OpenAPI specification](https://www.agentseo.dev/openapi.yaml): REST API contract for AgentSEO.
- [API catalog](https://www.agentseo.dev/.well-known/api-catalog): Well-known API discovery document.
- [MCP server card](https://www.agentseo.dev/.well-known/mcp/server-card.json): Hosted MCP server discovery for compatible agents.
- [Agent skills index](https://www.agentseo.dev/.well-known/agent-skills/index.json): Skill documents exposed by AgentSEO.
## Documentation
- [Async jobs & retries](https://www.agentseo.dev/docs/async-jobs/index.md): Queue work, poll with a deadline, and handle terminal states.
- [Search & extraction](https://www.agentseo.dev/docs/api-reference/search/index.md): Search results, page extraction, and SERP analysis.
- [Content & page quality](https://www.agentseo.dev/docs/api-reference/content/index.md): Briefs, content gaps, on-page checks, internal links, and publishing decisions.
- [Rank & AI visibility](https://www.agentseo.dev/docs/api-reference/visibility/index.md): Rank history, local audits, AI Overview evidence, and mention tracking.
- [Keywords & competitors](https://www.agentseo.dev/docs/api-reference/keywords/index.md): Keyword ideas, metrics, clustering, domain comparisons, and traffic estimates.
- [Backlinks](https://www.agentseo.dev/docs/api-reference/backlinks/index.md): Link profiles, referring domains, intersections, and outreach opportunities.
- [Opportunities & signals](https://www.agentseo.dev/docs/api-reference/opportunities/index.md): Find opportunities, prepare briefs, and inspect social demand signals.
- [Jobs & webhooks](https://www.agentseo.dev/docs/api-reference/system/index.md): Poll results, stream job events, and manage webhook delivery.
- [Documentation](https://www.agentseo.dev/docs/index.md): AgentSEO developer documentation hub.
- [Quickstart](https://www.agentseo.dev/docs/quickstart/index.md): Set up an API key and make the first AgentSEO API request.
- [API Reference](https://www.agentseo.dev/docs/api-reference/index.md): Reference for AgentSEO REST endpoints, request parameters, authentication, response envelopes, and credit costs.
- [Authentication](https://www.agentseo.dev/docs/authentication/index.md): How agents authenticate with AgentSEO using API keys and attribution headers.
- [SDKs](https://www.agentseo.dev/docs/sdks/index.md): JavaScript and Python SDK entry points for AgentSEO.
- [Use Cases](https://www.agentseo.dev/docs/use-cases/index.md): Common SEO workflows agents can automate with AgentSEO.
- [Workflow Bundles](https://www.agentseo.dev/docs/workflows/index.md): ICP workflow bundles for launch, refresh, AI visibility, programmatic SEO, and competitor content gap work.
- [Errors](https://www.agentseo.dev/docs/errors/index.md): Error codes, retry guidance, and async job status handling.
- [Changelog](https://www.agentseo.dev/docs/changelog/index.md): Product and API changes for AgentSEO.
## Core
- [AgentSEO](https://www.agentseo.dev/index.md): SEO API for AI agents and developers, with compact JSON outputs for search, extraction, rank tracking, content gap analysis, SERP outlines, AI overview extraction, local SEO, keyword workflows, and opportunity-to-refresh briefs.
- [Pricing](https://www.agentseo.dev/pricing/index.md): Plans and credit usage for AgentSEO API workflows.
## Insights
- [Blog](https://www.agentseo.dev/blog/index.md): Research, proof, and operating notes for developers, growth engineers, agencies, and builder-marketers shipping agentic SEO workflows.
## Integrations
- [Integrations](https://www.agentseo.dev/integrations/index.md): AgentSEO integration options for automation tools and coding agents.
- [Claude Code Integration](https://www.agentseo.dev/integrations/claude-code/index.md): Use AgentSEO from Claude Code workflows.
- [Claude Desktop Integration](https://www.agentseo.dev/integrations/claude-desktop/index.md): Connect AgentSEO to Claude Desktop through MCP.
- [OpenClaw Integration](https://www.agentseo.dev/integrations/openclaw/index.md): Use AgentSEO with OpenClaw agents.
- [n8n Integration](https://www.agentseo.dev/integrations/n8n/index.md): Build SEO automations in n8n with AgentSEO.
- [Make Integration](https://www.agentseo.dev/integrations/make/index.md): Build SEO automations in Make with AgentSEO.
## Research
- [Research and Benchmarks](https://www.agentseo.dev/research/index.md): Public AgentSEO benchmarks, methods, limitations, and machine-readable evidence.
- [Search Payload Size Benchmark](https://www.agentseo.dev/research/payload-benchmark/index.md): A three-query comparison of provider task-result payloads, AgentSEO normalized JSON, and enriched agent output.
- [Editorial and Evidence Policy](https://www.agentseo.dev/editorial-policy/index.md): How AgentSEO researches, tests, reviews, updates, discloses automation, and corrects technical content.
## Company
- [About](https://www.agentseo.dev/about/index.md): Company and product background for AgentSEO.
- [Contact](https://www.agentseo.dev/contact/index.md): Contact AgentSEO for product, support, and partnership questions.
- [Status](https://www.agentseo.dev/status/index.md): Service status and availability information.