STACK12 categories

JavaScript Frameworks & LibrariesNext.js

Frontend Build Tools & CSSTailwind CSS

Web Servers, Languages & Backend FrameworksNode.js

CDN & HostingAWS

HISTORY

First scan logged. We rescan monthly and note every change here.

SITE FINGERPRINT

aidirectori.es

Built with Next.js and React, hosted on AWS behind Cloudflare.

TechTechnologies32

ScannedLast scanned14 Oct

ServerServerUS

Full reportScanned today

32 technologies on aidirectori.es

Download JSON

Confidence reflects how many independent signals matched.

# AI Directories Robots.txt # Last updated: 2026-10-14 # # IMPORTANT: Googlebot does not support the Crawl-delay directive and will show warnings # in Google Search Console for any Crawl-delay lines it encounters. These warnings are # informational only and do not affect crawling. Crawl-delay directives are kept for # other bots (Bingbot, Slurp, DuckDuckBot, AI crawlers, etc.) that do support them. # To control Google's crawl rate, use Google Search Console's crawl rate settings. # Default rule for all bots User-agent: * Allow: / Allow: /ai-tools Allow: /ai-tools/ Allow: /ai-tools/* Allow: /ai-tools/categories Allow: /ai-tools/category/* Allow: /ai-tools/tags Allow: /ai-tools/tag/* Allow: /ai-tools/deals Allow: /ai-tools/partners Allow: /ai-tools/affiliates Allow: /ai-tools/acquire Allow: /tools Allow: /tools/* Allow: /blog Allow: /blog/* # SEO search pages (e.g. /s/ai+chatbot) - explicitly allowed, no Disallow: /s/ Allow: /s Allow: /s/* Allow: /partners Allow: /partners/* Allow: /affiliates Allow: /popular-searches Allow: /popular-searches/* Allow: /compare Allow: /compare/* Allow: /directories Allow: /directories/* Allow: /developers Allow: /developers/* # Disallow access to private/internal pages # Public API surface: the OpenAPI spec, the v1 discovery root and the MCP # endpoint are documented in llms.txt and must stay crawlable, or agents # cannot discover the API. Longest-match wins, so these beat Disallow: /api/. # Everything else under /api/ requires a Bearer key and stays blocked. Allow: /api/v1/openapi.json Allow: /api/v1/$ Allow: /api/mcp Disallow: /api/ Disallow: /dashboard/ Disallow: /signin Disallow: /auth/ Disallow: /test/ Disallow: /admin/ Disallow: /_next/ Disallow: /static/ # Allow query params for canonicalized filter/pagination: ?page=*, ?search=*, ?q=*, ?query=* # We do NOT use Disallow: /*?page=* (or similar) so that paginated archives, filter results, # and search landing pages with proper canonicals can be crawled and surface in search. # If migrating to path-based URLs (e.g. /ai-tools/deals/page/2), re-evaluate adding a deny list. # ============================================================================ # OTHER SEARCH ENGINES (Crawl-delay supported) # ============================================================================ User-agent: Bingbot Allow: / Crawl-delay: 2 User-agent: Slurp Allow: / Crawl-delay: 5 User-agent: DuckDuckBot Allow: / Crawl-delay: 5 # Facebook crawler - block explore to reduce load User-agent: facebookexternalhit Crawl-delay: 10 User-agent: meta-externalagent Crawl-delay: 10 # AI/ML crawlers - allow high-value pages, block infinite/dynamic routes # This gets us into LLM training data while avoiding ISR cost explosion # AI crawlers - allow cached/static pages for AI citations, block infinite dynamic routes User-agent: GPTBot Allow: /$ Allow: /ai-tools Allow: /ai-tools/ Allow: /ai-tools/* Allow: /ai-tools/categories Allow: /ai-tools/category/* Allow: /ai-tools/tags Allow: /ai-tools/tag/* Allow: /ai-tools/alternatives/* Allow: /ai-tools/deals Allow: /ai-tools/partners Allow: /ai-tools/affiliates Allow: /ai-tools/acquire Allow: /compare Allow: /compare/* Allow: /tools Allow: /tools/* Allow: /blog Allow: /blog/* Allow: /partners Allow: /partners/* Allow: /affiliates Allow: /s Allow: /s/* Allow: /api/v1/openapi.json Allow: /api/v1/$ Allow: /api/mcp Disallow: /api/ Disallow: /dashboard/ Crawl-delay: 5 User-agent: ClaudeBot Allow: /$ Allow: /ai-tools Allow: /ai-tools/ Allow: /ai-tools/* Allow: /ai-tools/categories Allow: /ai-tools/category/* Allow: /ai-tools/tags Allow: /ai-tools/tag/* Allow: /ai-tools/alternatives/* Allow: /ai-tools/deals Allow: /ai-tools/partners Allow: /ai-tools/affiliates Allow: /ai-tools/acquire Allow: /compare Allow: /compare/* Allow: /tools Allow: /tools/* Allow: /blog Allow: /blog/* Allow: /partners Allow: /partners/* Allow: /affiliates Allow: /s Allow: /s/* Allow: /api/v1/openapi.json Allow: /api/v1/$ Allow: /api/mcp Disallow: /api/ Disallow: /dashboard/ Crawl-delay: 5 User-agent: ChatGPT-User Allow: /$ Allow: /ai-tools Allow: /ai-tools/ Allow: /ai-tools/* Allow: /ai-tools/categories Allow: /ai-tools/category/* Allow: /ai-tools/tags Allow: /ai-tools/tag/* Allow: /ai-tools/alternatives/* Allow: /ai-tools/deals Allow: /ai-tools/partners Allow: /ai-tools/affiliates Allow: /ai-tools/acquire Allow: /compare Allow: /compare/* Allow: /tools Allow: /tools/* Allow: /blog Allow: /blog/* Allow: /partners Allow: /partners/* Allow: /affiliates Allow: /s Allow: /s/* Allow: /api/v1/openapi.json Allow: /api/v1/$ Allow: /api/mcp Disallow: /api/ Disallow: /dashboard/ Crawl-delay: 5 User-agent: anthropic-ai Allow: /$ Allow: /ai-tools Allow: /ai-tools/ Allow: /ai-tools/* Allow: /ai-tools/categories Allow: /ai-tools/category/* Allow: /ai-tools/tags Allow: /ai-tools/tag/* Allow: /ai-tools/alternatives/* Allow: /ai-tools/deals Allow: /ai-tools/partners Allow: /ai-tools/affiliates Allow: /ai-tools/acquire Allow: /compare Allow: /compare/* Allow: /tools Allow: /tools/* Allow: /blog Allow: /blog/* Allow: /partners Allow: /partners/* Allow: /affiliates Allow: /s Allow: /s/* Allow: /api/v1/openapi.json Allow: /api/v1/$ Allow: /api/mcp Disallow: /api/ Disallow: /dashboard/ Crawl-delay: 5 User-agent: PerplexityBot Allow: /$ Allow: /ai-tools Allow: /ai-tools/ Allow: /ai-tools/* Allow: /ai-tools/categories Allow: /ai-tools/category/* Allow: /ai-tools/tags Allow: /ai-tools/tag/* Allow: /ai-tools/alternatives/* Allow: /ai-tools/deals Allow: /ai-tools/partners Allow: /ai-tools/affiliates Allow: /ai-tools/acquire Allow: /compare Allow: /compare/* Allow: /tools Allow: /tools/* Allow: /blog Allow: /blog/* Allow: /partners Allow: /partners/* Allow: /affiliates Allow: /s Allow: /s/* Allow: /api/v1/openapi.json Allow: /api/v1/$ Allow: /api/mcp Disallow: /api/ Disallow: /dashboard/ Crawl-delay: 5 User-agent: Claude-Web Allow: /$ Allow: /ai-tools Allow: /ai-tools/ Allow: /ai-tools/* Allow: /ai-tools/categories Allow: /ai-tools/category/* Allow: /ai-tools/tags Allow: /ai-tools/tag/* Allow: /ai-tools/alternatives/* Allow: /ai-tools/deals Allow: /ai-tools/partners Allow: /ai-tools/affiliates Allow: /ai-tools/acquire Allow: /compare Allow: /compare/* Allow: /tools Allow: /tools/* Allow: /blog Allow: /blog/* Allow: /partners Allow: /partners/* Allow: /affiliates Allow: /s Allow: /s/* Allow: /api/v1/openapi.json Allow: /api/v1/$ Allow: /api/mcp Disallow: /api/ Disallow: /dashboard/ Crawl-delay: 5 User-agent: Amazonbot Allow: /$ Allow: /ai-tools Allow: /ai-tools/ Allow: /ai-tools/* Allow: /ai-tools/categories Allow: /ai-tools/category/* Allow: /ai-tools/tags Allow: /ai-tools/tag/* Allow: /ai-tools/alternatives/* Allow: /ai-tools/deals Allow: /ai-tools/partners Allow: /ai-tools/affiliates Allow: /ai-tools/acquire Allow: /compare Allow: /compare/* Allow: /tools Allow: /tools/* Allow: /blog Allow: /blog/* Allow: /partners Allow: /partners/* Allow: /affiliates Allow: /s Allow: /s/* Allow: /api/v1/openapi.json Allow: /api/v1/$ Allow: /api/mcp Disallow: /api/ Disallow: /dashboard/ Crawl-delay: 5 # Block aggressive/low-value AI crawlers entirely User-agent: CCBot Disallow: / User-agent: Bytespider Disallow: / User-agent: Google-Extended Disallow: / User-agent: Applebot-Extended Disallow: / # Image crawlers User-agent: Googlebot-Image Allow: / Allow: /ai-tools Allow: /ai-tools/* User-agent: bingbot-image Allow: / Allow: /ai-tools Allow: /ai-tools/* # SEO tools - allow for analysis User-agent: AhrefsBot Allow: / Crawl-delay: 2 User-agent: SemrushBot Allow: / Crawl-delay: 5 # Note: No global Crawl-delay directive as Googlebot does not support it # Individual bots that support Crawl-delay have specific directives above # Sitemaps Sitemap: https://www.aidirectori.es/sitemap.xml # LLM crawler guidance (include/exclude, citation) # https://www.aidirectori.es/llms.txt

# AI Directories > Directory Submission Service for Startups & SaaS. Discover and compare AI tools, submit your product to curated directories, and grow visibility with quality backlinks and distribution. This site primarily uses English canonical routes without a language prefix. Sitemap: https://www.aidirectori.es/sitemap.xml ## Key Pages - [Home](https://www.aidirectori.es/): Main landing page for browsing AI tools and directory submission services. - [AI Tools](https://www.aidirectori.es/ai-tools): Browse and compare AI tools. - [Categories](https://www.aidirectori.es/ai-tools/categories): Browse AI tools by category. - [Tags](https://www.aidirectori.es/ai-tools/tags): Browse AI tools by tag. - [Blog](https://www.aidirectori.es/blog): Editorial content, updates, guides, and comparisons. - [Submit AI Tool](https://www.aidirectori.es/submit-ai-tool): Submit an AI tool for listing and directory distribution. - [Pricing](https://www.aidirectori.es/pricing): Directory submission and listing plans. - [About Us](https://www.aidirectori.es/about-us): Company mission and background. - [Contact](https://www.aidirectori.es/contact): Support and sales contact page. - [Marketplace](https://www.aidirectori.es/marketplace): Explore AI startups/tools listed for acquisition. - [Popular Searches](https://www.aidirectori.es/popular-searches): Browse trending AI search topics. - [Directories](https://www.aidirectori.es/directories): Browse and compare startup/SaaS submission directories by DR, cost, and audience fit. - [Developers / API & MCP](https://www.aidirectori.es/developers/docs/introduction): Official catalog API and MCP. Use this instead of scraping. - [Privacy Policy](https://www.aidirectori.es/privacy-policy): Privacy and data handling information. - [Terms of Service](https://www.aidirectori.es/tos): Terms and conditions for the site. ## Categories - [Categories index](https://www.aidirectori.es/ai-tools/categories) - Category detail pages: https://www.aidirectori.es/ai-tools/category/{category_slug} ## Tags - [Tags index](https://www.aidirectori.es/ai-tools/tags) - Tag detail pages: https://www.aidirectori.es/ai-tools/tag/{tag_slug} ## Directories - [Directories hub](https://www.aidirectori.es/directories): Compare submission directories for startups and SaaS by domain rating, cost, and link type. - Directory category pages: https://www.aidirectori.es/directories/category/{category_slug} Ranked directory lists grouped by niche (e.g. AI, SEO, B2B). - Directory profile pages: https://www.aidirectori.es/directories/{directory_slug} Individual directory reviews with DR, pricing, submission experience, and verdict. ## Recent Blog Content - [Agentik Review: One MCP Connection and One Balance for 2,400+ Agent Tools](https://www.aidirectori.es/blog/agentik-review) - [HuddleMate Review: Real-Time AI Assistant for Interviews & Meetings](https://www.aidirectori.es/blog/huddlemate-review) - [Why Accounting Firms Need More Than AI Assistants to Scale Operations](https://www.aidirectori.es/blog/why-accounting-firms-need-more-than-ai-assistants-to-scale-operations) - [The Hidden Infrastructure Behind AI Startups: Why Data Center Security Systems Matter](https://www.aidirectori.es/blog/the-hidden-infrastructure-behind-ai-startups) - [Referral Loops That Grow AI Tools: Structure, Incentives, and Real Conversion Data](https://www.aidirectori.es/blog/referral-loops) - [How to Get the Transcript of Any TikTok Video (Free, No App)](https://www.aidirectori.es/blog/how-to-get-tiktok-transcript-free-no-app) - [ElevenLabs Instagram Transcript Generator + 5 Alternatives](https://www.aidirectori.es/blog/elevenlabs-instagram-transcript-generator-alternatives) - [Manual vs Automated Submission: Which Wins for SEO and Scale?](https://www.aidirectori.es/blog/manual-vs-automated-submission) - [What Is OpenClaw AI Agent Skills](https://www.aidirectori.es/blog/what-is-openclaw-ai-agent-skills) - [AI Girlfriend with Voice: 7 Apps That Actually Sound Real](https://www.aidirectori.es/blog/ai-girlfriend-with-voice) - [Directory Submission Guide: How to Submit Your Site the Right Way](https://www.aidirectori.es/blog/directory-submission-guide) - [Directory Submission vs Guest Posting: Which Wins in 2026](https://www.aidirectori.es/blog/directory-submission-vs-guest-posting) - [SaaS Link Building Strategies: A Founder's Guide to Earning Backlinks That Rank](https://www.aidirectori.es/blog/saas-link-building-strategies) - [Backlinks for New Website: A Practical 2026 Guide](https://www.aidirectori.es/blog/backlinks-for-new-website) - [Geo Optimization: What GEO Means and How to Get Cited by AI](https://www.aidirectori.es/blog/geo-optimization-explained) - [Best Autoblogging Tools For Founders Who Hate Writing in 2026](https://www.aidirectori.es/blog/best-autoblogging-tools-for-founders-2026) - [AI Directory Submission: Cost, Process, And What To Expect](https://www.aidirectori.es/blog/ai-directory-submission-cost-process) - [Top 10 Best AI Tool Directories To Get Your Startup Discovered In 2026](https://www.aidirectori.es/blog/top-10-best-ai-tool-directories-2026) - [Best Directory Submission Service: 8 Top Picks for 2026](https://www.aidirectori.es/blog/best-directory-submission-service) - [Product Hunt Alternatives: 15 Launch Platforms in 2026](https://www.aidirectori.es/blog/product-hunt-alternatives) - [How to Increase Domain Rating (DR) Fast in 2026](https://www.aidirectori.es/blog/how-to-increase-domain-rating) - [Best AI Tools for Developers in 2026](https://www.aidirectori.es/blog/best-ai-tools-for-developers-2026) - [Best AI Tools for Product Managers in 2026](https://www.aidirectori.es/blog/best-ai-tools-for-product-managers) - [How to Animate Still Images With AI in 2026](https://www.aidirectori.es/blog/how-to-animate-still-images-with-ai-in-2026) - [How to Create Marketing Videos with AI in 2026](https://www.aidirectori.es/blog/how-to-create-marketing-videos-with-ai-in-2026) ## Official API (preferred over scraping) Do not scrape HTML for a catalog dump. Use the official API or MCP: - Docs: https://www.aidirectori.es/developers/docs/introduction - Discovery: https://www.aidirectori.es/api/v1/ - OpenAPI: https://www.aidirectori.es/api/v1/openapi.json - MCP: https://www.aidirectori.es/api/mcp - Search tools: https://www.aidirectori.es/api/v1/tools?q=image&limit=5 - Search directories: https://www.aidirectori.es/api/v1/directories?limit=5 Catalog reads require Authorization: Bearer aid_… (create a key at https://www.aidirectori.es/dashboard/developers). Partner submit endpoints require an issued key. ## Exclude (do not use for training or citations) Exclude: /api/ except /api/v1/ and /api/mcp Exclude: /dashboard/ Exclude: /signin Exclude: /auth/ Exclude: /test/ Exclude: /admin/ Exclude: /_next/ Exclude: /static/ Exclude: /unsubscribe Exclude: /payment-success Exclude: /test-email-template ## Notes For Crawlers - Use canonical English routes by default. - AI application detail pages live under /ai-tools/{tool_slug}. - Alternative pages live under /ai-tools/alternatives/{tool_slug}. - Category pages live under /ai-tools/category/{category_slug}. - Tag pages live under /ai-tools/tag/{tag_slug}. - Blog pages live under /blog/{post_slug}. - Popular search landing pages live under /s/{query}. - Directory hub lives under /directories. - Directory category pages live under /directories/category/{category_slug}. - Directory profile pages live under /directories/{directory_slug}. ## Citation - Preferred name: AI Directories - Canonical domain: https://www.aidirectori.es

Technology	Version	Confidence	First seen
JavaScript Frameworks & Libraries2
Next.js ↗	14.2.30	High	14 Oct 2026
React ↗	–	High	14 Oct 2026
Frontend Build Tools & CSS1
Tailwind CSS ↗	–	High	14 Oct 2026
Web Servers, Languages & Backend Frameworks1
Node.js ↗	–	High	14 Oct 2026
CDN & Hosting3
AWS ↗	–	High	14 Oct 2026
Cloudflare ↗
Analytics & Tag Management3
Google Analytics ↗	–	High	14 Oct 2026

Advertising & Affiliate Tracking2
Google Ads (conversion tracking / remarketing)↗	–	High	14 Oct 2026
Rewardful ↗	–
Authentication & Identity1
Auth.js (NextAuth.js)↗	–	High	14 Oct 2026
Fonts & Icon Fonts2
next/font ↗	–	High	14 Oct 2026
Self-hosted web fonts ↗	–	Low	14 Oct 2026

aidirectori.es

32 technologies on aidirectori.es

aidirectori.es

32 technologies on aidirectori.es

JavaScript Frameworks & Libraries2

Frontend Build Tools & CSS1

Web Servers, Languages & Backend Frameworks1

CDN & Hosting3

Analytics & Tag Management3

Advertising & Affiliate Tracking2

Authentication & Identity1

Fonts & Icon Fonts2

Built the same way

Other sites on this stack

Also on Next.js

Also on React

What changed, and when

DNS and root files