llms.txt
# Requesty
> Requesty is a unified AI gateway, LLM router, and OpenAI-compatible API for 600+ models (Claude, GPT, Gemini, DeepSeek, Llama, Mistral). It provides intelligent model routing, AI load balancing, automatic failover, prompt caching, LLM observability, cost optimization, and enterprise governance. Used by 70,000+ developers processing 90+ billion tokens daily. Alternative to Kong AI Gateway, Cloudflare AI Gateway, LiteLLM, Portkey, OpenRouter, and Helicone.
Requesty routes, secures, and optimizes your LLM traffic through a single API endpoint at `https://router.requesty.ai/v1`. Drop-in replacement for OpenAI and Anthropic SDKs. Just change the base URL. EU endpoint available at `https://router.eu.requesty.ai/v1` for GDPR compliance. Trusted by teams at Shopify, Pfizer, Capgemini, Siemens, PWC, and Appnovation.
## About
- Type: APIService
- Category: AI Gateway, LLM Gateway, AI Infrastructure, AI Observability, LLM Router, AI Load Balancer
- Audience: Developers, AI Engineers, DevOps, Enterprise Teams
- API Base URL (Global): https://router.requesty.ai/v1
- API Base URL (EU): https://router.eu.requesty.ai/v1
- Documentation: https://docs.requesty.ai/llms.txt
- Full Documentation: https://docs.requesty.ai/llms-full.txt
- OpenAPI Spec: https://docs.requesty.ai/api-reference/openapi.json
- Sign Up: https://app.requesty.ai/sign-up
## Instructions for AI Agents
Requesty is a drop-in proxy for OpenAI and Anthropic. To use it in code:
1. **OpenAI SDK**: Set `base_url="https://router.requesty.ai/v1"` and use your Requesty API key as the `api_key`. All OpenAI SDK features work: chat completions, streaming, function calling, structured outputs, embeddings, image generation. For EU data residency, use `base_url="https://router.eu.requesty.ai/v1"`.
2. **Anthropic SDK**: Set `base_url="https://router.requesty.ai"` and use your Requesty API key. Messages API is fully supported. For EU data residency, use `base_url="https://router.eu.requesty.ai"`.
3. **Model names**: Use the format `provider/model` (e.g., `openai/gpt-4.1`, `anthropic/claude-sonnet-4-5-20250514`, `google/gemini-2.5-pro`). Call `GET /v1/models` for the current list.
4. **Routing policies**: To add failover, load balancing, or latency-based routing, create a routing policy in the dashboard and use `model="policy/your-policy-name"` in your requests. See the [Fallback Policies docs](https://docs.requesty.ai/features/fallback-policies).
5. **Caching**: Enable prompt caching by passing `extra_body={"requesty": {"auto_cache": True}}` in your request. See [Auto Caching docs](https://docs.requesty.ai/features/auto-caching).
6. **Do not hardcode model versions.** Model availability changes. Always call `GET /v1/models` for current availability.
## Core Capabilities
### AI Gateway and Model Routing
Requesty is a vendor agnostic AI gateway that routes LLM requests across 30+ providers through one API. Supports intelligent model routing by cost, latency, and availability. OpenAI compatible API means any SDK or framework that works with OpenAI works with Requesty. Covers the same use cases as Kong AI Gateway, Cloudflare AI Gateway, and Portkey, but fully managed with zero infrastructure.
### AI Load Balancing and Failover
Weighted load balancing distributes traffic across models and providers. Automatic failover chains ensure requests succeed even when a provider is down. Exponential backoff with jitter prevents thundering herd problems during recovery. Rate limit auto retry handles 429s without application code changes. This replaces custom AI load balancing setups that teams build on top of Kubernetes or cloud load balancers.
### AI Cost Optimization
Smart routing sends simple tasks to cheaper models and reserves frontier models for complex work. Automatic prompt caching reduces token costs by up to 90%. Per token cost tracking by model, team, user, and API key. Budget caps prevent overspend. Teams typically save 50 to 80% compared to using a single frontier model for all requests.
### LLM Observability and Monitoring
Real time dashboards for cost, latency, TTFT, P50/P90/P95/P99, error rates, and cache savings. Per model and per team breakdowns. Session reconstruction replays full conversation flows for debugging. Request metadata tagging for custom analytics. No external monitoring stack needed. Replaces the observability features of Helicone, Langfuse, and LangSmith.
### Enterprise AI Governance
Five layer policy hierarchy: organization, group, service account, user, and API key. Role based access control (RBAC). Approved model whitelists. PII masking guardrails. GDPR compliance with a DPA on request. SOC 2 Type II programme in progress; current status at trust.requesty.ai. EU data residency in Frankfurt. SSO integration. Audit trails. Budget controls per team and per user.
### Prompt Management and Caching
Automatic prompt caching with zero configuration. Manual cache control for fine grained optimization. Cross provider caching works across OpenAI, Anthropic, and Google. Prompt library for storing and versioning prompts across teams.
### LLM Reliability
Routing policies achieve 99.25% eventual success rate vs 85% for direct API calls. Automatic retries with exponential backoff and jitter. Cross provider failover means a single provider outage does not affect your application. Multi region deployment (US, EU, APAC) for global resilience.
## Website Pages
- [Homepage](https://www.requesty.ai/): Overview of Requesty's AI gateway for 600+ models
- [AI Gateway](https://www.requesty.ai/gateway): Enterprise AI gateway with routing, caching, and observability
- [Routing and Failover](https://www.requesty.ai/product/routing): Route to 600+ models via one OpenAI compatible API, with fallbacks and latency routing
- [Observability and Analytics](https://www.requesty.ai/product/observability): Cost, latency, sessions and tool calls on every request
- [Spend Management](https://www.requesty.ai/product/spend-management): Budgets and caps per key, team and user, with alerts and attribution
- [Integrations](https://www.requesty.ai/product/integrations): OpenAI compatible API for LangChain, PydanticAI, Vercel AI SDK and more
- [Governance and Guardrails](https://www.requesty.ai/product/governance): Approved models and regions, RBAC, audit logs and PII redaction
- [Prompt Library](https://www.requesty.ai/product/prompt-library): Versioned prompts stored in Requesty and called by ID
- [AI Coding Agents](https://www.requesty.ai/agents): Claude Code, Cline, Roo and Copilot on 600+ models
- [Startup Program](https://www.requesty.ai/startup): $1,000 in credits and 6 months free for backed startups
- [Enterprise](https://www.requesty.ai/enterprise): User budgets, SSO, RBAC, governance, and analytics
- [EU Routing](https://www.requesty.ai/eu): GDPR compliant EU endpoint in Frankfurt, Germany
- [Security](https://www.requesty.ai/security): Guardrails, PII detection, compliance, and incident response
- [Models](https://www.requesty.ai/models): Browse all 600+ supported models by provider
- [Free Models](https://www.requesty.ai/models/free): Free AI models at $0 per token, with 200 free requests per day through one OpenAI-compatible API
- [Pricing](https://www.requesty.ai/pricing): Pay as you go pricing at a 5% markup on model cost, or 0% with your own provider keys
- [Blog](https://www.requesty.ai/blog): Deep dives, announcements, and feature releases
- [What is an AI Gateway?](https://www.requesty.ai/ai-gateway): Complete guide to AI gateways with feature comparison and use cases
- [AI Cost Optimization](https://www.requesty.ai/ai-cost-optimization): How to cut LLM spend by up to 80% with routing, caching, and model diversification
- [LLM Cost Calculator](https://www.requesty.ai/tools/llm-cost-calculator): Free interactive calculator to compare LLM API costs across 12 models
- [LLM Provider Status](https://www.requesty.ai/tools/llm-provider-status): Links to status pages for all major LLM providers
- [AI Gateway Glossary](https://www.requesty.ai/glossary): 28 definitions covering AI gateway, LLM routing, and infrastructure terms
- [AI Gateway Buyer's Guide](https://www.requesty.ai/buyers-guide): Compare 8 AI gateways side by side with evaluation checklist
## Documentation
- [Quickstart](https://docs.requesty.ai/quickstart): Setup guide. Route AI requests through Requesty in under 2 minutes
- [API Overview](https://docs.requesty.ai/api-reference/overview): Authentication, base URL, and error handling
- [EU Routing Guide](https://docs.requesty.ai/features/eu-routing): Full EU setup with approved models workflow
- [Fallback Policies](https://docs.requesty.ai/features/fallback-policies): Configure automatic model failover chains
- [Load Balancing](https://docs.requesty.ai/features/load-balancing-policies): Distribute traffic across models with weighted routing
- [Latency Routing](https://docs.requesty.ai/features/latency-routing): Automatically route to the fastest model
- [Auto Caching](https://docs.requesty.ai/features/auto-caching): Reduce costs with automatic prompt caching
- [Spend Limits](https://docs.requesty.ai/features/api-limits): Budget caps per API key or project
- [Bring Your Own Keys](https://docs.requesty.ai/features/bring-your-own-keys): Use your own provider API keys
- [Streaming](https://docs.requesty.ai/features/streaming): Real-time SSE streaming
- [Structured Outputs](https://docs.requesty.ai/features/structured-outputs): Enforce JSON schema responses across LLMs
- [Reasoning](https://docs.requesty.ai/features/reasoning): Extended thinking tokens with unified effort levels across providers
- [Image Generation](https://docs.requesty.ai/features/image-generation): Generate images via DALL-E, Stable Diffusion, and more
- [Image Understanding](https://docs.requesty.ai/features/image-understanding): Send images to models for analysis
- [PDF Support](https://docs.requesty.ai/features/pdf-support): Analyze PDF documents with multimodal models
- [Web Search](https://docs.requesty.ai/features/web-search): Enable models to search the web in real-time
- [Guardrails](https://docs.requesty.ai/features/guardrails): PII detection, prompt injection blocking, content filtering
- [Usage Analytics](https://docs.requesty.ai/features/usage-analytics): Real time dashboard for cost, requests, tokens, latency, and cache savings
- [Cost Tracking](https://docs.requesty.ai/features/cost-tracking): Per-model, per-user, per-key cost breakdowns
- [Performance Monitoring](https://docs.requesty.ai/features/performance-monitoring): Latency percentiles (P50/P90/P95/P99) and error rates
- [Request Metadata](https://docs.requesty.ai/features/request-metadata): Custom key-value metadata for analytics filtering
- [Session Reconstruction](https://docs.requesty.ai/features/session-reconstruction): Replay full conversation sessions for debugging
- [RBAC](https://docs.requesty.ai/features/rbac): Role-based access control across all features
- [Approved Models](https://docs.requesty.ai/features/approved-models): Whitelist which models your organization can use
- [MCP Gateway](https://docs.requesty.ai/features/mcp-gateway): Unified gateway for MCP servers
## Integrations
- [Claude Code](https://docs.requesty.ai/integrations/claude-code): Route Claude Code through Requesty
- [Cline](https://docs.requesty.ai/integrations/cline): Connect Cline AI agent to Requesty
- [Roo Code](https://docs.requesty.ai/integrations/roo-code): Connect Roo Code for multi-model access
- [VS Code Extension](https://docs.requesty.ai/integrations/VS-code-extension): Switch between LLMs inside VS Code
- [OpenClaw](https://docs.requesty.ai/integrations/openclaw): Connect OpenClaw to 600+ models
- [Anthropic Agent SDKs](https://docs.requesty.ai/integrations/anthropic-agent-sdks): Requesty as backend for Anthropic Agent SDK
- [LibreChat](https://docs.requesty.ai/integrations/librechat): Self-hosted chat UI with Requesty
- [OpenWebUI](https://docs.requesty.ai/integrations/openwebui): OpenWebUI with multi-model chat
## Frameworks
- [OpenAI SDK](https://docs.requesty.ai/frameworks/openai): Drop-in base_url swap for OpenAI Python/JS SDK
- [LangChain](https://docs.requesty.ai/frameworks/langchain): Use Requesty in LangChain chains and agents
- [PydanticAI](https://docs.requesty.ai/frameworks/pydantic-ai): Typed LLM responses with PydanticAI
- [Vercel AI SDK](https://docs.requesty.ai/frameworks/vercel-ai-sdk): Official Requesty provider for Vercel AI SDK
- [Haystack](https://docs.requesty.ai/frameworks/haystack): Integrate with Haystack pipelines
- [Python Requests](https://docs.requesty.ai/frameworks/requests): Direct HTTP calls with requests
- [Axios](https://docs.requesty.ai/frameworks/axios): JavaScript HTTP calls with Axios
- [LlamaIndex TS](https://docs.requesty.ai/frameworks/llamaindex-ts): Use with LlamaIndex TypeScript
## Comparisons
- [Requesty vs Kong AI Gateway](https://www.requesty.ai/vs/kong-ai-gateway): Enterprise AI gateway comparison. Purpose built LLM routing vs general API gateway
- [Requesty vs Cloudflare AI Gateway](https://www.requesty.ai/vs/cloudflare-ai-gateway): AI load balancing and fallback comparison. Smart routing vs basic proxying
- [Requesty vs OpenRouter](https://www.requesty.ai/vs/openrouter): Feature by feature comparison
- [Requesty vs LiteLLM](https://www.requesty.ai/vs/litellm): Managed gateway vs self hosted proxy
- [Requesty vs Helicone](https://www.requesty.ai/vs/helicone): Observability comparison
- [Requesty vs Portkey](https://www.requesty.ai/vs/portkey): Gateway feature comparison
- [LLM Gateway Alternatives](https://www.requesty.ai/llm-gateway-alternatives): Overview of all LLM gateway options
## Use Cases
- Route LLM requests by cost and latency for optimal model selection
- Replace direct OpenAI/Anthropic API calls with multi provider failover
- Manage API keys across multiple LLM providers with a single key
- Set up AI load balancing across models without custom infrastructure
- Enforce budget caps and spend limits per team, user, or API key
- Add LLM observability to existing applications without code changes
- Enable EU data residency for GDPR compliant AI workloads
- Centralize prompts for chatbots and agents across teams
- Build disaster recovery for LLM heavy applications with cross provider failover
- Implement global rate limiting for LLM APIs across providers
- Run A/B tests across LLM routers and models
- Evaluate alternatives to AWS Bedrock for model routing
## Blog: Guides and Best Practices
- [How to Route LLM Requests by Cost and Latency](https://www.requesty.ai/blog/how-to-route-llm-requests-by-cost-and-latency): Practical guide to intelligent model routing
- [Vendor Agnostic AI Gateway Options in 2026](https://www.requesty.ai/blog/vendor-agnostic-ai-gateway-options-2026): Comparison of all major AI gateway options
- [Alternatives to OpenAI for High Volume Workloads](https://www.requesty.ai/blog/alternatives-to-openai-for-high-volume-workloads): How to diversify providers and cut costs at scale
## Open Data
Requesty publishes its own production-gateway usage data as a free, citable research catalog at https://www.requesty.ai/data. Each note has a permanent URL with an interactive chart, key findings, caveats, and three machine-readable downloads (`data.json`, `data.csv`, `data.md`). All notes are released under CC BY 4.0, free to cite with attribution.
- Catalog homepage: https://www.requesty.ai/data
- Catalog index for AI agents: https://www.requesty.ai/data/llms.txt
- Full inline content of every note: https://www.requesty.ai/data/llms-full.txt
- License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
### Agentic
- [finish_reason mix per provider, April 2026](https://www.requesty.ai/data/finish-reason-mix-by-provider-april-2026): Which AI providers serve the most agentic traffic? Anthropic-direct returned `finish_reason = tool_calls` on 52% of successful April 2026 completions on the Requesty gateway, about 2× the next provider and 17× higher than OpenAI direct. OpenAI Responses (26%), Vertex (Claude) (23%) and Azure (23%) form a clear second tier; Vertex (Claude) at 23% vs Vertex (Gemini) at 13% shows the gap inside that route.
- [finish_reason mix per model, April 2026](https://www.requesty.ai/data/finish-reason-mix-by-model-april-2026): Which AI models are used most for tool calling? Claude Opus 4.6 returned `tool_calls` 59% of the time, Gemini 2.5 Flash 37%, while same-family Claude Sonnet 4.5 came in at 9% and the entire OpenAI lineup (GPT-4o, 4.1-mini/nano, 5-mini) sat under 4%.
- [Token-weighted tool_calls share per provider, April 2026](https://www.requesty.ai/data/tool-call-token-share-april-2026): What share of LLM output tokens is spent on tool calls vs chat? Anthropic emitted 38.8% of output tokens on tool_calls vs 54.2% of requests, so agentic completions run ~30% smaller than chat. OpenAI Responses showed the opposite (34.2% of tokens vs 26.4% of requests); Vertex (Claude) had the biggest negative gap (6.1% of tokens vs 27.6% of requests).
- [Reasoning-token share of provider output, April 2026](https://www.requesty.ai/data/reasoning-token-share-by-provider-april-2026): How much of LLM output is reasoning/thinking tokens? Groq led at 82%, followed by Coding (79%), xAI (60%), z.ai (51%). Frontier routes ran ~one-third: Vertex (Gemini) 40%, OpenAI 36%, OpenAI Responses 33%. Anthropic and Bedrock report 0% because extended thinking is delivered inline.
- [Family share within OSS-routed traffic, Nov 2025 - Apr 2026](https://www.requesty.ai/data/oss-family-share-jan-apr-2026): Which open-weight AI model is most popular in 2026? OSS-routed traffic went from Qwen-dominated (34-38% in Nov-Dec 2025) to DeepSeek-dominated (77% after the R1 launch in January) and back to a diversified state by April (DeepSeek 47%, Kimi 17%, MiniMax 15%). Qwen collapsed from 38% to under 4% almost overnight.
### Latency
- [Latency leaderboard per provider, April 2026](https://www.requesty.ai/data/provider-latency-leaderboard-april-2026): Which AI provider has the lowest latency in April 2026? xAI led p50 at 0.6 s (Novita 0.8, Azure 1.0, Mistral 1.4); Vertex (Claude) was slowest at 13.7 s, 23× the fastest and 2.8× slower than Vertex (Gemini) on the same Vertex route. Anthropic-direct sat mid-pack at 5.8 s with a 52.6 s p95 long tail.
- [Provider throughput density, April 2026](https://www.requesty.ai/data/provider-throughput-density-april-2026): How many tokens per second can each LLM provider sustain? Groq led at 320 output tok/sec, 2.5× the next-fastest, attributable to its custom inference silicon. Vertex (Gemini) 130, Mistral 120; OSS aggregator routes clustered at 23-26 tok/sec; Bedrock was slowest at 15 tok/sec, 21× behind Groq.
- [Streaming TTFT vs total latency, April 2026](https://www.requesty.ai/data/streaming-ttft-vs-total-april-2026): Which AI provider has the fastest time-to-first-token? Azure led TTFT at 593 ms with a 960 ms p50 total, the streaming-UX winner on both axes. xAI was among the fastest on total (5.68 s) but slowest to first token (3.27 s), suggesting buffered upstream behaviour. Vertex (Gemini) totals 3.05 s, Claude totals 8.03 s on the same route.
- [p50 latency YoY: April 2025 vs April 2026](https://www.requesty.ai/data/provider-latency-yoy-april-2026): Has LLM latency improved over the past year? Open-source aggregator routes compressed dramatically: xAI -93% (9.1 s → 0.6 s), DeepInfra -91%, DeepSeek -62%. Frontier providers barely moved (OpenAI -5%, Anthropic 0%). Vertex (Claude) is the only major route that got slower (+131%) as agentic Claude Code workloads landed on it.
- [Prompt-cache hit rate per provider, April 2026](https://www.requesty.ai/data/cache-hit-rate-by-provider-april-2026): Which AI providers have the highest prompt-cache hit rate? Anthropic-direct led at 77%, Bedrock Claude 57%, Vertex (Claude) 24%. Same Claude model family, 3× lower hit rate. Vertex (Gemini) 10% and Mistral 4% sat at the floor among major routes.
