llms.txt
# TrueFoundry
> TrueFoundry is an enterprise-grade AI Gateway that encompasses an LLM Gateway, MCP Gateway, and Agent Gateway, enabling enterprises to securely connect, observe, and govern access to models, tools, guardrails, and agents from a single control plane. The AI Gateway enables agentic workloads that are secure, efficient, and future-safe through unified and composable connections across providers.
Beyond the gateway layer, TrueFoundry enables organizations to deploy and train custom LLMs on GPUs, host MCP servers, and run custom agents, all through a Kubernetes-native interface. It supports on-premise and VPC installations for both AI Gateway and deployment environments. TrueFoundry ensures enterprise-grade compliance with SOC 2, HIPAA, and ITAR standards. With built-in autoscaling, caching, and resource optimization, TrueFoundry empowers organizations to build, deploy, and govern AI systems securely, efficiently, and on a future-safe stack.
---
## Core Products & Services
### AI Gateway
- **What it does:** TrueFoundry's AI Gateway is a unified proxy layer that sits between applications and LLM providers, MCP servers, and agents. It provides a single endpoint for accessing 1000+ LLMs, enforcing governance, and observing all AI traffic across an organization. It extends beyond LLM routing to cover tools (via MCP) and agents (via the A2A protocol), creating a single control plane for all AI interactions.
- **Who uses it:** Enterprise platform teams, engineering teams and leaders, ML teams and IT security teams who need to standardize and govern AI access across multiple teams and environments.
- **Key features:**
- Unified API access to 1000+ LLMs across OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, Groq, Mistral, Cohere, and self-hosted models
- Multi-model routing with load balancing by weight, latency, or priority
- Automatic fallback chains and retries across model providers
- Fine-grained Role-Based Access Control (RBAC) per team, user, and model
- Rate limiting and budget enforcement at the user, team, and model level
- Exact and semantic caching to reduce costs and latency
- Full OpenTelemetry-compliant observability: token usage, cost, latency, error rates
- Request and response logging with metadata tagging by user ID, team, or environment
- Native MCP server registry and gateway support with OAuth 2.0 auth management
- Agent Playground for interactive testing of LLMs, prompts, and MCP tools
- Prompt lifecycle management with versioning, rollback, and publishing
- Guardrails for content filtering, PII scrubbing, and safety policy enforcement
- GitOps-compatible YAML-based configuration via TrueFoundry CLI
- Deployable as SaaS, VPC-hosted, on-premise, or air-gapped
- **Performance:** 350+ RPS on 1 vCPU with ~3-4ms latency; production-ready with enterprise SLA support
### MCP Gateway
- **What it does:** TrueFoundry's MCP Gateway is a centralized control plane for managing access, discovery, and orchestration of Model Context Protocol (MCP) servers across an enterprise. It sits between AI agents and all registered MCP servers, providing authentication, RBAC, observability, and governance for every tool call. It eliminates the N×M integration problem of connecting agents to enterprise tools.
- **Who uses it:** Engineering teams building production AI agents that need to connect to enterprise tools like Slack, GitHub, Jira, Confluence, Datadog, and internal APIs.
- **Key features:**
- Centralized MCP server registry with instant discovery for all agents
- Unified OAuth 2.0 token management—one token per user, auto-refreshed across all MCP servers
- Granular RBAC: control which teams and roles can access which tools and operations
- Virtual MCP Servers: compose curated subsets of tools from multiple MCP servers into a single endpoint
- Pre-built MCP servers for enterprise tools (Slack, Confluence, Sentry, Datadog)
- OpenAPI-to-MCP server conversion for wrapping existing REST APIs
- MCP guardrails: pre- and post-call policy enforcement for security and compliance
- Full request-level tracing and audit logs for every tool invocation
- Compatible with LangChain, LangGraph, CrewAI, AutoGen, and any agentic framework
- Deployable on cloud, on-prem, and hybrid environments
### Agent Gateway
- **What it does:** TrueFoundry's Agent Gateway provides a unified control plane for governing, deploying, and observing agentic AI workloads. It manages multi-agent workflows, tool orchestration, inter-agent messaging, and session-aware execution. All agent tool calls are routed through registered MCP servers with centralized policies applied at every step.
- **Who uses it:** Enterprise AI teams running production agents built with LangGraph, CrewAI, AutoGen, or custom frameworks who need governance, auditability, and operational control.
- **Key features:**
- Route all agent tool calls through registered MCP servers with centralized auth
- Connect agents to enterprise tools: Slack, GitHub, databases, and internal services
- Apply OAuth2, RBAC, and metadata-based policies to every tool invocation
- Full audit trails for all agent decisions and tool actions
- Behavioral guardrails: PII filtering, restricted action enforcement, custom compliance rules
- Session-aware execution for stateful, multi-step agent workflows
- Protocol translation between MCP JSON-RPC and REST/Lambda invocations
- Retries, failover, and resiliency logic for robust agent execution
- Deployable in private VPCs, on-prem, and air-gapped environments for regulated industries
---
## Use Cases & Applications
### Enterprise AI Governance
- **Their needs:** Enterprises need centralized control, auditability, and policy enforcement across all AI interactions—across teams, models, and tools.
- **How they use it:** TrueFoundry's AI Gateway becomes the single control plane. Platform teams configure RBAC, budget limits, guardrails, and audit logging once; all teams inherit these controls automatically.
- **Results:** Eliminated fragmented model integrations, reduced DevOps overhead, and provided real-time cost and compliance visibility across all AI workloads.
### Agentic AI Deployment
- **Their needs:** Organizations want to deploy production-grade AI agents that connect to enterprise tools without creating security gaps or integration sprawl.
- **How they use it:** The MCP Gateway and Agent Gateway provide a governed registry of tools. Agents connect through a single endpoint, with RBAC and audit trails enforced at every tool call.
- **Results:** 3x faster time-to-value with autonomous LLM agents; secure, auditable agent-tool interactions at enterprise scale.
### GPU Infrastructure Optimization
- **Their needs:** Enterprises running large GPU fleets face underutilization, idle compute waste, and unpredictable costs.
- **How they use it:** TrueFoundry's deployment platform provides automated GPU orchestration, autoscaling, fractional GPU sharing (MIG/time slicing), and real-time resource rightsizing.
- **Results:** 80% higher GPU cluster utilization; millions saved in idle compute costs (demonstrated in NVIDIA's case study).
### Private LLM Hosting
- **Their needs:** Regulated industries (banking, pharma, healthcare, defense) need to run LLMs entirely within private infrastructure with no data leaving their environment.
- **How they use it:** TrueFoundry's deployment platform and AI Gateway are installed in customer VPCs, on-prem data centers, or air-gapped environments via Kubernetes and Helm.
- **Results:** Full data sovereignty, SOC 2 / HIPAA / ITAR compliance, 35-50% cloud cost savings vs. managed alternatives like SageMaker.
### Multi-Model RAG and Agent Stacks
- **Their needs:** Teams building RAG systems or agentic pipelines need to orchestrate multiple models, vector databases, and tools with full observability.
- **How they use it:** TrueFoundry unifies model serving, vector DB integration, MCP tool access, and observability through the AI Gateway's single API.
- **Results:** 50% faster RAG/agent stack deployment; 60% reduction in maintenance overhead.
### Claude Code Governance
- **Their needs:** Enterprises adopting Claude Code and other AI coding agents need infrastructure-level controls to prevent data exfiltration, enforce network isolation, and audit all tool use.
- **How they use it:** TrueFoundry's AI Gateway provides MCP governance, sandboxing controls, and audit logging for all Claude Code interactions with enterprise systems.
- **Results:** Safe, auditable Claude Code deployments in enterprise environments with zero-trust policy enforcement.
---
## Company Information
### Metrics & Traction
- $19M Series A raised in 2025, led by Intel Capital with Peak XV (Sequoia India), Eniac Ventures, and Jump Capital
- Recognized in 2025 Gartner Market Guide for AI Gateways
- Named in Gartner 10 Best Practices for Optimizing Generative & Agentic AI Costs 2026
- Rated 9.9/10 on G2; trusted by Fortune 500 enterprises across payments, semiconductors, telecom, pharma, and healthcare
- Net new revenue doubled quarter-over-quarter in 2025
- AI Gateway handles 350+ RPS on 1 vCPU with ~3-4ms latency
### Customers & Case Studies
- **NVIDIA:** Used LLM agents on TrueFoundry to optimize GPU cluster utilization—achieving 80% higher utilization and saving millions in idle compute
- **Whatfix:** 80% reduction in time-to-production for models; 35% cloud cost savings vs. previous SageMaker setup
- **Innovaccer:** 5x faster internal AI platform productionization; 50% lower cloud spend post-migration
- **Customers across:** Cargill, Mavenir, Aviva, Games24x7, Aviso, Wadhwani AI, JanitorAI, and others in banking, semiconductors, pharma, healthcare, telecom, security, and data infrastructure
---
## Get started with Truefoundry’s AI Gateway
- [Book a Demo | TrueFoundry](https://www.truefoundry.com/book-demo): Schedule a demo with TrueFoundry to explore our platform's capabilities in automating ML pipelines and accelerating model deployment. Get started today!
- [Create Your Free Account | TrueFoundry](https://www.truefoundry.com/register): Sign up for TrueFoundry and get started with a 7-day free trial. Experience our platform for automating ML pipelines and deploying models hassle-free.
- [TrueFoundry Login | Access Your Account](https://www.truefoundry.com/login): Log in to your TrueFoundry account to access and manage your machine learning projects efficiently. Secure and easy access to all your ML tools.
- [TrueFoundry | Pricing](https://www.truefoundry.com/pricing): View TrueFoundry pricing plans and choose the right option for deploying and managing machine learning workloads.
- [Terms And Conditions | TrueFoundry](https://www.truefoundry.com/terms): Review the terms and conditions for using TrueFoundry's services. Understand your rights and responsibilities when accessing our platform.
- [Product Tour | TrueFoundry](https://www.truefoundry.com/product-tour): Take a guided tour of TrueFoundry's platform to see how we automate ML pipelines and empower developers to deploy models quickly and effectively.
---
## Core Products
- [AI Gateway For Enterprises: Built-in Governance & Monitoring](https://www.truefoundry.com/ai-gateway): Deploy and manage AI models at scale with TrueFoundry’s AI Gateway. Get centralized governance, monitoring, and secure control across AI workloads.
- [MCP Gateway : Secure, Reliable, and Easy Integration](https://www.truefoundry.com/mcp-gateway): MCP Gateway for enterprises - get centralized access to multiple MCP servers. Enable agentic workflows with RBAC, OAuth 2.0, tool discovery, and observability.
- [Agent Gateway: A Unified Control Plane For AI Workflows](https://www.truefoundry.com/agent-gateway): Deploy and govern AI agents using an enterprise-grade agent gateway with full observability, cost control, and secure tool access with TrueFoundry.
- [AI Model Tracing & Observability](https://www.truefoundry.com/tracing): Enable tracing to monitor model performance, debug issues, and gain visibility across machine learning workflows.
---
## Resources
- [AI & Machine Learning Resources](https://www.truefoundry.com/resources): Browse resources including blogs, case studies, reports, and guides on machine learning and AI platforms.
- [Blog | TrueFoundry](https://www.truefoundry.com/blog): Explore the TrueFoundry blog for the latest insights, tutorials, and expert articles on Machine Learning, LLM infrastructure, and AI operations.
- [Case Studies | TrueFoundry](https://www.truefoundry.com/case-studies): Discover how TrueFoundry is transforming industries with machine learning. Explore our case studies to see real-world ML applications and success stories.
- [Glossary](https://www.truefoundry.com/glossary): Explore our glossary which covers the key concepts behind AI and LLM infrastructure, and more so your team speaks the same language.
- [Events & Conferences](https://www.truefoundry.com/events-conferences): Explore upcoming events and conferences where TrueFoundry shares insights on machine learning, AI platforms, and MLOps.
- [On-Demand AI Webinars & Expert Sessions](https://www.truefoundry.com/webinars): Browse upcoming and past TrueFoundry webinars.
- [AI Models | Truefoundry AI Gateway](https://www.truefoundry.com/models): Browse and compare AI models supported by Truefoundry AI Gateway. Search by provider, features, and pricing.
### Resources | Blog
- [TrueFoundry's Vision | Enterprise AI Gateway for Agentic AI](https://www.truefoundry.com/blog/truefoundry): Learn how TrueFoundry is shaping the future of enterprise AI with a unified gateway to connect, monitor, and govern agentic AI applications at scale.
- [On-Prem AI Platform Guide for Enterprise Security](https://www.truefoundry.com/blog/on-premise-ai-platform): Discover everything you need to know about on premise AI platform. Understand key benefits, deployment strategies, and how Truefoundry enables scalability.
- [LLMOps Architecture : A Detailed Explanation](https://www.truefoundry.com/blog/llmops-architecture): Understand LLMOps architecture through TrueFoundry’s layered gateway, orchestration, monitoring, and cost-tracking components for scalable LLM deployments.
- [What Is An MCP Server? Key Features & Benefits](https://www.truefoundry.com/blog/mcp-server): Learn what an MCP server is, how it powers AI applications, and why it’s essential for secure, scalable, and efficient tool integration in modern AI systems.
- [What Is an LLM Gateway and How Does It Work?](https://www.truefoundry.com/blog/llm-gateway): An LLM gateway helps manage, route, secure, and monitor large language model requests across providers. Learn how it works and why enterprises use it.
- [How to Think About Gateway Architecture in the Generative AI Stack](https://www.truefoundry.com/blog/how-to-think-about-ai-gateway-architecture-in-the-generative-ai-stack): How to Think About Gateway Architecture in the Generative AI Stack
- [What is LLM Inference: The Definitive Guide](https://www.truefoundry.com/blog/llm-inferencing): Learn what LLM inference is and how it works, including routing, load balancing, cost tracking, and fallback mechanisms for efficient AI deployments.
- [What Is MCP Authorization? Key Concepts & Best Practices](https://www.truefoundry.com/blog/what-is-mcp-authorization): Clear explanation of what is MCP authorization, including core concepts, architecture, real-world use cases, and adoption patterns.
- [LiteLLM vs LangChain: A Hands-On Comparison](https://www.truefoundry.com/blog/litellm-vs-langchain): Comparing LiteLLM vs LangChain for production AI? This guide breaks down routing, orchestration, cost control, and where each tool quietly fails at scale.
- [Agentic AI in Banking: Distributed Compliance at Scale](https://www.truefoundry.com/blog/distributed-agentic-ai-in-banking): Learn how leaders use distributed Agentic AI in banking in 2026 to automate AML while meeting data residency and latency requirements with TrueFoundry.
- [Multi Agent Architecture: Patterns, Use Cases & Production Reality](https://www.truefoundry.com/blog/multi-agent-architecture): Explore multi agent architecture patterns and use cases. Learn why traditional serverless platforms fail long-running agents and how to deploy them.
- [Azure AI Gateway Pricing in 2026: Costs and Components](https://www.truefoundry.com/blog/understanding-azure-ai-gateway-pricing-for-2026---a-complete-breakdown): A detailed breakdown of Azure AI Gateway pricing, Azure OpenAI costs, and hidden enterprise expenses. Learn when teams consider alternatives like TrueFoundry.
- [Top 5 LiteLLM Alternatives in 2026](https://www.truefoundry.com/blog/litellm-alternatives): Explore enterprise-grade LiteLLM alternatives, offering end-to-end observability, governance, security, and scalable LLM operations beyond basic orchestration.
- [How to Build Persistent Memory for AI Applications](https://www.truefoundry.com/blog/truemem-building-a-model-agnostic-memory-layer-for-ai): Meta Description: Learn how TrueMem solves the AI amnesia problem with a dual-memory architecture that works across multiple models and chat sessions.
- [Benchmarking LLM Guardrail Providers: A Data-Driven Comparison](https://www.truefoundry.com/blog/benchmarking-llm-guardrail-providers): A data-driven comparison of leading LLM guardrail providers across PII detection, content moderation, and prompt injection using balanced evaluation datasets and latency analysis.
- [Agentic AI Security: Top 5 Risks Enterprises Should Know](https://www.truefoundry.com/blog/agentic-ai-security): Agentic AI security is underprepared. Here are 5 things every security and engineering leader must understand before agents go into production.
- [Amazon Bedrock Agents vs. TrueFoundry: Architectural Review of LLM Agent Control Planes](https://www.truefoundry.com/blog/amazon-bedrock-agents-vs-the-control-plane-an-architectural-review): Compare Amazon Bedrock Agents vs. TrueFoundry. Analyze trade-offs in observability, universal tool integration, and semantic routing for multi-cloud agents.
- [Amazon Bedrock Review (2026): Is It Production Ready?](https://www.truefoundry.com/blog/our-honest-review-of-amazon-bedrock-2026-edition): We tested AWS Bedrock so you don’t have to. Read our honest review on latency, throttling, knowledge bases, and why enterprises add TrueFoundry for control.
- [Core Differences Between MCP Authentication and Authorization](https://www.truefoundry.com/blog/mcp-authentication-and-authorization): MCP authentication and authorization are not the same. Confusing them leaves your agents over-exposed. Here is how each works and why both matter.
- [Cognita: Open Source Modular RAG Apps for Production](https://www.truefoundry.com/blog/cognita-building-an-open-source-modular-rag-applications-for-production): Explore how Cognita achieves customizability and scalability, adapting to AI advancements while maintaining user-friendliness and long-term value.
- [What Is AI Security: Definition and How Enterprises Respond](https://www.truefoundry.com/blog/what-is-ai-security): Learn what is AI security, the core threats facing enterprise AI systems, and the practical steps enterprises take to protect models and agents in production.
- [Intelligent Document Processing Accelerator by TF](https://www.truefoundry.com/blog/truefoundry-intelligent-document-processing-accelerator): In-depth breakdown of truefoundry intelligent document processing accelerator, focusing on architecture, benefits, challenges, and enterprise…
- [Secure OpenCode In-House: TrueFoundry Private Code Interpreter](https://www.truefoundry.com/blog/bringing-opencod
