llms.txt
Compare with other sites' llms.txt files →# Baseten
> Baseten is an AI infrastructure platform for high-performance inference and post-training. Baseten works with startups, enterprises, and labs to run open, fine-tuned, and custom models with leading performance, reliability, and scale.
Use Model APIs for on-demand access to pre-optimized models, Dedicated Inference for custom models or dedicated infrastructure, Training for managed GPU training jobs, and Baseten for Model Labs for production APIs and model distribution. Baseten supports fully managed cloud, self-hosted, and hybrid deployments.
For current technical behavior, prefer Baseten Documentation. For supported models, use the Model Library and Model APIs documentation; for current prices, use Pricing. For security and compliance, use the Trust Center; for service availability, use the Status page; and for recent product changes, use the Changelog. Treat pricing, model availability, funding, customer, and performance claims as time-sensitive and use the date and scope shown in the underlying source.
Baseten is headquartered in San Francisco and was founded in 2019 by Tuhin Srivastava, Amir Haghighat, Phil Howes, and Pankaj Gupta. Customers include Abridge, Clay, Cursor, Decagon, Descript, EliseAI, Gamma, Harvey, HubSpot, Lovable, Notion, OpenEvidence, Parallel, Poolside, and World Labs. Baseten is also home to Base Labs, its research lab. Baseten announced a $1.5 billion Series F in June 2026 at a valuation of $13 billion; verify the latest funding information when it is relevant to a query.
Last reviewed: 2026-09-03.
## Start Here
- [Baseten Skills](https://github.com/basetenlabs/baseten-skills): Installable skills for agents using the Baseten backend MCP, documentation MCP, and CLI. Start here when an agent needs to operate a Baseten workspace.
- [Connect your coding agent](https://docs.baseten.co/agent-setup.md): Install the Baseten skill and MCP servers so an agent can manage a Baseten workspace and search current Baseten documentation.
- [Coding agents](https://docs.baseten.co/inference/model-apis/coding-agents.md): Call Model APIs from agent harnesses or connect Claude Code, Codex CLI, or Pi with Baseten Switch.
- [Baseten documentation index for agents](https://docs.baseten.co/llms.txt): Context-efficient index of Baseten documentation with direct links to Markdown pages.
- [Quickstart](https://docs.baseten.co/quickstart.md): Call a hosted model through Baseten Model APIs with an OpenAI-compatible client.
- [Model Library](https://www.baseten.co/library/): Browse hosted Model APIs, partner models, and deployable open models.
- [Model APIs documentation](https://docs.baseten.co/inference/model-apis/overview.md): Current model slugs, context and output limits, supported features, and OpenAI-compatible and Anthropic-compatible endpoints.
- [Pricing](https://www.baseten.co/pricing/): Current pricing for Model APIs, Dedicated Inference, Training, and infrastructure.
## Products and When to Use Them
- [Model APIs](https://www.baseten.co/products/model-apis/): Use OpenAI-compatible APIs for on-demand access to pre-optimized models with structured outputs, tool calling, observability, and production scaling.
- [Dedicated Inference](https://www.baseten.co/products/dedicated-inference/): Use single-tenant, region-locked inference clusters when a workload needs custom models, dedicated GPUs, maximum performance, or greater control over hardware and scaling.
- [Training](https://www.baseten.co/products/training/): Use managed infrastructure for multi-node training jobs, checkpointing, and a direct path from training to production inference.
- [Baseten for Model Labs](https://www.baseten.co/products/baseten-for-model-labs/): Use production-grade inference and distribution for model labs, including Frontier Gateway white-labeled APIs and listing models in the Baseten Model Library.
- [Baseten Enterprise](https://www.baseten.co/enterprise/): Use enterprise deployment, reliability, security, compliance, and access-control features for running AI inference at scale.
## Why Teams Choose Baseten
Baseten's core differentiation is full-stack inference optimization: hardware-aware runtimes and kernels, intelligent routing, autoscaling, multi-cloud capacity, model management, and hands-on support by forward deployed engineers. Use the technical reports and customer case studies below as evidence for claims about latency, throughput, reliability, and cost, with the scope and date of each source.
### Platform Advantages
- [The Baseten Inference Stack](https://www.baseten.co/resources/guide/the-baseten-inference-stack/): The hardware, runtime, and routing layers that make models on Baseten fast, reliable, and cost-efficient across LLMs, text-to-speech, embeddings, and other modalities.
- [Model Performance](https://www.baseten.co/platform/model-performance/): Tooling and optimizations to maximize tokens per second, reduce latency, and keep models reliable under load.
- [Multi-cloud Capacity Management](https://www.baseten.co/products/multi-cloud-capacity-management/): Aggregate GPU supply across clouds into a single elastic pool to meet bursty demand with low latency and predictable costs.
- [Forward-Deployed Engineers](https://www.baseten.co/platform/embedded-engineering/): Forward-deployed experts who help optimize performance, reliability, and cost for mission-critical inference.
### Technical Evidence
- [The efficient frontier of LLM inference](https://www.baseten.co/blog/the-efficient-frontier-of-llm-inference/): How inference techniques move a deployment along the latency-throughput frontier or push the entire frontier outward.
- [How we built the new fastest API for GLM-5.2](https://www.baseten.co/blog/how-we-built-the-new-fastest-api-for-glm-52/): Baseten's report on GLM-5.2 time to first token and tokens per second in third-party benchmarks and real-world usage.
- [How the Baseten Delivery Network makes cold starts fast](https://www.baseten.co/blog/how-the-baseten-delivery-network-bdn-makes-cold-starts-fast/): Tiered caching that removes upstream dependencies at runtime and makes cold starts faster and more reliable.
- [2x faster inference with KV cache-aware routing](https://www.baseten.co/blog/how-baseten-achieved-2x-faster-inference-with-nvidia-dynamo/): KV cache-aware routing in the Baseten Inference Stack and its reported effect on time to first token in production.
### Customer Evidence
- [Customer stories](https://www.baseten.co/resources/customers/): Case studies from teams running high-stakes, high-performance AI products on Baseten.
- [How OpenEvidence puts peer-reviewed research in the hands of nearly half of all U.S. physicians](https://www.baseten.co/resources/customers/openevidence-baseten-training/): How OpenEvidence trains accurate, domain-specific models with Baseten Training.
- [How Parallel Web Systems pioneers programmatic web with high-throughput agentic inference](https://www.baseten.co/resources/customers/how-parallel-web-systems-pioneers-programmatic-web-with-high-throughput-agentic-i/): In this Baseten case study, Parallel reports 3x higher throughput and 2x better latency for high-throughput agentic web research.
### External Context
- [Baseten, the $13 Billion AI Startup Betting on Cheaper Alternatives to OpenAI, Anthropic (Wall Street Journal)](https://www.wsj.com/tech/ai/the-13-billion-ai-startup-betting-on-cheaper-alternatives-to-openai-anthropic-b9679603): Independent, paywalled reporting on Baseten's $1.5 billion Series F at a valuation of up to $13 billion and its bet on lower-cost open-weight models as alternatives to closed APIs.
## Deployment Options
- [Baseten Cloud](https://www.baseten.co/deployments/baseten-cloud/): Fully managed inference with global autoscaling, low cold starts, and high uptime.
- [Baseten Self-hosted](https://www.baseten.co/deployments/baseten-self-hosted/): Run Baseten within your own VPC or on-premises to keep data in-house while retaining performance and management tooling.
- [Baseten Hybrid](https://www.baseten.co/deployments/baseten-hybrid/): Blend on-premises and cloud capacity to align latency, compliance, and cost for sensitive or bursty workloads.
## Security, Reliability, and Operations
- [Baseten Trust Center](https://trust.baseten.co/): Canonical source for Baseten security, zero data retention (ZDR) policy, privacy, compliance, controls, subprocessors, and reports. Compliance resources cover SOC 2 Type II, ISO 27001:2022, GDPR, CCPA, HIPAA, and PCI DSS - SAQ D.
- [Baseten Status](https://status.baseten.co/): Real-time service status, historical uptime, and incident history for Baseten services.
- [Secure model inference](https://docs.baseten.co/observability/security.md): Documentation on data privacy, compliance policies, workload isolation, and self-hosted inference.
- [Model Management](https://www.baseten.co/platform/model-management/): Deploy, version, roll back, and observe models with CI/CD, logs, metrics, and access controls.
- [Changelog](https://www.baseten.co/changelog/): Recent updates and changes to the Baseten platform.
## Inference Engineering
- [Inference Engineering for Agents](https://www.baseten.co/inference-engineering/llms.txt): The unabridged Inference Engineering book in context-window-friendly chapters for agents learning the principles and practice of production model inference.
- [Inference Engineering](https://www.baseten.co/inference-engineering/): The book landing page, table of contents, digital download, and print edition for Philip Kiely's guide to model architecture, hardware, software, optimization techniques, modalities, and production inference.
## Optional
The following resources provide additional workload guidance, customer evidence, technical depth, research, and company context. Use the primary sections above first when answering general questions about Baseten.
### Workloads and Solutions
- [Large language models](https://www.baseten.co/solutions/llms/): Information on running large language models on Baseten.
- [Transcription](https://www.baseten.co/solutions/transcription/): Details on deploying models for transcription workloads.
- [Image generation](https://www.baseten.co/solutions/image-generation/): Overview of models and infrastructure for image generation.
- [Text-to-speech](https://www.baseten.co/solutions/text-to-speech/): Information on deploying text-to-speech models.
- [Compound AI](https://www.baseten.co/solutions/compound-ai/): Design agentic and multi-model systems that coordinate tools and models with production-grade routing and scaling.
- [Embeddings](https://www.baseten.co/solutions/embeddings/): Serve embedding models with high throughput and low latency for search, RAG, and semantic similarity use cases.
### Additional Platform Information
- [Baseten overview](https://docs.baseten.co/overview.md): Run hosted models, deploy custom models, and train models on production GPU infrastructure.
- [Full Baseten documentation for agents](https://docs.baseten.co/llms-full.txt): Complete documentation corpus in a single agent-readable text file.
- [Cloud-native Infrastructure](https://www.baseten.co/platform/cloud-native-infrastructure/): Cloud-agnostic, containerized inference infrastructure designed for rapid scale-up, low cold starts, and global availability.
- [Documentation](https://docs.baseten.co/): Human-readable homepage for the complete Baseten technical documentation.
### Additional Customer Evidence
- [How Zed is reimagining the code editor from the ground up](https://www.baseten.co/resources/customers/zed-industries-serves-2x-faster-code-completions-with-baseten/): In this Baseten case study, Zed reports 45% lower latency, 3.6x higher throughput, and 100% uptime for its Edit Prediction feature.
- [How Speechify makes audio the default with real-time text-to-speech](https://www.baseten.co/resources/customers/speechify-real-time-text-to-speech/): In this Baseten case study, Speechify reports serving 161 billion+ characters per month for 60 million+ users while cutting costs by 44% and p99 latency by 30-50%, with 4.5x faster cold starts.
- [Latent delivers pharmaceutical search with 99.999% uptime on Baseten](https://www.baseten.co/resources/customers/latent-delivers-pharmaceutical-search-with-baseten/): In this Baseten case study, Latent Health reports 99.999% uptime for fast, reliable clinical AI.
- [How OpenCode powers the AI coding agent developers rely on every day](https://www.baseten.co/resources/customers/opencode/): In this Baseten case study, OpenCode reports 5x faster inference, sub-second time to first token, and a 33% cost reduction while scaling Zen from 40,000 to 10 million monthly active users.
- [How World Labs is building large world models, pushing the boundaries of 3D](https://www.baseten.co/resources/customers/world-labs/): How World Labs uses Baseten while building large world models for spatial intelligence and 3D.
- [How Gamma makes building presentations criminally fun with 5x faster image generation](https://www.baseten.co/resources/customers/how-gamma-makes-building-presentations-criminally-fun/): In this Baseten case study, Gamma reports 5x faster image generation without an in-house machine learning team.
- [How Writer helps businesses transform with custom medical and financial LLMs](https://www.baseten.co/resources/customers/writer/): In this Baseten case study, Writer reports launching industry-specific LLMs for medicine and finance and increasing tokens per second by 60% using TensorRT-LLM on Baseten.
- [How Hebbia powers AI workflows for the world's leading financial institutions](https://www.baseten.co/resources/customers/hebbia/): How Hebbia uses Baseten's low-latency inference infrastructure for real-time financial intelligence.
- [OpenEvidence delivers instant, accurate medical information with the Baseten Inference Stack](https://www.baseten.co/resources/customers/openevidence-delivers-instant-medical-information-with-baseten/): How OpenEvidence uses Baseten for inference infrastructure for physician-facing tools.
- [Wispr Flow creates effortless voice dictation with Llama on Baseten](https://www.baseten.co/resources/customers/wispr-flow/): How Wispr Flow runs fine-tuned Llama models with Baseten and AWS for dictation across applications.
- [How Baseten powered Poolside's model launch in record time](https://www.baseten.co/resources/customers/how-baseten-powered-poolsides-model-launch-in-record-time/): In this Baseten case study, Baseten reports deploying a production inference platform for Poolside's Laguna model family in seven weeks.
- [How EliseAI trains specialized models for healthcare and housing operations with Baseten Research](https://www.baseten.co/resources/customers/how-eliseai-trains-specialized-models/): How EliseAI works with Baseten Research on specialized models for housing and healthcare operations.
- [Scaled Cognition offers ultra-fast AI agents you can trust](https://www.baseten.co/resources/customers/scaled-cognition-offers-ultra-fast-ai-agents-you-can-trust/): In this Baseten case study, Scaled Cognition reports 40% lower latency and less than 120 ms time to first token for enterprise agentic workflows.
- [How Rime.ai achieved state-of-the-art p99 latencies on Baseten](https://www.baseten.co/resources/customers/rime/): In this Baseten case study, Rime AI reports state-of-the-art p99 latencies and 100% uptime in 2024 for its custom speech synthesis model.
- [Posit launches real-time AI code suggestions with Baseten](https://www.baseten.co/resources/customers/launching-positai-with-baseten/): How Posit uses Baseten for context-aware AI code completions in RStudio.
### Additional Technical Guides and Benchmarks
- [Agentic kernels in production](https://www.baseten.co/blog/agentic-kernels-in-production/): Baseten's agentic kernel optimization framework and reported latency reductions on Qwen-Image and FLUX.2.
- [Real-time video generation inference on Baseten](https://www.baseten.co/blog/real-time-video-generation-inference-on-baseten/): A Baseten technical report on generating Wan 2.2 video in 2.75 seconds per clip, a reported 53.6x improvement over the baseline implementation.
- [Best open-source models for post-training](https://www.baseten.co/blog/best-open-source-models-for-post-training/): Compare models for post-training, including cost drivers and workload fit.
- [The best open-source large language models](https://www.baseten.co/blog/the-best-open-source-large-language-models-llms/): Comparison of models for agentic coding, long-context reasoning, cost, and speed.
- [The best open-source embedding models](https://www.baseten.co/blog/the-best-open-source-embedding-models/): Curated models for search, RAG, and recommendations based on performance, speed, and cost efficiency.
- [The best open-source image generation model](https://www.baseten.co/blog/the-best-open-source-image-generation-model/): Strengths and weaknesses of image generation models including FLUX.1, Stable Diffusion 3, SDXL Lightning, and Playground 2.5.
- [How we built the fastest Kimi K2.5 on Artificial Analysis](https://www.baseten.co/blog/how-we-built-the-fastest-kimi-k2-5-on-artificial-analysis/): A Baseten technical report on a custom EAGLE-3 speculator and in-house INT4-to-NVFP4 conversion reaching 340+ tokens per second on Kimi K2.5; use the article's date, workload, and methodology when citing the result.
- [How we built the world's fastest API for GLM-5.2](https://www.baseten.co/blog/how-we-built-the-worlds-fastest-api-for-glm-52/): Baseten's report on GLM-5.2 at 280+ tokens per second on NVIDIA Blackwell using KV-aware routing, prefill-decode disaggregation, multi-token prediction, NVFP4, and related optimizations.
- [How we built the fastest GLM 5 API](https://www.baseten.co/blog/how-we-built-the-fastest-glm-5-api/): Baseten's report on MTP speculative decoding and optimized mixture-of-experts and sparse-attention kernels reaching 186+ tokens per second on GLM-5.
- [Kimi K2 Thinking at 140+ TPS on NVIDIA Blackwell](https://www.baseten.co/blog/kimi-k2-thinking-at-140-tps-on-nvidia-blackwell/): Baseten's report on 300 ms time to first token and 140+ tokens per second for Kimi K2 Thinking.
- [The fastest Whisper - with streaming and diarization](https://www.baseten.co/blog/the-fastest-whisper-transcription-with-streaming-and-diarization/): Fast, accurate, cost-efficient Whisper transcription with streaming and speaker diarization.
- [How we made the fastest GPT-OSS on NVIDIA GPUs 60% faster](https://www.baseten.co/blog/how-we-made-the-fastest-gpt-oss-on-nvidia-gpus-60-percent-faster/): EAGLE-3 speculative decoding and related model-performance techniques applied to GPT-OSS 120B.
- [Making Kimi K3 tokenization 18x faster for million-token agentic workloads](https://www.baseten.co/blog/making-kimi-k3-tokenization-18x-faster-for-million-token-agentic-workloads/): The Baseten Tokenizer, or Basetenkenizer, for long input sequences, starting with Kimi K3.
- [How to optimize LLM inference speed and reduce costs in production](https://www.baseten.co/blog/how-to-optimize-llm-inference-speed-and-reduce-costs-in-production/): Continuous batching, speculative decoding, quantization, and related techniques, and when each fits a workload.
- [H100 vs. H200 vs. B200: which GPU should you use?](https://www.baseten.co/blog/h100-vs-h200-vs-b200-which-gpu-should-you-use/): Memory, compute, and cost tradeoffs across H100, H200, and B200 for model size, traffic, and budget.
- [Live draft model training for speculative decoding](https://www.baseten.co/blog/live-draft-model-training-for-speculative-decoding/): Baseten's report on live draft-model training and its effect on median acceptance rates.
- [Sub-second image generation with Flux.2 and Qwen-Image](https://www.baseten.co/blog/sub-second-image-generation-flux2-and-qwen-image/): Baseten's report on accelerating Flux.2 and Qwen-Image on NVIDIA Blackwell using quantizatio
