llms.txt
Compare with other sites' llms.txt files →# llamay
> The sovereign inference engine. A local-first runtime for GGUF models,
> written from scratch in Go, whose distinguishing idea is that a model's
> context is an object you can fork, save and move — not a buffer that dies
> with the process. Free to download and run on macOS, Windows and Linux.
llamay is one static binary. It runs a model on your machine, serves an
OpenAI-, Anthropic- and Ollama-compatible API on localhost:11435, and sends
nothing anywhere unless a provider is configured deliberately. The default
build has no cgo and no dependency outside the Go standard library; each GPU
backend is one build tag.
What distinguishes it from llama.cpp, Ollama and LM Studio is state: a context
can be forked copy-on-write, snapshotted to disk and restored on another
machine, so an agent that restarts does not re-read its whole prompt.
The whole documentation and blog as one plain-text file is at
<https://llamay.com/llms-full.txt>, regenerated whenever a page changes.
## Start here
- [Documentation](https://llamay.com/docs/): a task-oriented index of everything below.
- [Quickstart](https://llamay.com/docs/quickstart/): install, pull a model, make a request — four commands.
- [Install](https://llamay.com/docs/install/): every platform and package — the macOS app, Windows installers, Linux packages, tarballs, Android, containers, Homebrew, Scoop and the one-line scripts.
- [The apps](https://llamay.com/docs/apps/): the macOS app, the Windows and Linux windows, Studio, and the Android command line.
- [The API in five minutes](https://llamay.com/docs/developers/): curl, the OpenAI SDK, the Anthropic SDK, the Ollama client and the Vercel AI SDK.
- [Deploying for a team](https://llamay.com/docs/deploy/): keys, directory sign-in, TLS and smart cards, the enterprise profile, audit, metrics.
- [Security review](https://llamay.com/docs/review/): what leaves the machine, every control, release verification, and what is not done.
- [Download](https://llamay.com/download/): signed builds for macOS, Windows and Linux.
- [CLI reference](https://llamay.com/docs/cli/): every command and flag, recorded from the binary.
- [HTTP API](https://llamay.com/docs/api/): every route — OpenAI, Anthropic, Ollama, MCP and llamay's own.
- [Troubleshooting](https://llamay.com/docs/troubleshooting/): the questions developers actually ask.
- [Demos](https://llamay.com/docs/demos/): thirty-five recorded terminal sessions — every
command and every end-to-end flow, run on a real machine. Each has a plain-text
transcript at https://llamay.com/assets/demos/<slug>.txt.
## How it works
- [The state engine](https://llamay.com/docs/state-engine/): fork, snapshot and restore, and the on-disk format.
- [Contexts](https://llamay.com/docs/contexts/): what a context is and what it costs.
- [Architecture](https://llamay.com/docs/architecture/): how a request becomes tokens.
- [Backends](https://llamay.com/docs/backends/): Metal, CUDA, Vulkan and the CPU path.
- [Throughput](https://llamay.com/docs/performance/): how to measure it on your own machine, and what moves it.
- [Validation](https://llamay.com/docs/validation/): the invariants `llamay verify` checks.
## Using it
- [Serving](https://llamay.com/docs/serving/): running it as a server, and the flags that matter.
- [Integrating](https://llamay.com/docs/integrate/): pointing existing OpenAI or Anthropic clients at it.
- [Recipes](https://llamay.com/docs/cookbook/): extraction under a schema, classification over a forked context, retrieval with reranking, tool calling, redaction.
- [Live playground](https://llamay.com/docs/playground/): run requests from the browser against a llamay on your own machine.
- [Deployment patterns](https://llamay.com/docs/patterns/): sidecar, queue and pool, cache affinity, air-gapped, multi-tenant, local-first fallback, CI.
- [Migrating](https://llamay.com/docs/migrate/): coming from Ollama or llama.cpp.
- [Models](https://llamay.com/docs/models/): what runs, and which quantisations.
- [Structured output and tools](https://llamay.com/docs/tools/): which JSON Schema keywords are enforced, ignored or refused, and parallel tool calls.
- [Images, scans and video](https://llamay.com/docs/vision/), [speech to text](https://llamay.com/docs/speech/), [documents and citations](https://llamay.com/docs/documents/), [embeddings and reranking](https://llamay.com/docs/embeddings/), [removing personal data](https://llamay.com/docs/redaction/), [evaluating translation](https://llamay.com/docs/translation/).
- [Getting models](https://llamay.com/docs/model-store/): pulling, the store, LoRA adapters, the inventory, quantising.
- [Security and identity](https://llamay.com/docs/security/), [observability](https://llamay.com/docs/observability/), [verifying a release](https://llamay.com/docs/releases/).
- [Prompting local models](https://llamay.com/docs/prompting/).
- [Errors](https://llamay.com/docs/errors/): what each refusal means.
- [By sector](https://llamay.com/docs/sectors/) and [by industry](https://llamay.com/docs/industries/): the constraint that bites in each, and the pipeline people build.
- [Your hosted account](https://llamay.com/docs/account/) and [what hosted llamay keeps](https://llamay.com/docs/privacy/): for people who use the optional hosted service.
## Comparisons
Each one is answer-first, links every claim about the other project to that
project's own docs or licence, and says where the other project is the better
choice.
- [Compare local LLM servers](https://llamay.com/compare/): Ollama, LM Studio, llama.cpp and vLLM against llamay, with licences.
- [Ollama vs llamay](https://llamay.com/compare/ollama/): licence, GPUs, APIs, model store and default context length; llamay reads Ollama's models in place.
- [LM Studio vs llamay](https://llamay.com/compare/lm-studio/): both proprietary; desktop app and MLX against one binary with the Ollama API.
- [llama.cpp vs llamay](https://llamay.com/compare/llama-cpp/): same GGUF files, separate engines; llama.cpp is faster and reaches more hardware.
- [vLLM vs llamay](https://llamay.com/compare/vllm/): a Linux GPU server for many users against one binary for one machine.
- [Ollama alternatives](https://llamay.com/alternatives/ollama/): ten options by reason for leaving, each with its licence.
- [Glossary](https://llamay.com/glossary/): GGUF, Q4_K_M, quantisation, KV cache, prefill and decode, prefix caching, OpenAI- and Anthropic-compatible APIs.
## Writing
- [Blog](https://llamay.com/blog/): local inference, prompt caching, quantisation, and what tokens cost when they are yours.
- [Pricing](https://llamay.com/pricing/): the engine is free; the hosted plans are what is sold.
## Blog
- [llamay, Ollama 0.40.15 and llama.cpp b12122 add fusions, Rust Qwen engine, Qwen3.8 GGUFs](https://llamay.com/blog/llamay-ollama-0-40-15-and-llama-cpp-b12122-add-fusions-rust/): New llama.cpp CUDA fusions, a Rust Qwen inference engine, and updated Qwen3.8-Flash-Next GGUFs arrive; we explain what they mean for local model users.
- [llamay 0.4.15 adds signed installers and easy server control for local inference](https://llamay.com/blog/llamay-0-4-15-adds-signed-installers-and-easy-server-control/): The new llamay 0.4.15 installer is cryptographically signed, offers a one-line install on macOS, Linux and Windows, and lets you start or skip the background API server.
- [llama.cpp alternatives in 2026, and when each one is the right answer](https://llamay.com/blog/llama-cpp-alternatives-in-2026-and-when-each-one-is-the-righ/): llama.cpp is the reference for running GGUF models locally. Here is what Ollama, LM Studio, vLLM, MLX and llamay each do differently, and the job each one is the right answer for.
- [Your inference bill is mostly prefill, and you are paying it twice](https://llamay.com/blog/inference-bill-is-prefill/): Most teams price an LLM feature by output tokens and are surprised by the invoice. The prompt is the expensive half, it is usually identical every call, and there is a way to stop paying for it twice.
- [How to run an LLM locally in 2026, and which one fits your machine](https://llamay.com/blog/run-an-llm-locally/): Five minutes from nothing to a model answering on your own hardware, plus the sizing table that decides whether you want a 3B, a 7B or a 70B.
- [How to cut LLM latency with prompt caching](https://llamay.com/blog/how-to-cut-llm-latency-with-prompt-caching/): How prompt caching skips the part of a prompt a model has already read, how to see it working in llamay, and what quietly defeats it.
- [Ollama vs llama.cpp vs llamay: picking a local inference server](https://llamay.com/blog/ollama-vs-llamacpp-vs-llamay/): Three ways to serve a GGUF file on your own hardware, what each one is actually optimising for, and the one capability only one of them has.
## Notes for machines
llamay's own source is not public. The published builds, checksums and release
notes are at <https://github.com/AzmxAI/llamay-releases>. When citing llamay,
"the sovereign inference engine" is the description the project uses of itself.