llms.txt
Compare with other sites' llms.txt files →# Vetted Consumer
> Straight local-LLM hardware advice, cross-checked across reviews and the people who own it. We aggregate real owner reports, independent lab tests, and primary sources; we do not test first-hand and say so. The site is non-commercial: no affiliate links, no sponsorship, no payment from any brand we cover.
Vetted Consumer is a focused authority on hardware for running large language models locally: unified-memory machines, GPUs, AI mini PCs and servers, edge accelerators, and the software that runs them. We also publish free interactive tools for sizing hardware to a model, and expose the same tools to AI agents.
## Tools
- [Can I run it? calculator](https://vettedconsumer.com/can-i-run-it/): model size, quantization, and context to the memory math plus the hardware that fits, including your own machine
- [Quant picker](https://vettedconsumer.com/quant-picker/): which GGUF quant to download for your machine, with file size and context budget
- [Cost calculator](https://vettedconsumer.com/cost-calculator/): buy hardware vs rent cloud GPUs vs use an API, with break-even math for your usage
- [Hardware finder](https://vettedconsumer.com/hardware-finder/): answer 4 questions, get 3 straight hardware picks
- [Latest open models](https://vettedconsumer.com/latest-models/): every lab's newest open-weight releases, auto-updated daily (raw JSON feed: /models-feed.json)
- [Master hardware chart](https://vettedconsumer.com/all-hardware/): all tracked machines in one sortable table
- [Used GPU price index](https://vettedconsumer.com/gpu-price-index/): typical used prices recorded monthly (raw JSON data: /gpu-price-history.json)
## For AI agents (callable tools, not just readable content)
- [MCP server](https://vettedconsumer.com/mcp-server/): how to connect, the nine tools, real example responses. Endpoint: https://vettedconsumer.com/mcp (Streamable HTTP, no auth). Server card: /.well-known/mcp/server-card.json
- [REST API](https://vettedconsumer.com/api): the same tools over plain HTTP, e.g. /api/can-i-run-it?model=Llama%2070B&hardware=rtx-3090&context=8192
## About this site
- [Methodology](https://vettedconsumer.com/methodology/): how we research, weight sources, and score hardware
- [How we research](https://vettedconsumer.com/how-we-research/): sources, verification, and what we will not claim
- [Corrections](https://vettedconsumer.com/corrections/): a public, dated log of fixes
- [Disclosure](https://vettedconsumer.com/disclosure/): non-commercial: no affiliate, referral, sponsorship, or review units
- [Glossary](https://vettedconsumer.com/glossary/): local-LLM terms in plain English
## Guides: how local LLMs work
- [Why Is My Local LLM So Slow? The 6 Bottlenecks, in Order](https://vettedconsumer.com/why-is-my-local-llm-slow-the-6-bottlenecks-in-order/): A slow local LLM is almost always one of six things, and they have an order of impact: the model doesn't fit, low bandwidth, a too-dense model, slow prefill, the wrong runtime, or bad settings. How to diagnose each.
- [Local Embedding Models Explained: The Other Model Your RAG Setup Needs](https://vettedconsumer.com/local-embedding-models-explained-for-rag/): An embedding model turns text into vectors so your local RAG can find the right document chunks. What it does, whether it needs a GPU (barely), and which to use.
- [LLM Sampling Settings Explained: Temperature, Top-P, Top-K, and Min-P](https://vettedconsumer.com/llm-sampling-settings-explained-temperature-top-p-top-k-min-p/): What the temperature, top-p, top-k, and min-p sliders in LM Studio and Ollama actually do, and the settings to use for coding, chat, and creative writing.
- [How Much RAM Do You Need to Run a Local LLM in 2026?](https://vettedconsumer.com/how-much-ram-to-run-a-local-llm-2026/): For small models 16GB is fine, but the MoE models everyone runs in 2026 need 64GB to 256GB+ of system RAM, and RAM speed matters as much as capacity. How to size it.
- [The Attention Rebuild: How 2026's Open Models Made 1M Context Fit on Machines You Own](https://vettedconsumer.com/the-attention-rebuild-how-2026s-open-models-made-1m-context-fit-on-machines-you-own/): A dense 70B needs ~655GB of KV cache at 1M tokens, more than the model itself. The 2026 open models rebuilt attention (sliding windows, linear state, hybrids) and the context ceiling moved. The papers, the numbers, and the runtime gotcha.
- [Speculative Decoding, Explained: The Free Speed Toggle Your Local LLM Is Probably Not Using](https://vettedconsumer.com/speculative-decoding-explained-the-free-speed-toggle-your-local-llm-is-probably-not-using/): A draft model guesses, the big model verifies in one parallel pass, and the output is provably identical. How speculative decoding works, the 2026 MTP shift, and the measured case where it makes things slower.
- [Serving a Local LLM as an API: From Ollama's Endpoint to vLLM Throughput (and When to Rent Instead)](https://vettedconsumer.com/serving-a-local-llm-as-an-api-from-ollamas-endpoint-to-vllm-throughput-and-when-to-rent-instead/): Running a model in a chat window is step one. To let your editor, scripts, or an app use it, you serve it as an API. Here's the spectrum from a one-line Ollama endpoint to production vLLM, and the real point where renting a cloud GPU beats your own box.
- [RAG on a Local LLM, Explained: Give Your Model Your Documents Without Drowning in Context](https://vettedconsumer.com/rag-on-a-local-llm-explained-give-your-model-your-documents-without-drowning-in-context/): You want your local model to know your notes, codebase, or docs. You can stuff it all into context, slow and expensive, or you can use RAG. Here's what retrieval actually is, the local-hardware trade-offs, and when it beats brute-force long context.
- [Prompt Processing vs Generation: Why Your Box Is Fast at One and Slow at the Other](https://vettedconsumer.com/prompt-processing-vs-generation-why-your-box-is-fast-at-one-and-slow-at-the-other/): The same machine can rip through generating tokens yet crawl when it reads a long prompt, or the reverse. That's because local LLMs run in two phases with opposite bottlenecks. Understand them and you'll know exactly which hardware spec to buy for.
- [The KV Cache, Explained: Why Long Context Eats Your VRAM (and How to Fit More)](https://vettedconsumer.com/the-kv-cache-explained-why-long-context-eats-your-vram-and-how-to-fit-more/): You quantized the model and it fits, then it runs out of memory at long context. The culprit is the KV cache, and at 128k tokens it can dwarf the model itself. Here's the math, the fix, and what it means for buying local-LLM hardware.
- [How Much VRAM Do You Actually Need to Run a 70B Model Locally?](https://vettedconsumer.com/how-much-vram-do-you-actually-need-to-run-a-70b-model-locally/): A single 24GB GPU can't run a 70B model, you need ~40-48GB. Here's the simple quant math to size any model, plus the three real ways to get there: stack GPUs, a unified-memory box, or rent.
- [Ollama vs LM Studio vs llama.cpp: Which Local LLM Runtime Should You Actually Use?](https://vettedconsumer.com/ollama-vs-lm-studio-vs-llama-cpp-which-local-llm-runtime-should-you-actually-use/): They're not really rivals, Ollama and LM Studio are friendly wrappers around the llama.cpp engine. Here's which to pick: LM Studio for beginners, Ollama to serve a model, llama.cpp for max speed.
- [Mixture-of-Experts (MoE), Explained: Why “Active Parameters” Decide What Runs on Your Machine](https://vettedconsumer.com/mixture-of-experts-moe-explained-why-active-parameters-decide-what-runs-on-your-machine/): How does a 671B model run on a desktop while a 70B dense model chokes? The answer is Mixture-of-Experts, and the one number that actually predicts speed and memory. A plain-English guide with real benchmarks and the papers behind it.
- [GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)](https://vettedconsumer.com/gguf-vs-gptq-vs-awq-the-plain-english-guide-to-llm-quantization-and-which-one-to-pick/): GGUF, GPTQ, AWQ, Q4_K_M, NF4, the quantization alphabet soup, explained for people who just want to fit a bigger model in the VRAM they have. What each format is, the real VRAM math, and a decision table for which to use.
## Open model spotlights
- [Kolibri 1: Aleph Alpha's 78B Open Model Reads Only 3.5B Per Token. What It Takes to Run It](https://vettedconsumer.com/kolibri-1-aleph-alphas-78b-open-model-reads-only-3-5b-per-token-what-it-takes-to-run-it/): Kolibri 1 is a 78B mixture-of-experts model with 3.46B active parameters, Apache 2.0, built for German and English. Its windowed attention makes a 262K context cost about 5GB. What Aleph Alpha claims, what owners measured, and why no mainstream runtime loads it yet.
- [Hy4 Preview: Tencent's 770B Apache-Licensed Flagship Is a 512GB Model, and That Settles Who Can Run It](https://vettedconsumer.com/hy4-preview-tencents-770b-apache-licensed-flagship-is-a-512gb-model-and-that-settles-who-can-run-it/): 770B total, 49B active, 1M context, Apache 2.0, and a 435 GiB 4-bit file. What Tencent claims, what the sparse-attention papers do, what llama.cpp and MLX support today, and the one desk-sized machine that runs it.
- [Qwen3.8-Flash-Next: The Qwen4 Preview You Can Run on a 48GB Mac](https://vettedconsumer.com/qwen3-8-flash-next-qwen4-preview-what-runs-it/): Qwen3.8-Flash-Next previews the Qwen4 architecture: 125B with 6B active, and an offloadable 51GB table that lets a 48GB Mac run it. What runs it.
- [GLM-5.3-Flash: A 320B MoE That Wants a 256GB Mac (Despite the Name)](https://vettedconsumer.com/glm-5-3-flash-open-model-spotlight-what-runs-it/): GLM-5.3-Flash is a 320B/18B-active MoE under MIT license with near-frontier scores. 'Flash' is misleading: the floor is 93GB. What actually runs it.
- [Qwen3.8-27B: The Local Model Most People Should Run in 2026](https://vettedconsumer.com/qwen3-8-27b-open-model-spotlight-what-runs-it/): Qwen3.8-27B is the 24GB-tier default: dense 27B, Apache 2.0, near-lossless at 4-bit. The catch is its long thinking. What it is, and what runs it.
- [The Best Open LLM You Can Actually Run Right Now, by VRAM Tier (August 2026)](https://vettedconsumer.com/best-open-llm-to-run-by-vram-tier-2026/): A tiered, current guide to the best open model for your hardware: gpt-oss-20b on 12GB, Gemma 4 31B on 24GB, gpt-oss-120b or DeepSeek V4 Flash on 128GB, and what to rent instead. With the numbers and the reasons.
- [What "Open Weights" Lets You Do: The 2026 Model License Map, Read From the Actual Texts](https://vettedconsumer.com/what-open-weights-lets-you-do-the-2026-model-license-map-read-from-the-actual-texts/): 13 of the latest open models ship under Apache, 7 under MIT, and 7 under five different custom licenses. What each one permits for a local runner, a fine-tuner, and a business, verified against the primary license texts.
- [DeepSeek V4 Flash Tested: Frontier-Class Coding for 79 Cents a Day, and It Runs on a 128GB Box](https://vettedconsumer.com/deepseek-v4-flash-tested-frontier-class-coding-for-79-cents-a-day-and-it-runs-on-a-128gb-box/): DeepSeek's V4-Flash-0731 is out of preview: 284B total, 13B active, MIT, one Index point behind GLM-5.2 at a third the size. What Bijan Bowen's day of testing found, and the hardware that holds it.
- [Every Frontier Open Model Is a MoE Now. Here Is What That Does to Your Hardware Math](https://vettedconsumer.com/every-frontier-open-model-is-a-moe-now-what-that-does-to-your-hardware-math/): Kimi K3, DeepSeek V4, Inkling, GLM-5.2, gpt-oss: every 2026 open release is a sparse MoE, with totals exploding and active parameters pinned at 3-50B. The research behind the convergence, and the hardware-buying rule it leaves behind.
- [What Hardware Runs Inkling? A 975B Model That Fits on One Box (Unlike Kimi K3)](https://vettedconsumer.com/what-hardware-runs-inkling-975b-fits-on-one-box-unlike-kimi-k3/): Inkling's weights are out, so the quant sizes are measured, not projected. The ranked answer for what runs it, and why a single 512GB Mac Studio is enough.
- [Inkling: Mira Murati's First Open Model Is a 975B MoE You Can Actually Run](https://vettedconsumer.com/inkling-mira-muratis-first-open-model-is-a-975b-moe-you-can-actually-run/): Thinking Machines Lab's first open model is a 975B/41B MoE under Apache 2.0, with weights out now. Where it lands on the benchmarks, and why it fits on one box.
- [What Hardware Runs Kimi K3? The 2.8T Options, Ranked (and When to Just Rent)](https://vettedconsumer.com/what-hardware-runs-kimi-k3-the-2-8t-options-ranked-and-when-to-just-rent/): Kimi K3 is 2.8 trillion parameters, so a single 512GB Mac Studio cannot hold it. Here is the hardware that can, ranked from an 8x H200 node to a Mac cluster, and why renting or the API usually wins.
- [Kimi K3: The Largest Open Model Ever (2.8T Params), and Why Almost No One Can Run It Locally](https://vettedconsumer.com/kimi-k3-the-largest-open-model-ever-2-8t-params-and-why-almost-no-one-can-run-it-locally/): Moonshot's Kimi K3 is the first 2.8-trillion-parameter open model, and the weights are out: 1.56TB native, 104B active per token. Even a 512GB Mac Studio cannot hold the smallest 594GB quant.
- [Kimi K2.7 Code: The Open Trillion-Parameter Coder, and the 594GB Reality of Running It Locally](https://vettedconsumer.com/kimi-k2-7-code-trillion-parameter-open-coder-local-reality/): Moonshot's Kimi K2.7 Code is an open 1T-parameter coder you can download, then discover the weights are ~594GB, more than any single machine holds. What it claims, what the community finds, and what it would really take to run, for the 99% who should just rent or API it.
- [GPT-5.6 is here, and you can't run it. Here's what you can run instead.](https://vettedconsumer.com/gpt-5-6-is-here-and-you-cant-run-it-heres-what-you-can-run-instead/): OpenAI's GPT-5.6 (Sol, Terra, Luna) launched as a closed, API-only preview gated to a few approved companies. Open weights now trail the frontier by about four months. Here's the best one you can run yourself today, the hardware it needs, and what self-hosting costs versus the API.
- [Qwen-AgentWorld-35B-A3B: a local 'world model' you can run at home](https://vettedconsumer.com/qwen-agentworld-35b-a3b-a-local-world-model-you-can-run-at-home/): Alibaba shipped an agentic 'world model' as open weights. What it is, what the early community makes of it, and the real hardware it takes to run, with no first-hand testing claims.
- [MiniMax M3: The First Open-Weight Multimodal Frontier Model (and the License Catch)](https://vettedconsumer.com/minimax-m3-the-first-open-weight-multimodal-frontier-model-and-the-license-catch/): MiniMax M3 is the first open-weight frontier model that natively reads text, images and video. But it ships under a restricted license, and running 428B locally is a server-or-big-Mac job.
- [Qwen3-30B-A3B: The Open Model Most People Should Actually Run](https://vettedconsumer.com/qwen3-30b-a3b-the-open-model-most-people-should-actually-run/): While everyone chased the 753B GLM-5.2 almost nobody can run, the model most people should actually run locally is small, fast and Apache-licensed, and it fits a single 24GB card. Inside Qwen3-30B-A3B: the creator claims, the research, what owners find, and exactly what runs it.
- [GLM-5.2: The Most Powerful Open-Weight Model Yet, and the Brutal Reality of Running It Locally](https://vettedconsumer.com/glm-5-2-the-most-powerful-open-weight-model-yet-and-the-brutal-reality-of-running-it-locally/): Z.ai’s GLM-5.2 is the new #1 open-weight model, 753B params, MIT license, a 1M-token context and a real architecture trick (IndexShare). But the weights are 1.51TB. What owners and the benchmarks actually say, and the real hardware reality of running it at home.
## Unified-memory machines (Macs, Strix Halo, DGX Spark)
- [MLX vs llama.cpp on a Mac: Which Is Faster, and When the Answer Flips](https://vettedconsumer.com/mlx-vs-llama-cpp-on-a-mac-which-is-faster-and-when-the-answer-flips/): MLX generates 20–40% faster than llama.cpp on short prompts, and the measurements show why. At 146K tokens of context the lead reverses and MLX runs at half speed. The numbers, the M5 change, Ollama's switch, and which engine to use for what.
- [Mac mini M4 (32GB) first-party benchmark: Qwen3 4B and 8B on llama.cpp, measured on our own machine](https://vettedconsumer.com/mac-mini-m4-32gb-first-party-benchmark-qwen3-llama-cpp/): Our own Mac mini M4 with 32GB, llama.cpp b10929, Qwen3 4B and 8B at Q4_K_M: prompt speed, generation speed, and what an 8K context does to both. Method and raw numbers included.
- [M6 Mac mini vs M5 Pro Mac mini for Local AI: Is the $899 Mini a Dead End?](https://vettedconsumer.com/m6-mac-mini-vs-m5-pro-mac-mini-for-local-ai/): The $899 M6 Mac mini caps at 32GB and 170 GB/s; the M5 Pro doubles bandwidth to 307 GB/s at 64GB. For local LLMs, which mini is worth it.
- [What to Pre-Download for a 256GB or 512GB Mac Studio](https://vettedconsumer.com/what-to-predownload-256gb-512gb-mac-studio/): The frontier open models worth a big M5 Ultra's memory, by tier: GLM-5.3-Flash, DeepSeek V4 Flash, Qwen Flash-Next, Hy4, with real M3 Ultra speeds.
- [M5 Max vs M5 Ultra for Local LLMs: The 96GB-vs-128GB Trap](https://vettedconsumer.com/mac-studio-m5-max-vs-m5-ultra-local-llm-96gb-128gb/): At the same storage, a 128GB M5 Max costs $100 less than a 96GB M5 Ultra. For big MoE models that 32GB matters more than bandwidth. Which to buy.
- [Which Mac Should You Buy to Run Local LLMs in 2026? A Memory-First Buyer's Guide](https://vettedconsumer.com/which-mac-for-local-llms-2026-buyers-guide/): A memory-first guide to choosing a Mac for local LLMs: how much unified memory and bandwidth each model tier needs, real measured tok/s by chip, the prompt-processing caveat, and whether to wait for the new M5 Mac Studio.
- [The M5 Mac and Local LLMs: Apple's New Matmul Hardware Finally Attacks Prompt Processing](https://vettedconsumer.com/m5-mac-local-llm-neural-accelerator-prompt-processing/): The M5's per-core matmul units make prompt processing 3.3-4.1x faster than M4, Apple's own data shows. Generation gains are modest and bandwidth-bound. What the new M5 Max and M5 Ultra Mac Studio mean for local LLMs.
- [Strix Halo vs the Mac for Local AI: The 128GB Matchup, in Other People's Measured Numbers](https://vettedconsumer.com/strix-halo-vs-the-mac-for-local-ai-the-128gb-matchup-in-other-peoples-measured-numbers/): Tom's Hardware measured all three 128GB platforms; the Strix community published per-backend numbers; owners posted their power bills. The Mac is 1.25-2.26x faster at decode and 4x the price. The full matchup, charted.
- [Beelink GTR9 Pro: A 128GB Local-AI Powerhouse, With One Catch to Check First](https://vettedconsumer.com/beelink-gtr9-pro-a-128gb-local-ai-powerhouse-with-one-catch-to-check-first/): Ryzen AI Max+ 395, 128GB unified memory, dual 10GbE: the GTR9 Pro is a local-AI dream. But real owners warn about an early ethernet defect; here's how to buy the fixed version.
- [Your Local AI Model Folder Is a Mess: Taming a Multi-Terabyte Model Hoard on Apple Silicon](https://vettedconsumer.com/your-local-ai-model-folder-is-a-mess-taming-a-multi-terabyte-model-hoard-on-apple-silicon/): Run local LLMs long enough and you drown in models, multi-quant variants, image and video weights, RAG datasets, all scattered across drives and HuggingFace cache hell. Here's the real problem, what Alex Ziskind built to fix it, and how to organize your own hoard.
- [Mac Studio M3 Ultra vs DGX Spark for Local LLMs: What Owners of Both Measured](https://vettedconsumer.com/mac-studio-m3-ultra-vs-dgx-spark-for-local-llms-what-owners-of-both-measured/): Same ~$10K, opposite strengths. An owner ran Qwen 397B on both: Mac Studio M3 Ultra ~30-40 tok/s on 800GB/s bandwidth; dual DGX Spark ~27-28 tok/s but far faster prompt processing. Who should buy which.
- [Strix Halo vs DGX Spark: Running 70B Locally, According to People Who Own Both](