llms.txt
Compare with other sites' llms.txt files →# Marginal — AI cost tracking: integration guide
Marginal ingests cost events for AI/LLM usage and shows spend sliced by
model, provider, and custom fields (customer, feature, ...).
This document is the complete integration contract, written for a coding
agent instrumenting an existing codebase. Human docs:
https://marginalhq.com/docs
## Your task
1. Find every AI/LLM call site: OpenAI, Anthropic, Vercel AI SDK, Gemini,
Bedrock, OpenAI-compatible hosts (Groq, Mistral, xAI, DeepSeek,
Together, Fireworks, Ollama, Azure), and raw HTTP calls to any of them.
2. Install the Marginal SDK and construct one shared client.
3. After each AI response, call track() with provider, the
response-reported model, the response's usage object, and fields.
4. Report to the user: which call sites you instrumented, which fields you
attached, and remind them to register those field keys in Marginal's
Fields page BEFORE traffic flows (unregistered keys are stripped).
## Rules
- Construct the client once (module scope), read the API key from the
MARGINAL_API_KEY env var. Never hardcode keys. Server-side code only.
- track() is synchronous, buffered, and never throws — call it inline
after the AI call. Do not await it, wrap it in try/catch, or move it
off the request path.
- Pass the provider's usage object AS-IS wherever a recipe below says so.
Do not convert, rename, or sum token counts yourself — the server
detects each provider's wire dialect (OpenAI Chat Completions, OpenAI
Responses, Anthropic, Gemini, Bedrock Converse) by its key names and
normalizes the cached/uncached split. Hand-converted counts are the #1
source of cost bugs; don't be that integration.
- model must be the RESPONSE-reported model (response.model / modelId),
not the alias that was requested — dated snapshots price differently.
- fields: flat object, string/number/boolean values, no nesting. Attach
what identifies the spend: customer (who), feature (which part of the
product). Do NOT add an environment field — environments (production/
staging) are separate Marginal projects with their own API keys. Derive
values from surrounding code where obvious; otherwise ask the user.
Never put prompts, completions, or PII in fields.
- On shutdown paths that exist anyway (SIGTERM handlers, serverless
handler end), call marginal.shutdown() / flush() so buffered events
drain. Do not invent new lifecycle hooks just for this.
## SDK setup
TypeScript/JavaScript (Node 18+):
npm install marginal-sdk
import { Marginal } from "marginal-sdk";
const marginal = new Marginal({ apiKey: process.env.MARGINAL_API_KEY });
marginal.track({ provider, model, usage, fields }); // LLM call
marginal.track({ cost: 0.05, fields }); // pre-computed dollars
Python (3.9+):
pip install marginal-sdk
from marginal import Marginal
marginal = Marginal(api_key=os.environ["MARGINAL_API_KEY"])
marginal.track(provider=..., model=..., usage=..., fields=...)
marginal.track(cost=0.05, fields=...)
Events buffer and flush automatically (every 5 s / at 100 events).
Integration warnings arrive via onError / on_error: "ignored-fields"
(a field key is not registered) and "unpriced-models" (model missing
from the price catalog) — surface these to the user, they are actionable.
## Event contract
Each event needs EITHER cost (finite number >= 0, dollars) OR the complete
triple provider + model + usage. Both -> cost wins. Neither -> rejected.
- provider: lowercased string, <= 64 chars. Namespaces the price lookup.
- model: <= 200 chars, response-reported.
- usage: the provider's usage object. Recognized token counts must be
non-negative integers; unrecognized keys are ignored (kept for audit),
never rejected. Do not send null values for token counts — omit or 0.
- fields: only keys registered in the project land; the rest are stripped
and reported back in the response's "ignored" array.
- Timestamps are server-assigned on arrival; the API accepts none.
Pricing happens at ingest: project custom prices, then catalog
"provider/model", then bare "model". Unknown model -> event lands with
cost = null (never a silent $0) and the model is reported in the
response's "unpriced" array; the user can add a custom price on the
Models page.
## Per-library recipes
### OpenAI (Node + Python) — also Azure and all OpenAI-compatible hosts
Chat Completions and the Responses API: pass usage as-is.
const c = await openai.chat.completions.create({ model, messages });
marginal.track({ provider: "openai", model: c.model, usage: c.usage, fields });
Python: usage=completion.usage.model_dump()
Streaming REQUIRES stream_options: { include_usage: true } (Python:
stream_options={"include_usage": True}) or usage never arrives. The final
chunk carries chunk.usage (other chunks: null); read model from that chunk.
For Azure / Groq / Mistral / xAI / DeepSeek / Together / Fireworks /
Ollama: identical recipe, change only the provider string ("azure",
"groq", "mistral", "deepseek", "xai", "together_ai", "fireworks_ai",
"ollama").
### Anthropic (Node + Python)
const m = await anthropic.messages.create({ model, max_tokens, messages });
marginal.track({ provider: "anthropic", model: m.model, usage: m.usage, fields });
Python: usage=message.usage.model_dump()
Streaming: usage is split across stream events — do NOT reassemble it.
Use the SDK helper: TS stream.finalMessage(), Python
stream.get_final_message(); then track the final message's model + usage.
### Vercel AI SDK (v7)
Map the SDK's normalized usage onto Marginal's canonical shape:
function toMarginalUsage(usage) {
return {
input_tokens: usage.inputTokenDetails.noCacheTokens ?? usage.inputTokens ?? 0,
cache_read_tokens: usage.inputTokenDetails.cacheReadTokens ?? 0,
cache_write_tokens: usage.inputTokenDetails.cacheWriteTokens ?? 0,
output_tokens: usage.outputTokens ?? 0,
};
}
generateText: model: result.response.modelId,
usage: toMarginalUsage(result.usage) // summed across steps
streamText: model: (await result.response).modelId,
usage: toMarginalUsage(await result.usage)
provider = whichever provider the model was routed to ("openai",
"anthropic", ...). AI SDK <= 6 has a flat usage object instead:
input_tokens: usage.inputTokens - (usage.cachedInputTokens ?? 0),
cache_read_tokens: usage.cachedInputTokens ?? 0,
output_tokens: usage.outputTokens.
### Google Gemini
JS: pass response.usageMetadata as-is; model: response.modelVersion.
Python renames the counts to snake_case — send them back under their wire
names (do not send None; use 0):
um = response.usage_metadata
usage = {
"promptTokenCount": um.prompt_token_count or 0,
"candidatesTokenCount": um.candidates_token_count or 0,
"thoughtsTokenCount": um.thoughts_token_count or 0,
"cachedContentTokenCount": um.cached_content_token_count or 0,
}
provider: "gemini". Streaming: the last chunk's usageMetadata has totals.
### AWS Bedrock
Converse: usage: response.usage as-is; model: the modelId you requested
(Converse does not echo it back); provider: "bedrock".
InvokeModel: the response body carries the model vendor's native usage
(Anthropic-shaped for Claude) — pass that as-is.
### Anything else (self-hosted, custom fine-tunes)
Canonical shape — input_tokens is the UNCACHED input count:
usage: { input_tokens, output_tokens, cache_read_tokens?, cache_write_tokens? }
Tell the user to add a custom price on the Models page or events stay
unpriced (they still land, with token counts and cost = null).
### Non-LLM spend (voice, images, tool calls, pre-computed costs)
marginal.track({ cost: 0.05, fields });
## HTTP fallback (no SDK)
POST https://api.marginalhq.com/v1/events
Authorization: Bearer mgl_...
Content-Type: application/json
{ "events": [ { "provider": "...", "model": "...", "usage": { ... },
"fields": { ... } } ] }
Limits: 500 events/request, 1 MB body. Response is 202 with per-event
acceptance: { accepted, duplicates?, rejected?, ignored?, unpriced? }.
Optional per-event "event_id" (1-128 chars) is an idempotency key: events
sharing one land once (30-day window; repeats are counted in accepted and
reported in duplicates). The SDKs stamp a UUID automatically.
## Verify before finishing
Trigger one request through each instrumented call site, then check:
(1) the project's Ingest log page shows each request tagged "success" —
a "warn" or "error" row spells out rejected events (with payloads),
stripped field keys, and unpriced models,
(2) the events appear in the project's Explorer page with the expected
model, cost, and fields,
(3) no "ignored-fields" / "unpriced-models" warnings from onError.
Nothing fails silently — if these pass, the integration is complete.