How to Stop LLMjacking: API Key Hygiene, Budgets, Gateway Logging

Layer 1 — API key hygiene

The attacker's job starts with getting a working key. Make keys boring to lose and useless once lost.

Short-lived over long-lived

• Prefer OAuth client credentials with refresh over static API keys. Where the provider supports it (AWS Bedrock via SigV4, GCP Vertex via workload identity federation, Azure OpenAI via managed identity), use the cloud-native identity, not a long-lived sk-*.
• For raw provider keys (OpenAI, Anthropic, Groq, Together, Replicate), rotate every 30–90 days on a schedule and immediately on any trigger (developer leaves, vendor disclosure, suspicious log event).

Scoped over blanket

• One key per project per environment. Not one org-wide key reused everywhere.
• Use provider-side scoping: OpenAI project keys with model allow-lists, Anthropic workspace scopes, Bedrock IAM policies restricted to specific model ARNs, Vertex IAM bound to specific publisher models.
• Pin monthly cost caps at the provider project level — a stolen key with a $500/month ceiling is not a headline.

Inventoried, never in source

• All keys through a secrets manager (Vault, AWS Secrets Manager, GCP Secret Manager, Doppler). No keys in .env files committed to Git, in Dockerfiles, in Helm values, or in CI logs.
• Run secrets scanning on every commit and every container image. Block merges with new secrets; alert on keys discovered in registries.
• Keep a live inventory: key ID, provider, project, owner, last-rotated date, monthly cap. If you cannot answer "which workload uses this key?" in 30 seconds, you cannot credibly rotate.

Layer 2 — Hard provider budgets

Every incident where a key stole seven figures ended at a provider that did not cap. Set the caps before you need them.

The pattern: set the monthly budget 20% above your expected spend, set a daily budget 2× your p95 daily spend, alert at 50% and 80% of each, and enforce (quota, IAM revoke, or deployment throttle) at 100%.

Layer 3 — Gateway logging and anomaly detection

Centralise every LLM call through a gateway you control (LiteLLM, PortKey, Helicone, bespoke) and structure-log every request. Three fields matter most:

• Identity — which agent / workload / user authenticated. Not the shared provider key; the identity on your side.
• Source IP and region — the egress address the request came from. Needed to spot "key used from a region I do not deploy in."
• Request rate and token volume per identity, per key, per model. Needed to spot sustained 10× baseline.

Alerts worth paging on

• Sustained >10× baseline RPS from any identity for more than 60 seconds.
• >5 minutes of >500 RPS on any single provider key — the Mandiant case was 1000+ RPS for hours.
• Any request from a source IP outside your workload CIDR ranges — including from cloud regions you do not deploy in.
• Projected daily spend > 2× p95 — rolling 1-hour window, projected linearly.
• HTTP 429 spikes from the provider — a cap being hit is almost always either a misconfigured app or an active LLMjacking attempt.
• New-user-agent / new-model combinations — attackers often default to the most expensive model your key has access to; a sudden shift to gpt-4o or claude-opus-4-7 from an identity that normally uses smaller models is a signal.

What to log

At minimum, per request: timestamp, identity, provider, model, route, prompt token count, completion token count, cost, latency, source IP, status code. Do not log prompt / completion bodies by default — treat those like payment data. If you need them for debugging, redact and sample.

The 72-hour LLMjacking defence checklist

1. Inventory every LLM API key your organisation holds. Owner, workload, last rotation, provider cap.
2. Set hard provider budgets on every project/workspace. Alert at 50%, enforce at 100%.
3. Rotate anything older than 60 days, anything that touched source or a dev laptop, and anything discoverable in a public GitHub search.
4. Route every call through your gateway. Block direct outbound from workloads to provider APIs at the egress layer where feasible.
5. Turn on anomaly alerts with the thresholds above. Page the on-call AI owner, not an email alias.
6. Patch the gateway. LiteLLM CVE-2026-40217 (RCE in the custom-code guardrail testing endpoint) is being actively weaponised — see the LLMjacking + LiteLLM RCE writeup.
7. Hunt the last 30 days: 429 bursts, outbound to provider APIs from pods / IPs that should not be calling them, sudden model-mix changes.

How TigerGate helps

• AI-SPM inventories every LLM provider key, which workloads hold them, last rotation date, and whether a hard provider budget is set. One dashboard, every provider.
• Secrets scanning catches AI API keys in source, images and CI logs before they reach a shared registry — the credential-theft side of the LLMjacking funnel.
• CSPM surfaces Bedrock, Vertex and Azure OpenAI access policies that are too broad, projects without budgets, and AI endpoints reachable from 0.0.0.0/0.
• eBPF runtime detection flags anomalous outbound to AI providers from your fleet — "pod talking to api.openai.com at 800 RPS from a region it never runs in" is a very loud signal.
• AI Security finds exposed LLM gateways on your perimeter and flags the admin / guardrail-testing routes that CVE-2026-40217-class RCEs target.