Your spend cap holds — until it meets a customer it has never seen.

A budget cap that works in every test can still let a brand-new customer through without metering a single request. Not "over budget" — never metered. HTTP 200, no error, no warning, and the cap configured correctly the entire time.

The crux

Somewhere between "read the cap" and "enforce the cap", the value is null — and null has to mean two different things at once.

if (resolvedCap === null || resolvedCap <= 0) return { allowed: true, metered: false };

That line is correct. null legitimately means this entity has no cap, don't meter it. It also means a cap exists but did not reach me. Nothing at the decision point can tell those apart, so the second case inherits the first case's benefit of the doubt, and the request sails through unmetered.

A first-seen customer is where this bites, because there is no stored row to fall back to. Concurrency isn't the bug — it's the amplifier: N simultaneous first requests all resolve before any of them has persisted spend, so every one of them reads the same empty state, and every one of them is let through.

Run it

node check.js --demo

Zero dependencies, no network, no setup — reproduces the class in nine lines.

Against your own gateway:

node check.js --base-url http://localhost:4000/v1 \
              --api-key "$PROXY_KEY" --model gpt-4o-mini --n 8

It invents a customer id the gateway has never seen, fires N concurrent completions for it, and reports how many reached the model. Give that customer a near-zero budget first, or the result means nothing. Three verdicts: the cap held, exactly one request won the race (enforcement exists but isn't atomic), or the cap doesn't apply to unknown customers at all.

Seen in the wild

BerriAI/litellm#40095 — eight concurrent first requests for a new customer all reached Bedrock and returned 200, persisting spend 264,000x the configured budget. Once the first spend created the customer row, the next identical request correctly returned 429.

The reported cause was a dropped field: the helper that copies budget fields onto the token copies four of them and omits the aggregate end_user_max_budget. Tracing it to the consumer explains the rest — _get_end_user_budget_counter reads exactly that field, finds None, skips the DB fallback because a first-seen customer has no row, and returns None. The caller appends only non-null counters, so the customer drops out of the reservation set entirely. Reservation still runs, still succeeds, meters nothing.

Two details worth carrying to your own code:

• The asymmetry is why it survives. The per-model budget field is copied; the aggregate one isn't. Per-model caps enforce correctly, so the feature looks alive in any deployment or test that uses them.
• A test already covered the helper. It asserts the copied fields one by one — and the omitted field is precisely the one the assertions never name. Coverage on the function is not coverage on the field.

The fix is two-part

Most people ship only the first:

1. Carry the cap through every hop that copies entity fields.
2. Make "cap configured but unresolvable" fail closed. A dropped field should be an error, not an unmetered request.

Without (2), the next refactor reintroduces this silently — and you find out from the invoice.

Check your own system

Three questions that surface this class without reading any code:

• Does a first-seen entity get metered, or only one that already has a row?
• Can your logs distinguish "no cap configured" from "cap didn't resolve"? If both produce the same line, you cannot detect this in production.
• Do concurrent first requests each see zero spend? Check-then-act with no atomic reservation is a cap in name only.

Get the code

Work with us

We take fixed-scope engagements on exactly this: agent and LLM spend that isn't holding, caps that don't enforce, and bills nobody can attribute. If the checker found something, or you'd rather we look properly — io@deemwar.com.