When the LLM Provider Finishes but Your Gateway Times Out

A durable usage outbox protects a receipt you have. It cannot recover a receipt you never received.

Imagine an upstream request crosses its processing boundary, but its usage response never reaches your gateway. A timeout tells the gateway that it stopped waiting. It does not tell it whether processing occurred, what usage accrued, or whether repeating the request is safe.

On October 24, 2026, we reproduced this boundary against FluxMeter commit b4ede374ed9ecc863ec71095c37fd89633161309. The provider was a local HTTP mock: it recorded processed=true and synthetic usage of 100 input tokens and 50 output tokens, then withheld its response until the gateway timed out. No real model inference occurred, no commercial provider was called, and the experiment establishes no actual provider charge.

The failure was reproduced; acceptance of the proposed Hybrid UNKNOWN lifecycle failed. Here, Hybrid means retaining synchronous admission and reservations while resolving uncertain execution asynchronously. It is a candidate design, not a shipped capability.

Gateway and Custody ran from the repository source; reservations, the outbox, and billing Lua used an isolated real Redis instance. Kafka acknowledgements were controlled test doubles. The Lua was extracted from BudgetEnforcerSink.java and invoked directly with synthetic window keys and arguments. Java, Flink, Kafka, and ClickHouse did not run as an end-to-end pipeline.

The test client factory shortened the gateway read timeout from 120 seconds to 0.15 seconds; the mock delayed for 0.6 seconds. Production source was unchanged. HTTP routes used FastAPI TestClient. Streaming failures exercised the application response iterator against a real local HTTP server, without establishing browser or reverse-proxy SSE behavior.

The original recorded results were 8 passed, 3 warnings in 5.40s and 64 passed, 7 warnings in 3.60s. The eight experiments include assertions confirming missing capabilities. Green tests mean the observed behavior matched those assertions; they do not mean UNKNOWN acceptance passed. The 64 related regressions are not an end-to-end guarantee. Warnings concerned deprecated FastAPI lifecycle hooks.

For a non-streaming timeout, the gateway retained its reservation and produced no usage event. The Redis outbox was empty. The HTTP route returned 500 without a reservation ID in the response. A hold remained, but the implementation supplied neither an explicit UNKNOWN execution state nor a receipt-recovery record.

By contrast, after usage had been supplied manually, simulated Kafka failure buffered an envelope in Redis. Restoring the ACK boundary let the outbox replay that same envelope and clear its pending entry, without another provider call. This tests delivery recovery for an existing record. It does not test recovery of a missing provider receipt.

The mock record was private test instrumentation, including an identifier unknown to the gateway. An operator manually correlated it to the reservation. Custody accepted that supplied payload; the experiment did not prove automatic lookup, authentication, or verification of a provider receipt. A Trusted Envelope carries FluxMeter custody context; its name does not establish independent provider attestation.

Each individual timed-out call caused one mock-provider execution in this scenario. We did not observe an automatic upstream retry inside that call. Sending the client request again with the same Idempotency-Key and X-Request-ID caused two executions and two distinct reservations across the two calls. Those correlation headers were not forwarded to the mock provider.

The current reservation-derived event ID can stabilize usage delivery for one reservation. It cannot deduplicate model execution when every client retry opens a new reservation. Execution identity, reservation identity, and usage-event identity need explicit relationships; they are not interchangeable.

Another experiment submitted identical token counts through Gateway ingest_usage twice for the same reservation, 20 milliseconds apart. The first submission was accepted; the second conflicted because the wrapper regenerated timestamp, which participates in the payload fingerprint. Replaying the unchanged Custody payload was idempotent. Reconstructing it with a new timestamp was not.

Preserve the original event timestamp and canonical payload for delivery retries. Keep delivery-attempt timestamps outside that immutable identity. A changed token count should remain a conflict unless an explicit correction protocol authorizes it; weakening fingerprint checks would conceal real disagreement.

The streaming case exposed a correctness problem. The mock sent SSE headers but no content or usage receipt before timeout. Because a positive reservation existed, the gateway finalizer still submitted a 0-input, 0-output usage event. It converted absent evidence into a concrete measurement.

When the test later supplied 100/50 usage under that same reservation-derived event ID, Custody returned conflict with retryable=false. Conflict detection was doing its job: a different payload claimed an already-used identity. The earlier decision to commit zero usage had consumed the identity needed for the eventual measurement.

The same distinction applies to a successful response whose usage field is absent: code that defaults missing counters to zero needs review. That additional path is visible in the source but was not exercised by Experiment #001. Partial stream estimates also need evidence labels; they must not silently become authoritative provider usage.

The minimum repair is to preserve uncertainty when evidence is missing, expose it to operators, and define when a measurement is sufficiently supported to enter billing. Confirmed zero usage is a valid outcome. Unknown usage is a different outcome.

Direct calls to Reservation.expire_due released the hold once; a repeated call released nothing. The test observed an expiry alert and deletion of the reservation record. Releasing the hold restored available budget. It did not establish that the upstream consumed zero resources.

With manually supplied usage and synthetic window arguments, the actual billing Lua deducted the sample cost of $0.000045 from a $10 balance and skipped a replay of the same window. This occurred both with an existing reservation and after expiry. The price came from the Python catalog for the test; Java pricing parity was not revalidated.

This supports atomic settlement and window replay protection within the tested component boundary. It does not establish exactly-once billing across provider execution, Kafka delivery, Flink windows, recovery, or invoices. HTTP 202 means Custody acceptance, not completed billing.

A hold deadline, an execution investigation deadline, and evidence retention serve different purposes. The code uses a 15-minute default reservation deadline and a 24-hour record TTL. Worker outages beyond record retention, concurrent expiry and settlement, and late-window routing require separate tests. Experiment #001 called expiry directly; it did not validate background scheduling.

First stop missing receipts from becoming final zero-usage events. Then persist a tenant-scoped execution identity before dispatch, bind it to a request fingerprint and reservation, and return a traceable outcome on timeout. Reuse of an execution key with a different request must conflict. Reuse with the same request must consult durable state before any new dispatch.

A durable execution claim still cannot make an arbitrary provider exactly once. A crash around dispatch can leave the outcome ambiguous. Unless the provider supports a verified idempotency or lookup contract, the safe state is UNKNOWN, followed by investigation or an explicit new execution decision, rather than blind resubmission.

Next preserve canonical usage payloads for retry, retain evidence independently of hold release, and implement idempotent transitions for receipt arrival and settlement. Separate execution state, evidence state, hold state, and settlement state so that releasing a hold cannot erase unresolved liability.

A proposed path is RESERVED → DISPATCHED → UNKNOWN → EVIDENCE_RECEIVED → SETTLED, with independent hold release. Terminal unresolved cases need an explicit operator policy. Corrections need an append-only link to the prior measurement and defined billing semantics; sending a new full-usage event could double count.

Provider-specific lookup adapters, automated reconciliation, and a rich audit interface can follow demonstrated demand. The absence of those features is not a reason to defer the zero-usage and retry-identity correctness work. Kafka → Flink → Redis remains the supported usage aggregation path; this proposal does not introduce a parallel aggregation path.

The downloadable test is the original eight-case harness. It flushes only its disposable Redis instance. The runner mounts the engine read-only, disables external networking during test execution, and puts output in a separate directory. Preparation requires Docker, a Python 3.11 FluxMeter API image, and test dependencies; the accompanying README explains the original local-image digest and the limits of rebuilding it.

Use a checkout of the recorded engine commit and the linked bundle. Run the command below from the bundle directory. The repository argument is an example relative path, not a private machine path. Read the script before running it; the harness deliberately asserts existing defects.

The original evidence includes exploratory reruns. An early harness failure used the wrong counter-key index; fixing that external assertion did not repair the product. Final results came from the final experiment and regression logs. The complete make demo-proof flow was inspected but not run, and exercises a different streaming-kill scenario.

Real-provider behavior and charges, tenant isolation end to end, provider receipt verification, crash recovery, concurrent expiry/settlement, Flink late events, and ClickHouse lifecycle audit remain unverified. The useful conclusion is narrower and actionable: durable delivery cannot substitute for durable knowledge of execution.