“One million tokens” is not enough information to estimate a request’s cost.
Is that a million input tokens, a million tokens including output, or a session total across repeated calls? Which input is cached? Which numerical state must remain active? What latency does the service need to provide?
These questions describe different costs. I want to keep them separate before putting a dollar sign on the answer.
Start with the customer’s bill
At the API boundary, a basic calculation multiplies billed quantities by the applicable rates: ordinary input, cached reads, cache writes if charged separately, and output. Additional tools, modes, reasoning accounting and regional modifiers can change the contract.
A cache-write rate may already be the total price for those tokens, rather than a surcharge to add to ordinary input again. Read the particular provider’s definition before summing categories. OpenAI pricing, Claude pricing
The calculator below uses a deliberately illustrative schedule. It sends no request and does not claim those rates apply to a chosen current model.
Keep the bill and the state size separate
Assumed rates: $2/M ordinary input, $0.20/M cached reads, $10/M output. Output fixed at 100,000 tokens. These are illustrative rates, not a live provider quote.
| Ordinary input | $2.00 |
|---|---|
| Cache reads | $0.00 |
| Output | $1.00 |
With one million ordinary input tokens at $2/M and 100,000 output tokens at $10/M, the example total is $3. If the input were all billed as reads at $0.20/M, it would be $1.20. That difference is arithmetic under the assumptions, not a guaranteed hit or a measured invoice. Output, writes and other operations must still be accounted for.
What state does the server hold?
A conventional full KV layout can be sized independently of API prices:
bytes = positions × layers × KV heads × head dimension × 2 (K and V) × bytes per elementFor 32 layers, eight KV heads, 128 dimensions and two-byte elements, one million positions require 131,072,000,000 logical bytes: about 131.07 decimal GB or 122.07 GiB for one sequence.
Weights, temporary buffers, allocator overhead and concurrent sequences are additional requirements. Sharding changes where these bytes reside; it does not make the logical state disappear. Grouped-query attention, quantized KV and other architectures change the stored representation. The full-head formula must not be applied unchanged to DeepSeek’s MLA.
The long-context article explains why this capacity calculation is separate from whether a model can use the evidence correctly.
Transfer is another calculation
If a full 1 GiB must cross a perfect 400 Gbit/s raw link, serialization alone takes about 21.5ms. Lookup, storage access, copies, contention and protocol work add to that lower bound. A pipeline may also overlap some transfers with computation.
This is not an end-to-end restore measurement. Real network bytes depend on representation, which blocks are missing, compression, topology and the implementation. The arithmetic checker records the simple assumptions.
A cache hit can reduce recomputation while introducing or exposing restore work. Looking only at the customer’s read price would not tell us which physical resource dominated.
Infrastructure cost needs a workload
To estimate the service’s cost, we need accelerator time, memory occupancy, network/storage work and utilization over an actual workload. Batching can share weight reads. A long output occupies state for longer. Queueing and a strict per-user latency target can require spare capacity that a pure throughput test does not reveal.
A high-throughput benchmark at a large batch and a fast interactive endpoint may have different economic operating points. A theoretical peak divided by a cloud hourly rate is not automatically the cost of a successful request.
Likewise, an API price does not expose the provider’s margin. It can reflect product choices, discounts and how workloads are pooled. Keep any margin hypothesis labelled as a hypothesis.
Measure cost per useful outcome
A session may include several model requests, tool calls, retries, compaction and warming. Summing only the final successful response undercounts its cost. Comparing token prices while the task succeeds less often can also be misleading.
For a bounded experiment I would publish the exact model/endpoint, input/output mix, cache settings, concurrency, measurement boundary and sample window. Show known usage, unresolved attempts and successful-task counts. Keep the raw receipt available where it contains only personal synthetic data.
The calculator teaches the relationships. It does not replace those measurements. The question worth answering is what it costs to complete the useful task at the required quality and latency, not how impressive a context number looks on its own.