How to cut agent costs with prompt caching
An agent resends its whole conversation on every turn. Measure what prompt caching saves, and find the changes in your code that break it.
An agent resends the whole conversation on every turn, so input tokens make up most of the bill. Prompt caching lowers that cost: the provider stores the start of the prompt, and a later request that begins with the same tokens pays a lower price for them.
See what a hit saves
We measured one conversation of 9,150 tokens on anthropic/claude-haiku-4-5. Charges are in Credits, and 250,000 Credits equal 1 USD.
| Request | Charge in Credits |
|---|---|
| Not cached | 2,535 |
| First request, which writes the cache | 3,168 |
| Later request, which reads the cache | 275 |
The first request costs 633 Credits more than a request without a cache. The second request saves 2,260 Credits, so the cache has paid for itself after one hit.
Ten requests on that conversation cost 25,350 Credits without a cache. With a cache, they cost 5,643 Credits, which is 78% less. Your own saving depends on the model, the length of the shared start, and how many requests reuse it.
Check your own hit rate
Read the usage of one request first. The receipt of a request contains two counters:
usage.cache_creation_tokenscounts the tokens that the request wrote to the cache.usage.cache_read_tokenscounts the tokens that the request read from the cache.
If cache_read_tokens stays at 0 from the second turn on, something in your code changes the start of the prompt. For a longer period, use the Cache savings column on the Usage page.
Find what breaks the cache
The cache matches from the first token to the first difference. Every token after that difference pays the full input price. Look for these causes:
- The system prompt contains a value that changes, such as a timestamp, a request ID, or a user name. Move the value to the latest user message.
- The tools arrive in a different order, or their JSON keys do. Sort both.
- The code edits or summarizes earlier messages on every turn. Do it rarely, and in large steps.
- The conversation moves between models. Each model keeps its own cache.
- The prompt is too short. Each model sets a minimum length.
Mark which requests belong together
Send the same value on every turn of a conversation, so that the provider can send repeated prompts to the same cache. On Chat Completions and Responses, use prompt_cache_key:
curl https://api.ohmygpt.com/v1/responses \
-H "Authorization: Bearer $OHMYGPT_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "openai/gpt-5.4-nano", "prompt_cache_key": "conv-8f2c", "instructions": "<the same long instructions every turn>", "input": "Hi"}'We sent this request twice with a prompt of 8,351 tokens. The second response reported 7,936 cached tokens.
Claude models cache only what a request marks. If you call Claude through Anthropic Messages, add cache_control yourself. Prompt caching describes the fields for each protocol.
Know when caching costs more
On Claude models, a cache write costs more than plain input. If a prompt is sent once, do not mark it. Caching pays when at least one later request reads the same start.