# Prompt caching (/docs/prompt-caching)

Providers cache the start of a prompt. When a later request begins with exactly the same tokens, those tokens are billed as cached input, at a fraction of the input price. OhMyGPT passes your prompt to the provider as you built it, so your hit rate comes from your own code: how stable the start of each prompt is, and whether the provider can tell which requests belong to one conversation.

Measured on `anthropic/claude-haiku-4-5` with the same 9,150-token conversation:

| Request | Cache | Cost |
|---|---|---|
| First turn | writes 9,147 tokens | 3,168 Credits |
| Next request with the same start | reads 9,147 tokens | 275 Credits |
| Same prompt, never cached | none | 2,535 Credits |

## Check your hit rate [#check-your-hit-rate]

- One request: its [receipt](/docs/costs#look-up-a-request) shows `usage.cache_read_tokens` and `usage.cache_creation_tokens`.
- Over time: the **Cache read** column on the [Usage](https://www.ohmygpt.com/usage) page, or `tokens.cache_read` from [`GET /v1/usage/summary`](/docs/api/usage/getUsageSummary) (admin key), grouped by model or by API key.

A low hit rate is normal for one-off requests. In chats and agent loops, where every turn resends the earlier turns, most input should come from the cache.

## Keep the start of the prompt identical [#keep-the-start-of-the-prompt-identical]

The cache matches from the first token up to the first difference. Everything after a change is billed as fresh input.

- Put what stays the same first: system prompt, tool definitions, long reference documents. Put the new user message last.
- Keep the system prompt byte-for-byte stable. Timestamps, request IDs, user names or randomly ordered examples in it break every hit; send them in the latest message instead.
- Append to the conversation. Editing, trimming or summarizing earlier messages invalidates the cache from that message on, so do it rarely and in large steps.
- Send the same tools in the same order on every turn, and serialize JSON with a stable key order.
- Stay on one model within a conversation. Each model keeps its own cache.
- Very short prompts are not cached. The minimum length depends on the model.

## Mark which requests belong together [#mark-which-requests-belong-together]

Send the same per-conversation value on every turn so the provider routes repeated prompts to the same cache:

| Endpoint | Field |
|---|---|
| Chat Completions, Responses | `prompt_cache_key`, or `provider.conversation_id` (up to 256 bytes) |
| Anthropic Messages | `metadata.user_id` |

```bash
curl https://api.ohmygpt.com/v1/responses \
  -H "Authorization: Bearer $OHMYGPT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "openai/gpt-5.4-nano", "prompt_cache_key": "conv-8f2c", "instructions": "<the same long instructions every turn>", "input": "Hi"}'
```

Sent twice with an 8,351-token prompt, the second response reported `"cached_tokens": 7936` in `usage.input_tokens_details`.

Claude caches only what the request marks for caching. When you call Claude through Chat Completions or Responses, OhMyGPT marks multi-turn requests (a conversation key, or an earlier assistant turn in the history) for you. Single-turn requests stay unmarked, because writing the cache costs more than plain input; in the table above, the never-cached prompt is the same request sent without an assistant turn. Through Anthropic Messages, the request reaches Claude as you sent it: add [`cache_control`](https://platform.claude.com/docs/en/build-with-claude/prompt-caching) yourself. Claude Code does this and sends its session ID on every request, so it works out of the box.