Prompt caching
Cached input costs a fraction of fresh input. Whether a request hits the cache depends on how your code builds it.
Providers cache the start of a prompt. When a later request begins with exactly the same tokens, those tokens are billed as cached input, at a fraction of the input price. OhMyGPT passes your prompt to the provider as you built it, so your hit rate comes from your own code: how stable the start of each prompt is, and whether the provider can tell which requests belong to one conversation.
Measured on anthropic/claude-haiku-4-5 with the same 9,150-token conversation:
| Request | Cache | Cost |
|---|---|---|
| First turn | writes 9,147 tokens | 3,168 Credits |
| Next request with the same start | reads 9,147 tokens | 275 Credits |
| Same prompt, never cached | none | 2,535 Credits |
Check your hit rate
- One request: its receipt shows
usage.cache_read_tokensandusage.cache_creation_tokens. - Over time: the Cache read column on the Usage page, or
tokens.cache_readfromGET /v1/usage/summary(admin key), grouped by model or by API key.
A low hit rate is normal for one-off requests. In chats and agent loops, where every turn resends the earlier turns, most input should come from the cache.
Keep the start of the prompt identical
The cache matches from the first token up to the first difference. Everything after a change is billed as fresh input.
- Put what stays the same first: system prompt, tool definitions, long reference documents. Put the new user message last.
- Keep the system prompt byte-for-byte stable. Timestamps, request IDs, user names or randomly ordered examples in it break every hit; send them in the latest message instead.
- Append to the conversation. Editing, trimming or summarizing earlier messages invalidates the cache from that message on, so do it rarely and in large steps.
- Send the same tools in the same order on every turn, and serialize JSON with a stable key order.
- Stay on one model within a conversation. Each model keeps its own cache.
- Very short prompts are not cached. The minimum length depends on the model.
Mark which requests belong together
Send the same per-conversation value on every turn so the provider routes repeated prompts to the same cache:
| Endpoint | Field |
|---|---|
| Chat Completions, Responses | prompt_cache_key, or provider.conversation_id (up to 256 bytes) |
| Anthropic Messages | metadata.user_id |
curl https://api.ohmygpt.com/v1/responses \
-H "Authorization: Bearer $OHMYGPT_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "openai/gpt-5.4-nano", "prompt_cache_key": "conv-8f2c", "instructions": "<the same long instructions every turn>", "input": "Hi"}'Sent twice with an 8,351-token prompt, the second response reported "cached_tokens": 7936 in usage.input_tokens_details.
Claude caches only what the request marks for caching. When you call Claude through Chat Completions or Responses, OhMyGPT marks multi-turn requests (a conversation key, or an earlier assistant turn in the history) for you. Single-turn requests stay unmarked, because writing the cache costs more than plain input; in the table above, the never-cached prompt is the same request sent without an assistant turn. Through Anthropic Messages, the request reaches Claude as you sent it: add cache_control yourself. Claude Code does this and sends its session ID on every request, so it works out of the box.