Prompt Caching
To reduce inference costs, you can enable prompt caching for supported providers and models.
For most providers, caching is enabled automatically. However, for some (e.g., Anthropic), you need to enable it separately for each message.
When caching is used (automatically — for supported models, or explicitly via the cache_control property), the system tries to route requests to the same provider whenever possible to take advantage of an already warm cache. If the provider with the cached prompt is unavailable, the request will be routed to the next most suitable provider.
Viewing Cache Usage
To find out how much cache was saved per transaction, you can:
- Click the "Export to Excel" button on the Transactions page
- Use
usage: {include: true}in your request to get cache tokens at the end of the response in Text Generation
The cache_discount field in the response body will show how much was saved through cache usage. Some providers, like Anthropic, provide a negative discount for writing to the cache but a positive discount (which reduces total cost) for reading from the cache.
OpenAI
Caching pricing changes:
- Cache write: free
- Cache read: (depending on the model) costs 0.25 or 0.50 of the original input price
Click here to see OpenAI's caching prices per model.
Prompt caching in OpenAI is automatic and requires no additional setup. The minimum prompt size is 1024 tokens.
Click here to learn more about OpenAI prompt caching and its limitations.
Grok
Caching pricing changes:
- Cache write: free
- Cache read: costs 0.25x the original price
Click here to view Grok caching prices per model.
Prompt caching with Grok is automatic and requires no additional setup.
Moonshot AI
Caching pricing changes:
- Cache write: free
- Cache read: costs 0.25x the original price
Prompt caching with Moonshot AI is automatic and requires no additional setup.
Click here to view Moonshot AI caching documentation.
Groq
Caching pricing changes:
- Cache write: free
- Cache read: costs 0.5x the original price
Prompt caching with Groq is automatic and requires no additional setup. Currently available on Kimi K2 models.
Click here to view Groq documentation.
Anthropic Claude
Caching pricing changes:
- Cache write: costs 1.25x the original price
- Cache read: costs 0.1x the original price
Prompt caching with Anthropic requires the use of cache_control breakpoints. There is a limit of four breakpoints. By default, the cache expires after 5 minutes, but you can extend it to 1 hour by specifying "ttl": "1h" in the cache_control object. It is recommended to use cache breakpoints for large bodies of text, such as character cards, CSV data, RAG data, book chapters, etc.
Click here to learn more about Anthropic prompt caching and its limitations.
cache_control can only be set on the text part of a composite message.
Cache TTL Options
Two cache TTL values are supported for Anthropic:
- 5 minutes (default):
"cache_control": { "type": "ephemeral" } - 1 hour:
"cache_control": { "type": "ephemeral", "ttl": "1h" }
The 1-hour TTL is useful for long sessions where you need to keep cached content for multiple requests without incurring additional cache write costs.
Examples
Example of caching system messages (default TTL of 5 minutes):
{
"messages": [
{
"role": "system",
"content": [
{
"type": "text",
"text": "You are a historian studying the fall of the Roman Empire. You know the following book very well:"
},
{
"type": "text",
"text": "HUGE TEXT BODY",
"cache_control": {
"type": "ephemeral"
}
}
]
},
{
"role": "user",
"content": [
{
"type": "text",
"text": "What triggered the collapse?"
}
]
}
]
}
Example of caching messages with a 1-hour TTL:
{
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Given the book below:"
},
{
"type": "text",
"text": "HUGE TEXT BODY",
"cache_control": {
"type": "ephemeral",
"ttl": "1h"
}
},
{
"type": "text",
"text": "Name all the characters in the above book"
}
]
}
]
}
DeepSeek
Caching pricing changes:
- Cache write: costs 1x the original price
- Cache read: costs 0.1x the original price
Prompt caching with DeepSeek is automatic and requires no additional setup.
Google Gemini
Implicit Caching
Gemini 2.5 Pro and 2.5 Flash models now support implicit caching, providing automatic caching functionality similar to OpenAI's auto-caching. Implicit caching works seamlessly — no manual setup or additional cache_control breakpoints are required.
Pricing changes:
- No charge for cache write or storage.
- Cached tokens cost 0.25x the original price
Note that the latency averages 3-5 minutes but may vary. Caching requests for Gemini 2.5 Pro requires at least 2048 tokens, and Gemini 2.5 Flash requires at least 1028 tokens.
Official announcement from Google
[TIP]
To maximize implicit cache hits, keep the beginning of your message arrays consistent between requests. Add changes (e.g., user questions or dynamic context items) at the end of the request.
:::
Cached Request Pricing Changes:
- Cache write: charged at the input token price plus 5 minutes of cache storage, calculated as follows:
Cache write cost = input token price + (cache storage price × (5 minutes / 60 minutes))
- Cache read: costs 0.25x the original price
Supported Models and Limitations:
Caching is only supported by select Gemini models. Please check the latest information in the Google Gemini API pricing documentation.
Cache entries expire after 5 minutes (Time-to-Live, TTL), and the TTL is not refreshed. After 5 minutes, the cache is deleted and a new one must be created.
To write to the cache with Gemini models, a minimum of 4096 tokens is typically required. Cached tokens count toward the maximum number of tokens the model can use. Gemini 2.5 Pro uses a minimum of 2048 tokens, and Gemini 2.5 Flash uses a minimum of 1028 tokens.
How Gemini Prompt Caching Works in BotHub:
Bothub simplifies Gemini cache management by handling all complexities:
- You do not need to manually create, update, or delete caches.
- You do not need to explicitly manage cache names or TTL.
How to Enable Gemini Prompt Caching:
For Gemini caching via Bothub, you must explicitly specify cache_control breakpoints in the message content, similar to Anthropic. We recommend using caching primarily for large content chunks (such as CSV files, long character cards, RAG data, or extensive text sources).
[TIP]
There is no limit on the number of cache_control breakpoints you can include in your request. Bothub will only use the last breakpoint for Gemini caching. Including multiple breakpoints is safe and can help maintain compatibility with Anthropic, but only the last one will be used for Gemini.
:::
Examples:
Example of caching system messages
{
"messages": [
{
"role": "system",
"content": [
{
"type": "text",
"text": "You are a historian studying the fall of the Roman Empire. Below is an extensive reference book:"
},
{
"type": "text",
"text": "HUGE TEXT BODY HERE",
"cache_control": {
"type": "ephemeral"
}
}
]
},
{
"role": "user",
"content": [
{
"type": "text",
"text": "What triggered the collapse?"
}
]
}
]
}
Example of caching user messages
{
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Based on the book text below:"
},
{
"type": "text",
"text": "HUGE TEXT BODY HERE",
"cache_control": {
"type": "ephemeral"
}
},
{
"type": "text",
"text": "List all main characters mentioned in the text above."
}
]
}
]
}