Use prompt caching when the same large context appears across many requests. Mark stable instructions, documents, examples, tool outputs, or tool definitions as cacheable so supported providers can reuse that context on later calls.
Prompt caching is different from response caching. Prompt caching still runs inference, but can reduce repeated input processing cost and latency. Response caching returns a previously generated answer for an identical request.
Prompt caching is provider- and model-specific. Unsupported providers ignore cache hints or route without cached pricing. Check the model page pricing table for cache read/write rates.
What to cache
Cache content that stays stable across requests:
- long system instructions
- reused RAG documents
- few-shot examples
- tool definitions
- large tool results that are reused in the next turn
Avoid caching content that changes every request, contains short one-off user input, or includes sensitive data that your policy does not allow to be stored by the selected provider.
Cache controls
Phaseo accepts an OpenRouter-style top-level cache_control hint on Chat Completions, Responses, and Anthropic Messages requests:
{
"cache_control": {
"type": "ephemeral",
"ttl": "5m"
}
}
ttl: "5m" for short-lived shared context and ttl: "1h" when the provider and model support longer-lived prompt cache entries. A top-level cache control is treated as an automatic/default cache policy by providers that support it.
You can also place cache_control directly on supported text, image, tool result, and tool definition blocks for explicit cache breakpoints:
{
"type": "text",
"text": "Large stable reference text...",
"cache_control": {
"type": "ephemeral",
"ttl": "1h"
}
}
provider_options:
{
"provider_options": {
"anthropic": {
"cache_control": {
"type": "ephemeral",
"ttl": "5m",
"scope": "last_user_message"
}
}
}
}
scope values:
| Scope | Behavior |
|---|
all_text | Add cache control to system text and user text/image blocks that do not already have cache control. |
last_user_message | Add cache control only to the latest user message. |
none | Do not apply a default cache policy. |
cache_control wins over the default policy.
Chat Completions
Use /v1/chat/completions when you are using OpenAI-compatible chat clients.
curl https://api.phaseo.ai/v1/chat/completions \
-H "Authorization: Bearer $PHASEO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "anthropic/claude-sonnet-4",
"messages": [
{
"role": "system",
"content": [
{
"type": "text",
"text": "You are a support assistant. Follow the company policy exactly.",
"cache_control": { "type": "ephemeral", "ttl": "1h" }
}
]
},
{
"role": "user",
"content": "Summarise the latest ticket."
}
]
}'
{
"prompt_cache_retention": "24h"
}
{
"provider_options": {
"openai": {
"prompt_cache_retention": "24h"
}
}
}
Responses
Use /v1/responses for new OpenAI-compatible text integrations and agent flows.
curl https://api.phaseo.ai/v1/responses \
-H "Authorization: Bearer $PHASEO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "anthropic/claude-sonnet-4",
"input": [
{
"role": "user",
"content": [
{
"type": "input_text",
"text": "Reference document: Refunds are available for 30 days when...",
"cache_control": { "type": "ephemeral", "ttl": "5m" }
},
{
"type": "input_text",
"text": "Answer this customer: Can I return an item after 20 days?"
}
]
}
]
}'
provider_options.google.cached_content:
{
"provider_options": {
"google": {
"cached_content": "cachedContents/abc123"
}
}
}
Anthropic Messages
Use /v1/messages when your client is Anthropic-compatible.
curl https://api.phaseo.ai/v1/messages \
-H "Authorization: Bearer $PHASEO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "anthropic/claude-sonnet-4",
"max_tokens": 512,
"system": [
{
"type": "text",
"text": "You are a careful support assistant. Use the policy below.",
"cache_control": { "type": "ephemeral", "ttl": "1h" }
}
],
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Policy: refunds are available for 30 days when...",
"cache_control": { "type": "ephemeral", "ttl": "5m" }
},
{
"type": "text",
"text": "Can this customer return an item after 20 days?"
}
]
}
],
"tools": [
{
"name": "lookup_order",
"description": "Look up order status.",
"input_schema": {
"type": "object",
"properties": {
"order_id": { "type": "string" }
},
"required": ["order_id"]
},
"cache_control": { "type": "ephemeral", "ttl": "5m" }
}
]
}'
system text blocks- message text and image blocks
- tool result blocks
- tool definitions
Usage and pricing fields
When a provider returns cache usage, Phaseo normalizes it into common usage fields.
| Field | Meaning |
|---|
input_tokens_details.cached_tokens | Cached input tokens read from a provider prompt cache. |
output_tokens_details.cached_tokens | Cached input tokens written to a provider prompt cache. |
cached_read_text_tokens | Pricing meter for cache reads. This is cached input text reused from a provider cache. |
cached_write_text_tokens | Pricing meter for cache writes when the provider has one cache-write price. |
cached_write_text_tokens_5m | Cache write tokens for a 5 minute TTL when the provider reports TTL-specific writes. |
cached_write_text_tokens_1h | Cache write tokens for a 1 hour TTL when the provider reports TTL-specific writes. |
Practical checks
After you add prompt caching:
- Send one request to create or warm the cache.
- Send a second request with the same cacheable content.
- Check the response usage and request details for cached read/write fields.
- Compare latency and cost over repeated calls, not just the first call.
Related pages
