If you’ve ever worked with AI APIs—whether you’re building an app, running batch jobs, or just automating tasks—you’ve probably noticed that costs can add up fast. Every token you send gets processed and billed, even if you’re sending the same context over and over.
That’s where prompt caching comes in. It’s a feature that lets AI models store and reuse parts of your prompt without re-processing them every single time. The result? Faster responses and lower costs, especially when you’re working with long or repetitive contexts.
Here’s what you need to know about how it works, which models support it, and when it actually makes a difference.
How Prompt Caching Works
When you send a prompt to an AI model via API, the model processes every token—your instructions, any documents or context you’ve included, and your actual question. If you’re asking follow-up questions or running similar queries, you’re often resending the same background information each time.
Prompt caching allows the model to store the unchanging parts of your prompt—like a long document, a system instruction, or a knowledge base—so it doesn’t have to re-read and re-process them with every request. The model keeps that cached content in memory for a set period (usually a few minutes), and only processes the new or changing parts of your prompt.
For example, if you’re analyzing a 50-page PDF and asking multiple questions about it, the model can cache the PDF content and only process your individual questions. Instead of paying to process 50 pages ten times, you pay for it once, plus the cost of your ten short questions.
Which AI Models Support Prompt Caching
As of 2026, several major AI providers offer prompt caching, though the implementation and pricing vary:
- Anthropic’s Claude models support prompt caching across Claude 3.5 Sonnet, Claude 3 Opus, and Claude 3 Haiku. Cached tokens cost about 90% less than regular input tokens.
- OpenAI introduced caching features for GPT-4 Turbo and GPT-4o, with automatic caching on repeated content in certain API usage patterns.
- Google’s Gemini API supports context caching for Gemini 1.5 Pro and Gemini 1.5 Flash, allowing you to cache up to millions of tokens for reuse.
Check your provider’s documentation for specifics—caching behavior, time limits, and pricing differ across platforms.
When Prompt Caching Actually Saves You Money
Prompt caching isn’t useful for every use case. It shines when you’re working with:
- Long, repeated context: Analyzing the same document, codebase, or dataset multiple times
- System instructions: Using detailed system prompts (e.g., a custom persona or set of rules) across many API calls
- Batch processing: Running similar queries over a fixed knowledge base
- Chatbots with static context: Customer support bots that reference the same product documentation or FAQ
If your prompts are short, unique, or constantly changing, caching won’t help much. The savings come from avoiding repeated processing of large, static content.
How to Use Prompt Caching in Practice
Implementation depends on your provider, but the general workflow is similar:
- Structure your prompt so the static content (the part you want cached) comes first
- Use the provider’s API parameters to enable caching—Claude uses a
cache_controlparameter, for example - Send your requests within the cache window (typically 5–10 minutes) to reuse the cached content
- Monitor your usage and costs to confirm you’re seeing savings
Most providers automatically manage cache expiration, so you don’t need to manually clear it. Just be aware that if you wait too long between requests, the cache expires and you’ll pay full price again.
The Bottom Line
Prompt caching is one of those quiet features that doesn’t get much attention, but can make a big difference if you’re using AI at scale. It won’t change how you write prompts day-to-day, but if you’re working with APIs, long documents, or repetitive tasks, it’s worth setting up.
Check your AI provider’s documentation, test it with your workflow, and watch your token costs drop.
Want one useful AI idea delivered to your inbox every day? Subscribe to the One Two Three AI newsletter and stay sharp on what actually matters in AI.