Definition
Prompt caching
Reusing the processed form of a long, unchanging prompt prefix across requests, cutting input-token cost and latency.
Many applications send the same long instructions with every request. Prompt caching lets the model provider store the processed prefix and reuse it, so repeated requests pay a fraction of the normal input price for that part.
It only works if the stable content comes first and stays byte-identical: system instructions, then conversation history, then the new message. Anything volatile placed before the stable block breaks the cache, so cache-hit rate is worth monitoring.
Related services
Related terms