> ## Documentation Index
> Fetch the complete documentation index at: /docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Prompt caching

> The network reuses computation for prompt prefixes it has recently seen, automatically, with no code changes.

Before generating, a model must process your entire prompt. When the beginning of a request exactly matches the beginning of a recent one, same system prompt, same history, the network reuses that work instead of redoing it. You opt into nothing: the longest matching prefix is found automatically, time-to-first-token drops on the reused portion, and cached input tokens are billed at a discounted rate (live per-model rates on the dashboard's [Models](https://platform.pearlresearch.ai/models) page). Responses are unaffected, a cache hit changes cost and latency, never output quality.

## Seeing your hit rate

Cache hits show up in the response's usage as `prompt_tokens_details.cached_tokens`, the portion of `prompt_tokens` that was served from cache:

```json theme={null}
"usage": {
  "prompt_tokens": 5210,
  "completion_tokens": 318,
  "total_tokens": 5528,
  "prompt_tokens_details": {
    "cached_tokens": 4864
  }
}
```

Cached input tokens are also metered separately in your dashboard's Analytics tab, so you can watch the aggregate hit rate of a workload, not just one request.

## Structuring prompts for cache hits

Matching is **exact and prefix-only**: one differing token invalidates everything after it. The rule that follows is simple, put what never changes first, and what always changes last:

```python theme={null}
# Cache-friendly: everything stable first, everything variable last.
messages = [
    {"role": "system", "content": STATIC_INSTRUCTIONS},   # identical every request
    *conversation_history,                                # grows append-only
    {"role": "user", "content": user_input},              # changes every request
]

# Cache-hostile: a timestamp up front breaks the prefix on every call.
messages = [
    {"role": "system", "content": f"Now: {datetime.now()}. " + STATIC_INSTRUCTIONS},
    ...
]
```

* **Multi-turn chat caches naturally**, each turn re-sends the history, which is a growing prefix of the last request. This is where the discount compounds most.
* **Keep system prompts and tool definitions byte-stable.** Timestamps, random IDs, or per-user greetings at the top of the prompt zero out the cache; if the model needs the current time, put it in the latest user message instead.
* **Shared static context pays twice**, a long document or codebase prefix queried repeatedly is processed once and reused across all the follow-up questions.

## What caching does not promise

The cache is a best-effort accelerator, not a contract: entries are per-model, live for a limited time, and are evicted as capacity is needed. Never build correctness on a hit, treat `cached_tokens: 0` as a normal, always-possible outcome that only costs you the regular input rate.

> [!TIP]
>
> Agentic workloads, long system prompts, tool definitions, and an append-only transcript, are the textbook caching case: every loop iteration replays a huge stable prefix. Structure the prompt once, and the discount applies to the whole run.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.