The network reuses computation for prompt prefixes it has recently seen, automatically, with no code changes.
Before generating, a model must process your entire prompt. When the beginning of a request exactly matches the beginning of a recent one, same system prompt, same history, the network reuses that work instead of redoing it. You opt into nothing: the longest matching prefix is found automatically, time-to-first-token drops on the reused portion, and cached input tokens are billed at a discounted rate (live per-model rates on the dashboard’s Models page). Responses are unaffected, a cache hit changes cost and latency, never output quality.
Seeing your hit rate
Cache hits show up in the response’s usage as prompt_tokens_details.cached_tokens, the portion of prompt_tokens that was served from cache:
Cached input tokens are also metered separately in your dashboard’s Analytics tab, so you can watch the aggregate hit rate of a workload, not just one request.
Structuring prompts for cache hits
Matching is exact and prefix-only: one differing token invalidates everything after it. The rule that follows is simple, put what never changes first, and what always changes last:
# Cache-friendly: everything stable first, everything variable last.messages = [ {"role": "system", "content": STATIC_INSTRUCTIONS}, # identical every request *conversation_history, # grows append-only {"role": "user", "content": user_input}, # changes every request]# Cache-hostile: a timestamp up front breaks the prefix on every call.messages = [ {"role": "system", "content": f"Now: {datetime.now()}. " + STATIC_INSTRUCTIONS}, ...]
Multi-turn chat caches naturally, each turn re-sends the history, which is a growing prefix of the last request. This is where the discount compounds most.
Keep system prompts and tool definitions byte-stable. Timestamps, random IDs, or per-user greetings at the top of the prompt zero out the cache; if the model needs the current time, put it in the latest user message instead.
Shared static context pays twice, a long document or codebase prefix queried repeatedly is processed once and reused across all the follow-up questions.
What caching does not promise
The cache is a best-effort accelerator, not a contract: entries are per-model, live for a limited time, and are evicted as capacity is needed. Never build correctness on a hit, treat cached_tokens: 0 as a normal, always-possible outcome that only costs you the regular input rate.
Agentic workloads, long system prompts, tool definitions, and an append-only transcript, are the textbook caching case: every loop iteration replays a huge stable prefix. Structure the prompt once, and the discount applies to the whole run.
{"docsConfig":{"theme":"mint","$schema":"https://mintlify.com/docs.json","name":"Pearl Inference Platform","colors":{"primary":"#2A2623","light":"#EFEDEA","dark":"#2A2623"},"logo":{"light":"https://mintcdn.com/pearl-research/qjVPdghcpi_iWwgo/logo/light.svg?fit=max\u0026auto=format\u0026n=qjVPdghcpi_iWwgo\u0026q=85\u0026s=a969569c61ca3aa40012ac0f3dbbe445","dark":"https://mintcdn.com/pearl-research/qjVPdghcpi_iWwgo/logo/dark.svg?fit=max\u0026auto=format\u0026n=qjVPdghcpi_iWwgo\u0026q=85\u0026s=eacd546258e1218dea6ff14f55e89b4c"},"favicon":{"light":"https://mintcdn.com/pearl-research/jPtzRu8ZOdp5WYZT/favicon/pearl-light.svg?fit=max\u0026auto=format\u0026n=jPtzRu8ZOdp5WYZT\u0026q=85\u0026s=97c85bf573b7866d8612883b555a32c7","dark":"https://mintcdn.com/pearl-research/jPtzRu8ZOdp5WYZT/favicon/pearl-dark.svg?fit=max\u0026auto=format\u0026n=jPtzRu8ZOdp5WYZT\u0026q=85\u0026s=9dc503cf63b998d03fa165c37e0921e1"},"api":{"playground":{"display":"interactive"},"examples":{"prefill":true,"autogenerate":false},"url":"full"},"appearance":{"default":"light","strict":true},"background":{"color":{"light":"#EFEEEB"}},"navbar":{"links":[{"href":"mailto:support@pearlresearch.ai","label":"Support"}],"primary":{"type":"button","label":"Dashboard","href":"https://platform.pearlresearch.ai"}},"seo":{"metatags":{"canonical":"https://platform.pearlresearch.ai/docs"}},"fonts":{"heading":{"family":"Urbanist"},"body":{"family":"Inter"}},"contextual":{"options":["copy","view","chatgpt","claude","perplexity","mcp","cursor","vscode"],"display":"header"},"styling":{"latex":false},"navigation":{"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"pearl-research","hasAudioNarration":false,"actualSubdomain":"pearl-research","customDomains":[],"gitSource":{"type":"github","owner":"duplex-foundation","repo":"cloud-platform-docs","deployBranch":"main","contentDirectory":"","isPrivate":true,"mountPath":"","label":"","_id":"6afac147dd50ae575c58a540"},"entitlements":{"AI_CHAT":{"status":"ENABLED"},"WEB_MCP":{"status":"ENABLED"},"NARRATION":{"status":"ENABLED"}},"buildId":"6aff1808a4a19f80cc241c5b","clientVersion":"0.0.4270","searchFilterCounts":[{"language":"default","tag":"Docs","count":37},{"language":"default","tag":"Integrations","count":16},{"language":"default","tag":"Inference API","count":14},{"language":"default","tag":"API Reference","count":12},{"language":"default","tag":"Account \u0026 Billing","count":6},{"language":"default","tag":"Get Started","count":6},{"language":"default","tag":"Management API","count":5},{"language":"default","tag":"Guides","count":2}]},"assistantConfig":{"enableAskAiButton":false,"enableStarterQuestions":false,"enableContextualStarterQuestions":false,"enableAskAiSearchBar":true},"basePath":"/docs","hasChatPermissions":true,"hasWebMcpPermissions":true,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-rbac-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"site-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"starter-pricing","enabled":true,"variant":{"name":"control","enabled":true,"payload":{"type":"string","value":"control"},"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"wildcard-email-auth","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"googleFontsInlined":true},"customCss":[]}
{"page":{"pagePath":"prompt-caching","slug":"/prompt-caching","pageMetadata":{"title":"Prompt caching","description":"The network reuses computation for prompt prefixes it has recently seen, automatically, with no code changes.","href":"/prompt-caching"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsx(_components.p, {\n children: \"The network reuses computation for prompt prefixes it has recently seen, automatically, with no code changes.\"\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,code:`code`,li:`li`,p:`p`,pre:`pre`,span:`span`,strong:`strong`,ul:`ul`,..._provideComponents(),...props.components},{CodeBlock,Heading,Tip}=_components;if(!CodeBlock)_missingMdxReference(`CodeBlock`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!Tip)_missingMdxReference(`Tip`,true);return _jsxs(_Fragment,{children:[_jsxs(_components.p,{children:[`Before generating, a model must process your entire prompt. When the beginning of a request exactly matches the beginning of a recent one, same system prompt, same history, the network reuses that work instead of redoing it. You opt into nothing: the longest matching prefix is found automatically, time-to-first-token drops on the reused portion, and cached input tokens are billed at a discounted rate (live per-model rates on the dashboard’s `,_jsx(_components.a,{href:`https://platform.pearlresearch.ai/models`,children:`Models`}),` page). Responses are unaffected, a cache hit changes cost and latency, never output quality.`]}),`\n`,_jsx(Heading,{level:`2`,id:`usage`,children:`Seeing your hit rate`}),`\n`,_jsxs(_components.p,{children:[`Cache hits show up in the response’s usage as `,_jsx(_components.code,{children:`prompt_tokens_details.cached_tokens`}),`, the portion of `,_jsx(_components.code,{children:`prompt_tokens`}),` that was served from cache:`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`8`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`8`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"usage\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"prompt_tokens\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`5210`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"completion_tokens\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`318`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"total_tokens\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`5528`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"prompt_tokens_details\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"cached_tokens\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`4864`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),`\n`,_jsx(_components.p,{children:`Cached input tokens are also metered separately in your dashboard’s Analytics tab, so you can watch the aggregate hit rate of a workload, not just one request.`}),`\n`,_jsx(Heading,{level:`2`,id:`structure`,children:`Structuring prompts for cache hits`}),`\n`,_jsxs(_components.p,{children:[`Matching is `,_jsx(_components.strong,{children:`exact and prefix-only`}),`: one differing token invalidates everything after it. The rule that follows is simple, put what never changes first, and what always changes last:`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`12`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`12`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# Cache-friendly: everything stable first, everything variable last.`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`messages `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` [`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` {`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"system\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#D4D4D4`},children:`STATIC_INSTRUCTIONS`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}, `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# identical every request`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` *`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`conversation_history, `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# grows append-only`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` {`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"user\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: user_input}, `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# changes every request`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# Cache-hostile: a timestamp up front breaks the prefix on every call.`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`messages `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` [`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` {`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"system\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Now: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`datetime.now()`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`. \"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` +`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#D4D4D4`},children:` STATIC_INSTRUCTIONS`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`},`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#D4D4D4`},children:` ...`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`})}),`\n`]})})}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`Multi-turn chat caches naturally`}),`, each turn re-sends the history, which is a growing prefix of the last request. This is where the discount compounds most.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`Keep system prompts and tool definitions byte-stable.`}),` Timestamps, random IDs, or per-user greetings at the top of the prompt zero out the cache; if the model needs the current time, put it in the latest user message instead.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`Shared static context pays twice`}),`, a long document or codebase prefix queried repeatedly is processed once and reused across all the follow-up questions.`]}),`\n`]}),`\n`,_jsx(Heading,{level:`2`,id:`guarantees`,children:`What caching does not promise`}),`\n`,_jsxs(_components.p,{children:[`The cache is a best-effort accelerator, not a contract: entries are per-model, live for a limited time, and are evicted as capacity is needed. Never build correctness on a hit, treat `,_jsx(_components.code,{children:`cached_tokens: 0`}),` as a normal, always-possible outcome that only costs you the regular input rate.`]}),`\n`,_jsx(Tip,{children:_jsx(_components.p,{children:`Agentic workloads, long system prompts, tool definitions, and an append-only transcript, are the textbook caching case: every loop iteration replays a huge stable prefix. Structure the prompt once, and the discount applies to the whole run.`})})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"config":{},"pageMetadata":{"title":"Prompt caching","description":"The network reuses computation for prompt prefixes it has recently seen, automatically, with no code changes.","href":"/prompt-caching"}}},"mdxSourceWithNoJs":null},"mdxExtracts":{"tableOfContents":[{"title":"Seeing your hit rate","slug":"usage","depth":2,"children":[]},{"title":"Structuring prompts for cache hits","slug":"structure","depth":2,"children":[]},{"title":"What caching does not promise","slug":"guarantees","depth":2,"children":[]}],"codeExamples":{}},"jsLinks":[],"subdomain":"pearl-research","siteName":"Pearl Inference Platform","title":"Prompt caching - Pearl Inference Platform","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{}},"lastModified":"2026-11-16T15:31:09.513Z","originalFileLocation":"prompt-caching.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=\"light\";if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"light\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"The network reuses computation for prompt prefixes it has recently seen, automatically, with no code changes.\">\u003cmeta data-mint-head name=\"application-name\" content=\"Pearl Inference Platform\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/docs/mintlify-assets/_mintlify/favicons/pearl-research/k9be1f6J_artjsFA/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/docs/mintlify-assets/_mintlify/favicons/pearl-research/k9be1f6J_artjsFA/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/docs/mintlify-assets/_mintlify/favicons/pearl-research/k9be1f6J_artjsFA/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/docs/mintlify-assets/_mintlify/favicons/pearl-research/k9be1f6J_artjsFA/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/docs/mintlify-assets/_mintlify/favicons/pearl-research/k9be1f6J_artjsFA/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/docs/mintlify-assets/_mintlify/favicons/pearl-research/k9be1f6J_artjsFA/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/docs/mintlify-assets/_mintlify/favicons/pearl-research/k9be1f6J_artjsFA/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/docs/mintlify-assets/_mintlify/favicons/pearl-research/k9be1f6J_artjsFA/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/docs/mintlify-assets/_mintlify/favicons/pearl-research/k9be1f6J_artjsFA/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Prompt caching - Pearl Inference Platform\">\u003cmeta data-mint-head property=\"og:description\" content=\"The network reuses computation for prompt prefixes it has recently seen, automatically, with no code changes.\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"Pearl Inference Platform\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://platform.pearlresearch.ai/docs/prompt-caching\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://pearl-research.mintlify.site/docs/_mintlify/api/og/v2?division=Inference+API\u0026amp;title=Prompt+caching\u0026amp;description=The+network+reuses+computation+for+prompt+prefixes+it+has+recently+seen%2C+automatically%2C+with+no+code+changes.\u0026amp;theme=8053f473600c09a776e77e44\u0026amp;signature=86c644e5da931fdf7ff64fa246eeb143c2bad013daf59d2fd0bf4aa55bfbd3b8\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Prompt caching - Pearl Inference Platform\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"The network reuses computation for prompt prefixes it has recently seen, automatically, with no code changes.\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://pearl-research.mintlify.site/docs/_mintlify/api/og/v2?division=Inference+API\u0026amp;title=Prompt+caching\u0026amp;description=The+network+reuses+computation+for+prompt+prefixes+it+has+recently+seen%2C+automatically%2C+with+no+code+changes.\u0026amp;theme=8053f473600c09a776e77e44\u0026amp;signature=86c644e5da931fdf7ff64fa246eeb143c2bad013daf59d2fd0bf4aa55bfbd3b8\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/docs/mintlify-assets/_mintlify/favicons/pearl-research/k9be1f6J_artjsFA/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"Pearl Inference Platform\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#2A2623\">\u003clink data-mint-head rel=\"canonical\" href=\"https://platform.pearlresearch.ai/docs/prompt-caching\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/docs/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/docs/prompt-caching.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://platform.pearlresearch.ai/#organization\",\"name\":\"Pearl Inference Platform\",\"url\":\"https://platform.pearlresearch.ai\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/pearl-research/qjVPdghcpi_iWwgo/logo/light.svg?fit=max\u0026auto=format\u0026n=qjVPdghcpi_iWwgo\u0026q=85\u0026s=a969569c61ca3aa40012ac0f3dbbe445\"}},{\"@type\":\"WebSite\",\"@id\":\"https://platform.pearlresearch.ai/docs#website\",\"name\":\"Pearl Inference Platform\",\"url\":\"https://platform.pearlresearch.ai/docs\",\"publisher\":{\"@id\":\"https://platform.pearlresearch.ai/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://platform.pearlresearch.ai/docs/prompt-caching#webpage\",\"url\":\"https://platform.pearlresearch.ai/docs/prompt-caching\",\"name\":\"Prompt caching\",\"description\":\"The network reuses computation for prompt prefixes it has recently seen, automatically, with no code changes.\",\"dateModified\":\"2026-11-16T15:31:09.513Z\",\"isPartOf\":{\"@id\":\"https://platform.pearlresearch.ai/docs#website\"},\"breadcrumb\":{\"@id\":\"https://platform.pearlresearch.ai/docs/prompt-caching#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://platform.pearlresearch.ai/docs/prompt-caching#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Inference API\",\"item\":\"https://platform.pearlresearch.ai/docs/inference\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Prompt caching\",\"item\":\"https://platform.pearlresearch.ai/docs/prompt-caching\"}]},{\"@type\":[\"Article\",\"TechArticle\"],\"@id\":\"https://platform.pearlresearch.ai/docs/prompt-caching#article\",\"headline\":\"Prompt caching\",\"name\":\"Prompt caching\",\"description\":\"The network reuses computation for prompt prefixes it has recently seen, automatically, with no code changes.\",\"url\":\"https://platform.pearlresearch.ai/docs/prompt-caching\",\"mainEntityOfPage\":{\"@id\":\"https://platform.pearlresearch.ai/docs/prompt-caching#webpage\"},\"dateModified\":\"2026-11-16T15:31:09.513Z\",\"publisher\":{\"@id\":\"https://platform.pearlresearch.ai/#organization\"},\"isPartOf\":{\"@id\":\"https://platform.pearlresearch.ai/docs#website\"}}]}\u003c/script>\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@type\":\"WebSite\",\"name\":\"Pearl Inference Platform\",\"publisher\":{\"@type\":\"Organization\",\"name\":\"Pearl Inference Platform\"},\"provider\":{\"@type\":\"Organization\",\"name\":\"Mintlify\",\"url\":\"https://mintlify.com\"}}\u003c/script>","agentPointerHref":"/docs/prompt-caching.md","siteName":"Pearl Inference Platform","nav":{"global":null,"tabs":[{"tab":"Docs","groups":[{"group":"Get Started","pages":[{"title":"Build on Pearl Inference Platform","sidebarTitle":"Introduction","href":""},{"title":"Quickstart","href":"/quickstart"},{"title":"Models","href":"/models"},{"title":"Concepts","href":"/concepts"},{"title":"Data residency","href":"/data-residency"},{"title":"Model deprecation","href":"/deprecation"}]},{"group":"Inference API","pages":[{"title":"Overview","sidebarTitle":"Overview","href":"/inference"},{"title":"Streaming","href":"/streaming"},{"title":"Function calling","href":"/function-calling"},{"title":"Structured outputs (JSON mode)","sidebarTitle":"Structured outputs","href":"/json-mode"},{"title":"Reasoning models","sidebarTitle":"Reasoning","href":"/reasoning"},{"title":"Vision","description":"Ask questions about images with GLM-5.3 Flash, sent inline as base64 data URIs.","href":"/vision"},{"title":"Prompt caching","href":"/prompt-caching"}]},{"group":"Account \u0026 Billing","pages":[{"title":"Plans \u0026 rate limits","description":"Start on Basic with lower limits, unlock higher Basic limits with a credit purchase of at least $5, or request Pro or Enterprise.","href":"/plans"},{"title":"Credits \u0026 payments","href":"/credits"},{"title":"Auto-recharge","href":"/auto-recharge"},{"title":"Usage \u0026 analytics","href":"/analytics"},{"title":"Team \u0026 roles","href":"/team"},{"title":"API keys","href":"/api-keys"}]},{"group":"Integrations","pages":[{"title":"Connect your tools to Pearl","sidebarTitle":"Overview","href":"/integrations"},{"title":"OpenCode setup","sidebarTitle":"OpenCode","href":"/integrations/opencode"},{"title":"Claude Code setup","sidebarTitle":"Claude Code","href":"/integrations/claude-code"},{"title":"Codex setup","sidebarTitle":"Codex","href":"/integrations/codex"},{"title":"Pi setup","sidebarTitle":"Pi","href":"/integrations/pi"},{"title":"Open WebUI setup","sidebarTitle":"Open WebUI","href":"/integrations/open-webui"},{"title":"n8n setup","sidebarTitle":"n8n","href":"/integrations/n8n"},{"title":"LiteLLM setup","sidebarTitle":"LiteLLM","href":"/integrations/litellm"},{"title":"Cline setup","sidebarTitle":"Cline","href":"/integrations/cline"},{"title":"Zed setup","sidebarTitle":"Zed","href":"/integrations/zed"},{"title":"Goose setup","sidebarTitle":"Goose","href":"/integrations/goose"},{"title":"Aider setup","sidebarTitle":"Aider","href":"/integrations/aider"},{"title":"Oh My Pi setup","sidebarTitle":"Oh My Pi","href":"/integrations/oh-my-pi"},{"title":"OpenAI SDK setup","sidebarTitle":"OpenAI SDK","href":"/integrations/openai-sdk"},{"title":"OpenAI Responses setup","sidebarTitle":"OpenAI Responses","href":"/integrations/openai-responses"},{"title":"Anthropic SDK setup","sidebarTitle":"Anthropic SDK","href":"/integrations/anthropic-sdk"}]},{"group":"Guides","pages":[{"title":"Coding agent setup","href":"/guides/coding-agents"},{"title":"Migrate from OpenAI","href":"/guides/migrate-from-openai"}]}]},{"tab":"API Reference","pages":[{"title":"API Reference","href":"/reference/inference"}]}]},"basePath":"","title":"Prompt caching - Pearl Inference Platform","deploymentId":"dpl_53YqbBPsHdrqSGqXiWPGU7csRVod","buildId":"dpl_53YqbBPsHdrqSGqXiWPGU7csRVod","dynamics":[],"chunks":[]}