AI-103 Practice Question 37: Prompt Caching Prefix
Microsoft Azure AI Apps and Agents Developer Associate (AI-103) · Test 4 · question 7 of 10
A support assistant sends a 4,000-token system message (product catalogue and answering policy, unchanged for months) on every request. To help the model with relative dates, the application prefixes each system message with a line reading Current time: 2026-05-12T09:41:07Z, then appends the user's question. Costs are high and latency is poor, and every response reports cached_tokens as 0 even under steady traffic.
What should the team change?
Show answer and explanation
Answer:
Prompt caching has two entry conditions: the prompt must be at least 1,024 tokens, and the first 1,024 tokens must be identical between requests. A second-resolution timestamp at the very top changes the prefix on every single call, and Microsoft is explicit that "a single character difference in the first 1,024 tokens results in a cache miss, which is characterized by a cached_tokens value of 0", the symptom reported here. The documented best practice is the fix: place stable or repeated content at the beginning of the prompt and dynamic content at the end.
- A is wrong because retention controls how long a cached prefix survives, not whether one is ever written. With a prefix that never repeats there is nothing for a longer lifetime to preserve. - B is wrong because caching is already enabled by default on supported models; the prompt shape, not the deployment type, is the problem. (Provisioned deployments do get a larger cache read discount, but they also don't support prompt cache breakpoints.) - D is wrong and would make things worse: a prompt shorter than 1,024 tokens is not cacheable at all, so trimming the system message below the threshold removes the only chance of a hit.
Reference: Prompt caching with Azure OpenAI in Microsoft Foundry Models