AI-300 Practice Question 10: Prompt Cache Prefix Ordering
Microsoft Machine Learning Operations Engineer Associate (AI-300) · Test 1 · question 10 of 10
A support assistant on a Standard deployment sends a 3,500-token prompt on every call. The system message opens with a line carrying the current UTC timestamp and the signed-in user's ID, followed by roughly 3,000 tokens of fixed policy instructions and few-shot examples; the user's question is appended last. Traffic is steady all day, but cached_tokens comes back as 0 on virtually every response and input-token spend keeps climbing. What should you change first?
Show answer and explanation
Answer:
Prompt caching reuses processed input-token computations only for prompts that have identical content at the beginning. Two conditions must both hold: the request is at least 1,024 tokens long, and the first 1,024 tokens are identical. A single character difference in the first 1,024 tokens results in a cache miss, reported as cached_tokens: 0, and a timestamp plus a user ID at the very top of the system message changes on literally every call. Microsoft's first listed best practice is to place stable or repeated content at the beginning of the prompt and dynamic content at the end, which is exactly what option C does, turning the 3,000 tokens of fixed instructions into a cacheable prefix billed at a discount on input-token pricing.
- A is wrong because it disqualifies the prompt entirely. A request must be a minimum of 1,024 tokens to be eligible for caching at all, so shrinking below that guarantees every request is processed at full price. - B is wrong because retention governs how long an existing cached prefix stays alive, not whether one is ever written. With a per-request timestamp leading the prompt no two calls share a prefix, so there is nothing in the cache for a longer lifetime to preserve. - D is wrong because a breakpoint only marks where a reusable prefix ends. The prefix still begins with content that differs on every call, so it can never match. The parameter is also limited to GPT-5.6 and later model families, and earlier models return a 400 error for it.
Reference: Prompt caching with Azure OpenAI in Microsoft Foundry Models