AI-103 Practice Question 65: Chat Completions History Truncation

The chat completions API is stateless: the caller is responsible for what goes in messages on every request, and Microsoft's guidance is direct: "it's your responsibility to ensure that the prompt and completion fall within the token limit," with truncation strategies such as removing the oldest complete turns while preserving the system message. That is exactly the app's problem: it resends unbounded history instead of managing it.

- A is wrong because a deployment's SKU governs throughput and capacity (tokens per minute, concurrency), not the model's maximum context length, which is a property of the model itself. - C is wrong because prompt caching reduces latency and cost for a repeated prefix; it does not change how many tokens count toward the context window; the full prompt is still subject to the limit. - D is wrong because Foundry Agent Service threads have a large but still finite message limit (up to 100,000 messages) and the service manages truncation for *its own* context assembly; switching architectures is a bigger change than the problem calls for, and it doesn't describe what the chat completions caller itself must do.

Reference: Work with chat completion models