AI-103 Practice Question 45: Mini Model for Classification

Microsoft's own model-choice guidance sizes model families to task complexity: flagship models add reasoning depth at the cost of latency, while their mini and nano variants scale latency and cost down and are the right fit once a task no longer needs that extra reasoning. A fixed five-way classification over a short prompt is exactly that simpler case; a mini-class model reaches comparable accuracy on the narrow task at a fraction of the latency and cost.

- A is wrong because the flagship model's deeper reasoning adds latency and cost that a bounded, low-complexity classification task doesn't need; accuracy on this task doesn't scale with reasoning depth the way it does on genuinely hard problems. - B is wrong because capping max_tokens only limits how long the completion can be; it doesn't change the larger model's per-token processing cost or the latency inherent in a bigger model, which is what's driving the slowdown here. - C is wrong because parallel instances of an expensive model improve throughput and availability under load, not the fundamental per-request cost and latency of using an oversized model for a simple task.

Reference: GPT-5 vs GPT-4.1 - choosing the right model for your use case