AIF-C01 Practice Question 54: Top p Nucleus Sampling
AWS Certified AI Practitioner (AIF-C01) · Test 6 · question 4 of 10
While tuning a model in the Amazon Bedrock playground, a developer raises Top P from 0.5 to 0.9 and leaves every other inference parameter at its default. In Amazon Bedrock, what is the effect of that change?
Show answer and explanation
Answer:
AWS defines Top P as *the percentage of most-likely candidates that the model considers for the next token*. The model computes the cumulative probability distribution over possible next tokens and considers only the top P percent of it. A higher value increases the size of the pool and allows the model to consider less likely outputs; a lower value trims the pool to the safest tokens. Going from 0.5 to 0.9 therefore makes the output more varied (option D).
- A is wrong because response length is controlled by a separate family of parameters: maximum response length, penalties and stop sequences. Top P has no bearing on how long the answer is. - B is wrong because that describes near-greedy decoding, which you approach by driving *temperature* toward zero. Top P defines which tokens are eligible, not how often the top one wins. - C is wrong because a fixed *count* of candidates is Top K, not Top P. Top K of 50 means 50 tokens; Top P of 0.9 means however many tokens it takes to accumulate 90 percent of the probability mass, which may be two tokens or two hundred, depending on the prompt.
Reference: Influence response generation with inference parameters