GCP Generative AI Leader Practice Question 48: Top p Nucleus Sampling
Google Cloud Generative AI Leader · Test 5 · question 8 of 10
A product team is tuning a model that drafts short marketing taglines. At each step, the model ranks every possible next word by probability, then keeps adding words from most probable to least probable only until the running total of their probabilities reaches a chosen threshold. The final word is then sampled from that shortened, dynamically sized list rather than from the model's entire vocabulary.
Which output parameter controls this behavior?
Show answer and explanation
Answer:
Top-p, also called nucleus sampling, builds its candidate list dynamically: it adds tokens from most probable to least probable until their combined probability reaches the top-p value, then samples the next token only from that set. The size of the candidate set changes from step to step depending on how confident the model's probability distribution is, which is exactly the behavior described.
Option A is wrong because temperature reshapes the probability distribution itself (making it flatter or sharper) rather than cutting the candidate list off at a cumulative probability threshold. Option B is wrong because top-k keeps a fixed count of the highest-probability tokens no matter how much total probability they add up to, while the scenario describes a threshold on the running total, not a fixed count. Option D is wrong because max output tokens limits the overall length of the response and has nothing to do with how an individual token is chosen.
Reference: Content generation parameters