GCP Generative AI Leader Practice Question 42: Reinforcement Learning Human Feedback
Google Cloud Generative AI Leader · Test 5 · question 2 of 10
A model vendor wants its chatbot to sound more helpful and less abrupt. Human raters review pairs of candidate responses to the same prompt and rank which one they prefer. Those rankings train a separate model that then scores new responses, and the chatbot is tuned to produce responses that earn a higher score over time.
Which machine learning approach does this describe?
Show answer and explanation
Answer:
This is reinforcement learning from human feedback (RLHF). Human raters rank candidate responses, and those rankings train a reward model that scores how well a response matches human preference. The chatbot is then optimized, through reinforcement learning, to produce outputs that earn a higher score from that reward model. This is exactly how vendors commonly align a model's tone and helpfulness after initial training.
Option A is wrong because the raters are not clustering data without guidance; they are actively supplying preference labels that become a reward signal, which is the opposite of an unsupervised task. Option C is wrong because zero-shot prompting describes how a model is used at inference time with no examples in the prompt, not a training technique for changing model behavior. Option D is wrong because the raters are not writing new training text; they are ranking existing responses, and that ranking data feeds a reward model rather than augmenting a text corpus.
Reference: RLHF on Google Cloud