AIF-C01 Practice Question 56: Human Evaluation Brand Voice
AWS Certified AI Practitioner (AIF-C01) · Test 6 · question 6 of 10
A marketing team must choose between two foundation models for writing campaign copy. The deciding criterion is the brand's voice (warm, concise, never pushy), and the brand team are the only recognised authority on what counts as on-brand. Automated similarity scores such as ROUGE and BERTScore have not lined up with their judgement in trials. Which Amazon Bedrock evaluation approach should the team use?
Show answer and explanation
Answer:
Amazon Bedrock evaluations offer both automatic and human-based paths. AWS describes model evaluation jobs that use human workers as the way to *bring human input to the model evaluation process*, where the workers *can be employees of your company or a group of subject-matter experts from your industry*, and human-based evaluations use a team of people who provide their ratings and preferences in relation to certain metrics. When the criterion is a subjective, company-owned notion of brand voice (and the team has already seen automated scores diverge from it) the people who define the standard are the correct evaluators (option C).
- A is wrong because the built-in automatic metrics compute things like accuracy and semantic robustness against a reference dataset. They measure whether the output is right, not whether it sounds like this particular brand. - B is wrong because a RAG evaluation scores how well a knowledge base *retrieves* relevant passages and grounds answers in them. The team is comparing two models' writing style, and there is no retrieval step in the question. - D is wrong because an LLM-as-a-judge job is a strong, cheap option for many subjective criteria, but here it is scoped to factual correctness, which is not the deciding criterion, and a judge model has no access to the brand team's private standard of what is on-brand.
Reference: Evaluate the performance of Amazon Bedrock resources