AI-103 Practice Question 57: RAG Chunking Strategy
Microsoft Azure AI Apps and Agents Developer Associate (AI-103) · Test 6 · question 7 of 10
A RAG pipeline embeds entire ten-page manual sections as single chunks. Retrieval usually returns a section that does contain the relevant sentence, but each chunk is so large that most of its content is unrelated to the question, bloating the prompt and burying the answer among irrelevant text. The team wants tighter, more targeted passages without losing the surrounding sentences a reader would need to understand a match that falls near a chunk boundary.
What should they change?
Show answer and explanation
Answer:
Azure AI Search's chunking guidance recommends defining "a fixed size that's sufficient for semantically meaningful paragraphs (for example, 200 words or 600 characters)" rather than whole sections, precisely because smaller chunks retrieve more precisely. The overlap addresses the other half of the problem: "overlapping a small amount of text between chunks can help maintain continuity and context" across a boundary, so a passage that starts near the end of one chunk still reads coherently. Shrinking chunk size without overlap would leave that boundary problem unsolved; the fix needs both.
- B is wrong because it's the opposite direction from what's needed; one vector per whole document makes every retrieval return the entire document, which is an even more extreme version of the bloated-context problem the team is trying to fix. - C is wrong because retrieving more of the same oversized chunks doesn't make any single chunk more targeted; it adds more irrelevant content to the prompt, not less. - D is wrong because embedding dimensionality affects how semantic similarity is measured, not what unit of text gets embedded and returned. It doesn't change chunk size, overlap, or precision.
Reference: Chunk documents for vector search