AI-300 Practice Question 19: Chunk Overlap Tuning
Microsoft Machine Learning Operations Engineer Associate (AI-300) · Test 2 · question 9 of 10
A retrieval-augmented assistant over long policy PDFs indexes them as fixed-size chunks with no overlap. Users report answers that stop halfway through an explanation, and inspection of the retrieved passages shows that a multi-step procedure is routinely cut in two at a chunk boundary, with only the first half retrieved. What is the recommended first adjustment to the indexing strategy?
Show answer and explanation
Answer:
Microsoft's chunking guidance addresses this failure directly: "When you chunk data based on fixed size, overlapping a small amount of text between chunks can help maintain continuity and context. We recommend starting with a chunk size of 512 tokens (approximately 2,000 characters) and an initial overlap of 25%, which equals 128 tokens. This overlap ensures smoother transitions between chunks without excessive duplication." Overlap means the second half of the procedure also appears at the tail of the preceding chunk, so a single retrieved chunk carries the whole step sequence (option C).
- A is wrong because it treats a chunking defect as a retrieval-depth problem. The boundary still splits the procedure, so no single chunk becomes coherent; you just pay for far more prompt tokens and dilute the context with less relevant passages. - B is wrong because keyword-only search changes how passages are matched, not where they are cut. It also gives up the semantic matching that lets the assistant answer paraphrased questions. - D is wrong because it makes the symptom worse. Single-sentence chunks maximize fragmentation: each one carries less surrounding context, so a multi-step procedure is now split across a dozen boundaries instead of one.
Reference: Chunk documents in vector search - Azure AI Search