Document chunk boundaries
How much neighbouring context is enough?
Adding nearby text can repair an incomplete passage, but it can also add duplication and conflicting material. Measure the information gained rather than choosing the largest context window available.
In this article
Identify what the selected passage is missing
Start with a failed answer and inspect the retrieved evidence. Does the passage need a section heading, one preceding condition, a table header or an entire explanation? These are different context requirements.
A fixed rule that adds the previous and next chunk is easy to implement. It can also pull in a new section, an obsolete policy or an unrelated table. Physical proximity in a file is useful evidence, but it is not the same as a logical relationship.
Prefer an explicit relationship when the document model provides one. A row can request its table header. A paragraph can request its heading path. A procedure step can request the condition attached to that procedure.
Count repeated evidence before counting tokens
Overlapping chunks can cause the same sentences to appear several times in the model request. Retrieving adjacent chunks may repeat the overlap again. This consumes space without adding information and can make one source appear more prominent than it should.
Deduplicate by source span or another reliable provenance relationship where possible. Text equality alone can be misleading because the same sentence may appear in different document versions or under different conditions.
Measure the assembled context, not just the stored chunk size. Metadata, headings and repeated labels all contribute to what the model receives. A configuration that looks compact in the index can still produce a large request after expansion.
Run a bounded comparison
Compare a few explicit policies against the same questions and documents. One policy might use only the retrieved passage. Another might add heading context. A third might add the relevant structural parent or a limited neighbouring span.
For each policy, inspect answer correctness, missing qualifications, unrelated content and request size. Include questions that span a boundary and questions that should remain within one section. The latter reveal whether expansion introduces material the answer did not need.
Use the chosen model's actual tokenisation and limits when estimating capacity. Do not rely on one universal characters-to-tokens conversion, especially for tables, identifiers or mixed-language documents. Keep room for the question, instructions and answer as well.
Set a rule that can fail visibly
Define a maximum expansion budget and what happens when the needed context exceeds it. The application may ask a narrower question, return source links or use a separate document-reading workflow. Silently truncating the end can remove the exception or note that expansion was meant to preserve.
Monitor how often expansion reaches its limit and which document types cause it. That information can justify a specialised ingestion rule instead of repeatedly increasing a global budget.
Choose the smallest policy that reliably preserves the meaning required by the task. The benefit should be visible in corrected examples, while the added cost remains understandable. More text is useful only when it gives the answer something necessary to work with.
Primary sources
Microsoft Learn: chunk size and overlap considerationsMicrosoft Learn: layout-aware splittingReferences checked 11 September 2026.