SAG / ARCHITECTURE NOTE
RAG Chunking: How Small Should You Split Documents?
Explains what chunking is, why it is needed, how it works, and how to apply it in SAG architecture, with practical checklists based on research and official documentation.
Definition in one sentence
Chunking is the process of dividing long documents into searchable semantic units while preserving their positions and hierarchy in the original.
The key answer: Chunks that are too large mix in irrelevant context, while chunks that are too small lose conditions and exceptions. Splitting by character count alone also breaks tables, headings, and paragraph relationships.
Why is this technology needed?
Chunks that are too large mix in irrelevant context, while chunks that are too small lose conditions and exceptions. Splitting by character count alone also breaks tables, headings, and paragraph relationships.
How it works
First, split content based on heading hierarchy, paragraphs, tables, and lists; then apply length limits and overlap. Store the document, section, version, and original position as metadata for each chunk.
Accuracy is not the only consideration in design. Latency, cost, data boundaries, update frequency, and failure behavior must also be defined to make results reproducible in production. It is safer to leave values that automation cannot determine with confidence as unmeasured or requiring review, rather than changing them to zero or marking them as successful.
Connection to SAG technology
If chunks are linked to SAG evidence artifacts, reports should let users return to the original page and passage. The principles for storing sources, hashes, and versions provide the foundation for this.
Practical checklist
- Set boundaries based on document structure first
- Measure context loss and duplicate retrieval rates together
- Preserve links from chunks back to the original
- Distinguish failure, empty results, and permission errors from success
- Revalidate before and after changes under the same conditions
Research and official documentation
Reference documents support the principles and recommendations. They do not guarantee search visibility, AI mentions, rankings, or revenue. The actual impact of implementation must be verified through observations of service data under the same conditions.
Selection criteria and a concrete application example
If you split a specification table while retaining only its numeric rows, the units, product names, and test conditions are lost. Preserve the headings and table headers together, and link them to their original positions. There is no optimal chunk size that applies to every document; choose one based on document type and evaluation questions.
Scope of application at SAG
This article covers research principles in search AI and extensible design. Read it in connection with SAG's page collection, evidence recording, and report verification structures, but do not interpret it to mean that every search algorithm in the papers has been incorporated into the production pipeline. Verify whether a capability is applied using the search module, evaluation data, and execution records.
How to continue reading about this technology
Compare RAG, GraphRAG, and Self-RAG papers and their conditions for application.
SAG / KNOWLEDGE LINKS