Chunking Strategies
| Strategy | Best For | Description |
|---|---|---|
| Semantic | General text | Groups semantically similar sentences |
| Fixed Size | Simple docs | Splits by character/token count |
| Recursive | Structured docs | Hierarchical splitting |
| Document | Natural sections | Preserves document structure |
| Code | Source code | Respects code syntax |
| Markdown | MD files | Splits by headers |
| CSV Row | Tabular data | One chunk per row |
| Agentic | Complex docs | AI-driven chunking |
Examples by Strategy
Semantic Chunking
Groups semantically related sentences together.cookbook/07_knowledge/chunking/semantic_chunking.py
Fixed Size Chunking
Simple chunking by character or token count.cookbook/07_knowledge/chunking/fixed_size_chunking.py
Recursive Chunking
Hierarchical splitting that preserves structure.cookbook/07_knowledge/chunking/recursive_chunking.py
Document Chunking
Preserves natural document boundaries.cookbook/07_knowledge/chunking/document_chunking.py
Code Chunking
Respects code structure and syntax.cookbook/07_knowledge/chunking/code_chunking.py
Markdown Chunking
Splits markdown by headers.cookbook/07_knowledge/chunking/markdown_chunking.py
CSV Row Chunking
One chunk per row.cookbook/07_knowledge/chunking/csv_row_chunking.py
Agentic Chunking
AI-driven chunking for complex documents.cookbook/07_knowledge/chunking/agentic_chunking.py
Custom Chunking Strategy
Build your own chunking logic.cookbook/07_knowledge/chunking/custom_strategy_example.py