Markdown Chunking
MarkdownChunking splits markdown on structure like headings, paragraphs, and sections.
Markdown chunking is a method of splitting markdown based on structure like headers, paragraphs and sections. This is useful when you want to process large markdown documents in smaller, manageable pieces.
| Parameter | Type | Default | Description |
|---|---|---|---|
chunk_size | int | 5000 | Approximate character target. Long atomic content and overlap can make final chunks larger. |
overlap | int | 0 | The number of characters to overlap between chunks. |
split_on_headings | Union[bool, int] | False | Heading-based splitting. False uses size-based chunking. True splits on all headings (H1-H6). An int from 1 to 6 splits on headings at or above that level, e.g. 2 splits on H1 and H2. |