Chunking
Markdown Chunking
Markdown chunking is a method of splitting markdown based on structure like headers, paragraphs and sections.
This is useful when you want to process large markdown documents in smaller, manageable pieces.
| Parameter | Type | Default | Description |
|---|---|---|---|
chunk_size | int | 5000 | The maximum size of each chunk. |
overlap | int | 0 | The number of characters to overlap between chunks. |