CSV row chunking is a method of splitting CSV files into smaller chunks based on the number of rows, rather than character count. This approach is particularly useful for structured data where you want to process CSV files in manageable row-based chunks while preserving the integrity of individual records.
ParameterTypeDefaultDescription
rows_per_chunkint100The number of rows to include in each chunk.
skip_headerboolFalseWhether to skip the header row when chunking.
clean_rowsboolTrueWhether to clean and normalize row data.
include_header_in_chunksboolFalseWhether to include the header row in each chunk.
max_chunk_sizeint5000Maximum character size for each chunk (fallback limit).