> ## Documentation Index
> Fetch the complete documentation index at: https://docs.seekr.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Choose a chunking method

> Segment ingested content with the markdown, semantic, or sliding window chunking methods, and attach per-chunk metadata.

The `chunking_method` parameter on `create_ingestion_job` controls how SeekrFlow segments your content into chunks before embedding. Choose the method that best fits your documents.

## Markdown chunking (default)

**Intelligent structure-aware chunking** that automatically detects logical content breaks:

* Respects document structure and heading hierarchies
* Keeps related content together (headings with their content)
* Preserves tables with their headers
* Groups small sections to optimize chunk sizes

## Semantic chunking

Call with `chunking_method="semantic"` to enable meaning-aware segmentation. This method detects sentence boundaries, embeds paragraphs, and groups them by meaning using hierarchical clustering, rather than relying on document position:

* Searches for topic shifts instead of raw heading boundaries to keep tightly related sentences together
* Automatically merges short paragraphs or bullets when they express the same idea
* Honors document structure when it provides strong signals but can span across headings if the semantics match
* Applies `token_count` and `overlap_tokens` as safety caps, splitting only when a semantic chunk would exceed those limits

**When to use it:**

* Long narrative content (wikis, blogs, requirements) where sections do not follow strict Markdown hierarchy
* Mixed-format documents where context spans multiple small headings or callouts

No additional markup is required. Tune `token_count` and `overlap_tokens` to control chunk granularity.

## Sliding window chunking

Call with `chunking_method="sliding"` to split content into fixed-size overlapping windows. This method has no structural awareness, which makes it fast and predictable. Use it for plain-text or homogeneous content that lacks clear document structure.

To force chunk boundaries at specific points, insert `---DOCUMENT_BREAK---` markers in your Markdown:

<CodeGroup>
  ```text Markdown theme={null}
  # Section 1

  This content will be in one chunk...

  ---DOCUMENT_BREAK---

  # Section 2

  This content will be in a separate chunk...

  ---DOCUMENT_BREAK---

  # Section 3

  This starts another chunk...
  ```
</CodeGroup>

Set `token_count` (for example, `1000`) and `overlap_tokens` (for example, `100`) in the same `create_ingestion_job` call to control chunk size and overlap.

Document break markers apply only to sliding window chunking. With `markdown` or `semantic` chunking, a marker does not create a boundary, and the literal marker text stays in the chunk. Remove the markers from your Markdown before you ingest it with either of those methods.

<Note>
  If the content between two document break markers exceeds `token_count`, the sliding window splits it into multiple chunks.
</Note>

### Add per-chunk metadata

With sliding window chunking, you can attach different metadata to different chunks by embedding a metadata block inside a section. Place the block between `---CHUNK_META_START---` and `---CHUNK_META_END---`, with a single line of JSON in between. The block is scoped to the section it appears in and is removed from the text before indexing and never becomes part of the chunk content.

<CodeGroup>
  ```text Markdown theme={null}
  # Introduction

  ---CHUNK_META_START---
  {"author": "Jane Smith", "source": "chapter_1", "classification": "public", "year": 2024}
  ---CHUNK_META_END---

  This is the first chunk of the document.

  ---DOCUMENT_BREAK---

  # Conclusion

  This chunk has no metadata block and falls back to the job-level metadata from the ingestion request.
  ```
</CodeGroup>

Per-chunk metadata follows the same [metadata rules](/flow/sdk/data-engine/manage-chunk-metadata#metadata-rules) as job-level metadata, but a block that breaks them is handled differently. An ingestion request with invalid metadata is rejected outright, while inside a block an invalid field is dropped with a warning and the rest of the chunk still ingests.

Per-chunk metadata behaves as follows:

* It is supported only with sliding window chunking.
* A section's block replaces the job-level metadata for that chunk. The two are not merged.
* A block applies to chunks only. It does not change the `metadata` on a file record, which keeps the job-level values.
* A section with no block inherits the job-level metadata from the ingestion request, or no metadata if the request supplied none.
* When a section is longer than `token_count` and splits into several chunks, every chunk from that section carries the section's metadata.
* When a section contains more than one block, the last block applies. Use a single block per section.

<Warning>
  Ingestion does not reject metadata blocks that appear with `markdown` or `semantic` chunking. The block is not applied, and its marker lines and JSON remain in the chunk text, where they become part of the embedded content. Use these markers only with `chunking_method="sliding"`.
</Warning>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.