Work out how many chunks a corpus becomes, how much the overlap adds to the tokens you actually embed, whether the retrieved passages fit the context you reserved, and what the vectors and index cost in memory.
Worked setups you can load into the form above. Each one is a decision the generator makes differently, and the reason it makes it.
A 256 token overlap on a 512 token chunk halves the stride, so you embed roughly twice the corpus. The cost is paid on every chunk and again in full on every re-index.
The embedding model accepts 8,000 tokens happily. Five 2,000 token chunks then need 10,000 tokens of context, and the truncation happens at request time rather than at ingestion.
Cohere's embed-english-v3.0 takes 512 tokens. A larger chunk is rejected by the API rather than truncated, so the ingestion run fails partway and leaves the index half built.
A passage spanning a boundary is split, and neither half holds the whole answer. This presents as the model being unable to answer a question whose answer is definitely in the corpus.
These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
The window advances by size minus overlap, so the stride divides the corpus. 512 with 128 of overlap embeds a third more tokens, not a quarter more, and it is paid again on every full re-index.
Instead:Compute from the stride. Split on structure so boundaries fall between paragraphs, which needs far less overlap than a fixed window.
A model with an 8,000 token window will embed 8,000 token chunks happily. Retrieving five of them then needs 40,000 tokens of context, and the truncation happens at request time.
Instead:Size against the context you reserved for retrieved text, divided by how many chunks you retrieve.
A passage spanning a boundary is split, and neither half contains the whole answer, so neither retrieves well. It presents as the model being unable to answer a question whose answer is definitely in the corpus.
Instead:Ten to twenty percent of the chunk size, or split on structure.
An HNSW graph adds a neighbour list per node, and the working set has to be resident in memory for queries to be fast rather than merely correct.
Instead:Budget both. float16 halves the vectors and product quantisation cuts further, both at a recall cost worth measuring.
Characters divided by four is off by up to 69% on real content, and the whole plan is built on that one number.
Instead:Measure it with a real tokenizer. The token counter on this site does it in your browser.
Chunking arithmetic is short, and skipping it is why an ingestion run costs a third more than expected and why five retrieved passages will not fit in the context reserved for them.
The window advances by size minus overlap, so that stride is what divides the corpus. At 512 tokens with 128 of overlap the stride is 384, which means you embed a third more tokens than the corpus contains, not a quarter more. The extra is paid on every chunk and paid again in full on every re-index, whether that re-index is a chunking change, a model change or a corpus refresh.
document 2000 tokens, chunk 512, overlap 64
stride = 512 - 64 = 448
chunks = 1 + ceil((2000 - 512) / 448) = 5
embedded = 5 x 512 = 2560 tokens, for 2000 tokens of textAn embedding model with an 8,000 token input window will happily embed 8,000 token chunks. Then you retrieve five of them and need 40,000 tokens of context for the passages alone. The binding constraint is what you reserved in the prompt for retrieved text, and the failure shows up at request time as truncation rather than at ingestion time as an error.
A single vector represents the whole chunk. Make it long and it averages several topics together, matching everything weakly and spending most of the retrieved passage on text that is not relevant. Make it short and it embeds a sentence with no surrounding meaning, so it matches on wording rather than subject. 256 to 512 tokens suits prose; small chunks suit dense reference material where each entry is self-contained.
Raw float32 vectors are the part people budget for. An HNSW index adds a neighbour list per node, roughly M times two links at the base layer with M commonly 16, and the working set has to be resident for queries to be fast rather than merely correct. float16 halves the vectors, product quantisation cuts much further, and both cost recall.
A passage that spans a boundary ends up split, and neither half contains the whole answer, so neither retrieves well on its own. This is the bug that presents as a model unable to answer a question whose answer is definitely in the corpus. Splitting on structure, so boundaries fall between paragraphs, achieves the same thing with far less overlap than a fixed window needs.
It does arithmetic on the numbers you give it. The token count per document is the input that matters most and is the one most often estimated: characters divided by four is wrong by up to 69% on real content, so measure it with the token counter on this site rather than guessing. No price list is bundled, deliberately, so the cost figure appears only when you supply your own rate. Nothing here predicts retrieval quality, which depends on your embedding model, your queries and your corpus, and is only answerable by evaluating against real questions.