Context Budgeting
The context budget includes instructions, history, retrieved chunks, and expected output. Sending every result is rarely optimal. Select direct evidence, preserve exceptions, remove duplicate passages, and order sources deliberately. Use token-aware budgeting and reserve space for the answer.
For policy content, avoid compressing away conditions and exceptions. If context must be summarized, preserve source IDs and distinguish summaries from exact passages. Test ordering and truncation behavior.
Implementation example
function selectChunks(chunks, maxTokens, estimate) {
const selected=[]; let used=0;
for (const c of chunks) {
const n=estimate(c.text);
if (used+n<=maxTokens) { selected.push(c); used+=n; }
}
return selected;
}💡 A greedy token budget is a baseline; production selection should account for relevance, diversity, and mandatory clauses.
Key Takeaways
- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
Knowledge Check
What should context selection prioritize?