Repository navigation
Batch context capacity cleanup discards live continuation contexts unnecessarily #4181
Description
Bug
BatchContextStore.store() can discard a live, unrelated batch's continuation context even when no live-capacity eviction is needed. This loses the messages/tools/model needed to process later CCR retrieval results.
Reproduced on upstream main 0126fba63f5af3c8022085d0d033dce56fe5db65, Linux x64, Python 3.13.14, through the real public BatchContextStore.store/get API (no mocked store).
Two cases in the same capacity policy:
- Fill a store with
max_contexts=3witha,b,c, then replacec. The code runs_cleanup_oldest()before overwritingc, soadisappears although the update does not increase cardinality. - Fill a two-context store with an older-created live batch and a newer-created expired batch, then insert a new batch.
_cleanup_oldest()sorts by creation time without reclaiming expired entries first, so it evicts the live batch and retains the expired entry until a laterget()removes it.
Actual baseline output:
{"replaceAtCapacity":{"a":false,"b":true,"c":true},"insertWithExpiredCapacity":{"live":false,"expired":false,"new":true}}Expected: replacement retains all unrelated live contexts and refreshes the replacement's TTL; insertion reclaims expired capacity before applying the existing oldest-live eviction policy. The configured capacity remains bounded.
The two deterministic public-API regression tests fail on main with get("first") is None and get("live") is None, respectively. A focused fix is being verified; no provider/network credentials are required to reproduce this cache defect.
Activity
oxura commentedon Oct 13, 2026
Fixed and published on oxura/headroom@74ed96bd (
fix/batch-context-capacity-20261013). Replacement now refreshes the existing batch without evicting another; insertion reclaims expired entries before applying the existing oldest-live policy. No cache-cap increase, new dependency, retry, or provider behavior change.Real behavior proof: Linux x64, Python 3.13.14, upstream main 0126fba. Ran a standalone
asyncio.runscript against the actualBatchContextStore.store/getAPI. For max_contexts=3, store a,b,c then replace c: baseline loses a; patched API retains a,b,c and returns the replacement request's updated model. For max_contexts=2 with an expired newer-created entry and a live older-created entry, insert a new batch: baseline loses the live context; patched API retains live+new and removes expired. No mocked store and no provider needed.Two deterministic regressions fail before and pass after. The four-file batch consumer suite passes 119 tests, 14 skipped with the actual built Rust extension loaded; skipped live-provider cases are not claimed. Normal commit hooks and the complete pre-push gate passed, including Rust workspace checks/build and 181 critical Python tests. No live cloud-provider, Windows or Mac validation claimed.
I currently have 10 open PRs, so this is published as a verified fork branch rather than violating the repository's author cap. A PR can be opened once an existing slot is freed.