Checklist
Describe the bug
When a previously chunked request is resumed in Scheduler.get_new_batch_prefill(), PrefillAdder.add_chunked_req() deducts its tokens from rem_chunk_tokens. If the resumed chunk consumes the remaining chunk budget, the scheduler still enters the waiting-queue loop without checking the updated budget state.
Each waiting request can then perform prefix-cache matching, and a host-cache hit can call init_load_back(). The request is rejected only later, when add_one_req() computes a non-positive truncation length.
Expected behavior: once the resumed chunk exhausts the per-iteration chunk budget, stop scanning the waiting queue for that scheduling pass.
Actual behavior: waiting requests perform cache matching and may start host-cache load-back even though none can be admitted under the exhausted chunk budget.
This is separate from #35537, which tracks a sticky batch_is_full state across scheduling iterations. This issue occurs within one scheduling pass immediately after resuming chunked_req.
Reproduction
The issue can be reproduced deterministically at the scheduler level:
- Set
Scheduler.chunked_req to a request whose next chunk consumes all remaining chunked_prefill_size tokens.
- Put another request in
Scheduler.waiting_queue.
- Run
Scheduler.get_new_batch_prefill().
add_chunked_req() reduces rem_chunk_tokens to zero, but the waiting request still reaches init_next_round_input() and cache matching before late rejection in add_one_req().
The existing PrefillAdder.budget_state() already returns AddReqResult.OTHER when rem_chunk_tokens <= 0; the scheduler does not consult it after resuming the chunked request.
Environment
- SGLang version: current
main
- Reproduction scope: scheduler control flow in
get_new_batch_prefill() and PrefillAdder
- Model/device requirements: none; the issue is reproducible with mocked scheduler requests and cache state
Checklist
Describe the bug
When a previously chunked request is resumed in
Scheduler.get_new_batch_prefill(),PrefillAdder.add_chunked_req()deducts its tokens fromrem_chunk_tokens. If the resumed chunk consumes the remaining chunk budget, the scheduler still enters the waiting-queue loop without checking the updated budget state.Each waiting request can then perform prefix-cache matching, and a host-cache hit can call
init_load_back(). The request is rejected only later, whenadd_one_req()computes a non-positive truncation length.Expected behavior: once the resumed chunk exhausts the per-iteration chunk budget, stop scanning the waiting queue for that scheduling pass.
Actual behavior: waiting requests perform cache matching and may start host-cache load-back even though none can be admitted under the exhausted chunk budget.
This is separate from #35537, which tracks a sticky
batch_is_fullstate across scheduling iterations. This issue occurs within one scheduling pass immediately after resumingchunked_req.Reproduction
The issue can be reproduced deterministically at the scheduler level:
Scheduler.chunked_reqto a request whose next chunk consumes all remainingchunked_prefill_sizetokens.Scheduler.waiting_queue.Scheduler.get_new_batch_prefill().add_chunked_req()reducesrem_chunk_tokensto zero, but the waiting request still reachesinit_next_round_input()and cache matching before late rejection inadd_one_req().The existing
PrefillAdder.budget_state()already returnsAddReqResult.OTHERwhenrem_chunk_tokens <= 0; the scheduler does not consult it after resuming the chunked request.Environment
mainget_new_batch_prefill()andPrefillAdder