[Bug]: set_splitting_ops_for_v1 early return skips the empty-splitting-ops guard — breakable cudagraph + cudagraph_mode=PIECEWISE + spec decode silently rejects every draft #53030

CompilationConfig.set_splitting_ops_for_v1 returns early when the compilation
mode is not VLLM_COMPILE, which skips the guard that would otherwise disable
piecewise cudagraphs for an empty splitting_ops. The resulting configuration —
piecewise cudagraphs with attention inside the graph — silently produces wrong
results for any batch whose per-request query length differs from the captured
one. Speculative decoding is the case where that happens in practice, and the state is reachable in stock vLLM via VLLM_USE_BREAKABLE_CUDAGRAPH (auto-enabled for several architectures).

The early return

def set_splitting_ops_for_v1(self, all2all_backend, data_parallel_size=1):
    if self.mode != CompilationMode.VLLM_COMPILE:
        if self.splitting_ops is None:
            self.splitting_ops = []
        return                     # <-- skips everything below

The guard that is skipped:

elif len(self.splitting_ops) == 0:
    if self.cudagraph_mode == CUDAGraphMode.PIECEWISE:
        logger.warning_once(
            "Piecewise compilation with empty splitting_ops does not contain "
            "piecewise cudagraph. Setting cudagraph_mode to NONE. ...")
        self.cudagraph_mode = CUDAGraphMode.NONE

So mode=NONE + cudagraph_mode=PIECEWISE leaves splitting_ops=[] and
cudagraph_mode=PIECEWISE, which the guard exists to prevent. The guard itself
is correct; it is simply not reached.

Why the resulting state is unsound

With splitting_ops=[] the attention op is inside the captured region. Piecewise
graph entries are keyed by BatchDescriptor (forward_context.py), whose fields
are:

num_tokens: int
num_reqs: int | None      # "Can be None for PIECEWISE ... can handle any number of requests"
uniform: bool             # "True if all the requests in the batch have the same number of tokens"
has_lora: bool
num_active_loras: int

No field records the per-request query length. So these two batches share a key:

_is_compatible states the intent explicitly:

# desc.uniform_token_count=None (PIECEWISE) can handle any uniform_token_count

That assumption is sound only while attention is outside the graph. With
splitting_ops=[] it is not: attention's behaviour depends on query_start_loc,
which is frozen at capture and replayed against a different query structure.

Result: wrong logits on the verify pass, no error raised.

Observed effect

8xMI325X (gfx942), Qwen3.8-27B (architecture Qwen3_5ForConditionalGeneration) + DFlash2 drafter, TP=8, num_speculative_tokens=7,
VLLM_USE_BREAKABLE_CUDAGRAPH=1 (see Scope below for why that flag matters):

The first two rows share the same compilation mode (NONE) — the only difference
is whether the forward is captured into a graph, which isolates the graph as the
cause. The third row is the corrected configuration; end-to-end it is ~3x a
non-speculative deployment of the same model on a coding workload (187-207 vs
66.8 tok/s).

A non-speculative deployment of the same model is unaffected, because it only ever
produces uniform 1-query decode batches — there is only one query structure, so
nothing collides. That is why this is easy to miss: it needs speculative decoding
to surface.

Scope — how this state is reached

We initially believed we reached the unguarded state through a local patch. That
was wrong: everything involved is stock vLLM.

1. VLLM_USE_BREAKABLE_CUDAGRAPH is an upstream env
   (envs.py: "Experimental: breakable cudagraph does not rely on torch.compile"),
   and config/vllm.py auto-enables it for an architecture allow-list
   (DeepseekV4, Inkling, KimiK3, KimiLinear, ...) unless the env is explicitly set.
2. With it enabled, compilation mode is forced to NONE while
   cudagraph_mode can remain piecewise-capable, and
   BreakableCUDAGraphWrapper captures the forward without torch.compile —
   its docstring says exactly that ("PW CUDA graph without torch.compile").
   set_splitting_ops_for_v1's early return then leaves splitting_ops=[]
   without the downgrade the guard would have applied.
3. Our deployment set the env to 1 for a model outside the allow-list
   (the Qwen3.8-27B model above); an allow-listed model with speculative decoding and cudagraph_mode
   explicitly set to PIECEWISE should reach the same state (the allow-listed
   defaults are FULL variants, which are keyed differently — see below); we have
   not tested one directly.

Why FULL variants are safe (and a workaround): in _init_candidates, the
separate-decode routine used by the FULL modes captures one graph per decode
query length and puts it in the key:

if separate_decode_routine and decode_mode:
    for decode_query_len in decode_query_lens:
        desc = BatchExecutionDescriptor(
            cg_mode=decode_mode,
            num_reqs=rounded_num_reqs,
            uniform_token_count=decode_query_len,   # <- query length in the key
        )
if mixed_mode:
    # for PIECEWISE graphs there is no limit on requests when replaying
    num_reqs = None                                  # <- PIECEWISE: no query length

and _is_compatible's first clause checks uniform_token_count. So under
FULL_AND_PIECEWISE a verify batch and a plain decode batch cannot share a key,
which matches what we measure: our DeepSeek-V4 production deployment
(allow-listed, breakable auto-enabled, FULL_AND_PIECEWISE, DSpark speculative
decoding) has a healthy ~3.2 mean acceptance length. Switching
cudagraph_mode from PIECEWISE to any FULL variant therefore avoids the
collision — though for this drafter the FULL path has its own unresolved
problem on our hardware (GPU memory faults, reported separately on the DFlash2
PR), so we run VLLM_USE_BREAKABLE_CUDAGRAPH=0 instead.

Suggested fixes (either would have caught this)

• Move the splitting_ops == [] and cudagraph_mode == PIECEWISE -> NONE guard so
  the early return cannot skip it.
• Or add the per-request query length to BatchDescriptor when speculative
  decoding is enabled, so verify batches cannot share a key with plain decode.

The second is the more general fix: it removes the dependence on "attention is
outside the graph" being maintained by every future code path.