fix(qwen3.8-27b/dual-fast): unpin FlashInfer decode buffer to bound the MTP c≥4 Xid 31 crash

[A1RM4X]

Slug: vllm/qwen38-27b-dual-fast (W4A8 + MTP n=4 + fp8 KV + prefix caching, hybrid GDN)
Upstream: vllm-project/vllm#40756 (thread comments 2026-08-17 — brasrox retraction + sempai SM86 corroboration)

Problem

This slug crashes the engine in ~15 s at MAX_NUM_SEQS >= 4 with the drafter on:

NVRM: Xid 31, MMU Fault: ENGINE GRAPHICS, faulted @ 0x18_a0000000
      FAULT_PDE ACCESS_TYPE_VIRT_READ   (unmapped page, both TP ranks)
torch.AcceleratorError: CUDA error: an illegal memory access was encountered

Root cause (vllm#40756, 2026-08-17 thread): FlashInfer plan() reuses one pinned host buffer per wrapper and copies it out asynchronously with nothing guarding it. The MTP drafter re-plans the decode wrapper K−1×/step, so a stale plan feeds the split-KV merge (PersistentVariableLengthMergeStatesKernel) a garbage row (observed: row 33,621 of an 85-row buffer) → Xid 31. It needs ≥4 genuinely concurrent in-flight MTP requests to lose the race; c≤3 and the SPEC_N=0 path never crash.

The prior documented mitigations were SPEC_N=0 (kill the drafter) or MAX_NUM_SEQS=1 (no concurrency). This patch fixes the crash at full async speed so the drafter and concurrency can coexist.

The fix

New patch models/qwen3.8-27b/vllm/patches/vllm-flashinfer-decode-pin/ — an idempotent, marker-gated Python patcher + install.sh, run from the compose entrypoint before vllm serve. It flips pin_memory=True, → pin_memory=False, on the _pin_memory_int_workspace_buffer allocations in flashinfer/decode.py (the decode wrapper MTP re-plans). pin_memory=False forces a synchronous host→device plan copy each step, closing the stale-read window.

Safety (matches this repo's "never serve unpatched" convention):

• Auto-detects flashinfer via import flashinfer → no-ops if not installed (a non-FlashInfer backend config boots clean).
• Guard: count(pin_memory=True,) must equal the number of _pin_memory_int_workspace_buffer* = torch.empty( allocs in each touched file, else hard-fail (exit 2) — the compose refuses a half-patched serve.
• Idempotent (marker-gated), py_compiles every file it writes.
• FI_PINQ_LIB_ALL=1 also unpins prefill/sparse/pod (the validated full mirror); default is decode-only.

Validation (v0.27.1 / flashinfer 0.6.16.post3 / 2× RTX 3090 SM86, TP=2)

Config	c=4, 120 s	result
stock (pinned)	—	crash ~15 s
decode.py unpin only (this patch)	OK=128, ERR=0, 0 new Xid	survive
full mirror (+ prefill/sparse/pod)	OK=126 + OK=345 (300 s soak)	survive, 425 s clean

• decode.py is necessary and sufficient — A/B: unpinning prefill/sparse/pod without decode.py still crashed; decode.py alone is the fix.
• End-to-end re-verified inside the pinned v0.27.1 image: install.sh → decode.py: 4 pin_memory True->False (py_compile OK), idempotent re-run already patched, decode 0/4 and the other three 4/0.
• Perf-neutral at c=1 (the clean A/B, since stock crashes at c=4): decode narrative 86.31→84.47 (−2.1%), code 102.32→101.02 (−1.3%), prefill ~0% — all within CV. At c=4 it unlocks 72.4 narrative / 85.0 code tok/s per-request (~3.4× c=1 aggregate rate).

What this does and does NOT change — deliberately

• Bounds the crash, not VRAM. The slug still ships MAX_NUM_SEQS=1 because of the separate W4A8 16K-prompt OOM documented in the compose header (peak 23,872 MiB at N=2, 1.75 GB over the 0.90 budget). That is a different constraint; this patch does not touch it. At the shipped N=1 default the patch is inert. It becomes load-bearing the moment MAX_NUM_SEQS is raised — then the drafter and concurrency coexist without the Xid.
• Status unchanged: the slug stays 🧪 Experimental. A patch that fixes a crash does not by itself promote a tier that still has no 8-pack / NIAH / soak.

Gate status (honesty section — per CONTRIBUTING)

Gate	Status
test-patch-attribution.sh	✅ PASS (71 patch entries)
test-compose-status-drift.sh	✅ PASS
compose YAML parses (docker compose config)	✅
patcher end-to-end in pinned v0.27.1 image	✅ (above)
c=4 crash repro + 425 s soak	✅ (above)
perf (c=1 A/B + c=4)	✅ (above)
bash scripts/report.sh --full (rig + verify-full + verify-stress + soak-continuous + bench)	⛔ NOT run — see below
8-pack quality gate (BenchLocal)	⛔ NOT run — throughput only

Why the full battery isn't attached: this PR changes one load-bearing patch on one slug; the compose's serving config (weights, sampler, KV, TP) is byte-identical to what's already validated on this slug. The patch is verified to (a) apply cleanly in the pinned image, (b) not change the c=1 path measurably, and (c) eliminate the c=4 crash. I have not run the repo's report.sh --full / 8-pack here, so I'm flagging it as the remaining validation rather than claiming it — the maintainer or a 2×3090 community rig should run report.sh --full as the merge gate. Known open question the patch does NOT resolve: whether the W4A8 16K-prompt OOM also clears at higher concurrency (separate from the Xid).

Attribution

Root-cause analysis and the c=4 reproduction/validation are mine (A1RM4X, 2× RTX 3090 rig). The bug is upstream's — the fix is a one-file change in flashinfer (unpin the decode wrapper's plan buffer, or guard the async copy-out); I've packaged it as a vendored patch in the club-3090 shape per CONTRIBUTING. Credits where due: the 2026-08-17 vllm#40756 thread comments (brasrox's retraction + sempai's SM86 corroboration) are what identified the decode wrapper as the buffer.

Files

models/qwen3.8-27b/CHANGELOG.md                              (new)
models/qwen3.8-27b/vllm/compose/dual/autoround-int4/mtp.yml  (mount + entrypoint invoke)
models/qwen3.8-27b/vllm/patches/vllm-flashinfer-decode-pin/  (new: install.sh, patcher, README)
scripts/lib/profiles/patches.yml                              (register the patch)

Drop-when

When flashinfer lands a fix for the pinned-buffer plan reuse (a pin_memory=False default on the decode wrapper, or a sync guard on the async copy-out) and the pin moves past it: remove the compose mount + entrypoint call + this patch dir, and retire the patches.yml row.