feat(qwen3.8): add incubating DFLASH15 fast TP2 profile

[cristianflorett]

PR title

Add incubating Qwen3.8-27B DFLASH15 fast TP2 profile

Summary

Related issue: #1081

Type of change

• New compose variant
• New patch / sidecar
• Registry and documentation updates

Verification

• Full rig report: report.sh --full, all five stages PASS, exit 0

• Complete profile header with status/caveats/siblings

• BENCHMARKS row added

• CHANGELOG entry added

• Add vllm/qwen38-27b-dual-dflash15-fast for the complete Qwen3.8-27B-W4A16-AutoRound-fast target plus the external DFlash2 W4A16 drafter (SPEC_N=15).

• Add lookup-augmented drafting, split-KV/spec-decode support, hybrid KV/CUDAGraph sizing, and the WSL2 UVA fallback as boot-time, fail-closed patches.

• Register the model artifact, patch delivery, compose profile, and disk-count regression test.

• Keep the canonical envelope conservative: TP=2, BF16 KV, FlashAttention, GPU_MEMORY_UTILIZATION=0.85, MAX_NUM_SEQS=1, MAX_MODEL_LEN=244320, and custom all-reduce/P2P disabled.

• Keep MAX_NUM_SEQS=2 experimental; it is not promoted to the default.

• Preserve multimodal support: no --language-model-only; a real base64 PNG request returned HTTP 200 with a coherent image description.

Measured evidence

Rig: 2× RTX 3090, asymmetric PCIe x16/x4, 268.8/280 W, no NVLink, WSL2, custom all-reduce off.

• Canonical decode bench (3 warmups + 5 measured, repeated twice): ~93.69 narrative / ~163.82 code tok/s.
  • Representative measured decode runs: narrative 96.01, 81.09, 97.25, 84.47, 92.46 (mean 90.26, CV 7.9%); code 126.99, 165.62, 165.31, 169.88, 159.02 (mean 157.36, CV 11.1%). Mean TTFT: 72 ms / 74 ms.
• On our 2× RTX 3090 PCIe x16/x4 rig, this profile exceeded our earlier run of the Club's PCIe dual-dflash.yml (82 / 125 narrative/code tok/s) by approximately +14.3% / +31.1%. This is a same-rig result; the profiles still differ in context envelope, image/pin, weights/drafter and settings, and NVLink rows are excluded.
• Local comparison against the existing fast tiers (same local 2× RTX 3090 PCIe x16/x4 rig, using adjusted local recipes):
  • vs dual-superfast (70.17 / 158.37): +33.6% narrative / +3.4% code.
  • vs dual-ultrafast (80.94 / 179.52): +15.8% narrative / −8.7% code.
  • DFLASH15 reached a 244,320-token served envelope on this desktop-constrained .85 setup, with stress validation to 224,840 tokens. Its architectural ceiling remains 262,144 tokens; headless rigs with additional VRAM headroom may reach the full ceiling.
  • The local dual-ultrafast run reached approximately 170K under the same host policy.

These are same-rig directional comparisons, not a universal ranking: dual-ultrafast remains stronger for code throughput, while DFLASH15 offers the stronger validated long-context envelope on this rig.

• Prefill measured runs: 1162.9, 1340.9, 1355.2 tok/s @10k (mean 1286.33); 1132.8, 1080.5, 1079.3 tok/s @90k (mean 1097.53).
• Peak VRAM: 41,979 MiB total.
• verify-full: 9/9 PASS.
• verify-stress: 8/8 PASS; the earlier validation reached 224,319 tokens and the official full report reached 224,840 tokens (92% of 244,320).
• Continuous soak: PASS, 73 MiB growth, 0 errors, 0 silent-empty turns, 100% TPS retention.
• Official bash scripts/report.sh --full: exit 0, all five stages PASS (verify-full, verify-stress, soak, bench, agentic). Stress ladder filled to 224,840 tokens (92% of 244,320) with all needles recalled; soak reported 0 MiB growth and 100% TPS retention. The report's post-soak bench was 106.35/175.34 decode tok/s (narr/code), with 1336.73/1104.18 prefill tok/s @10K/@90k; the fresh-boot runs above remain the headline BENCHMARKS row.
• Agentic 12-turn ramp reached 35,512 accumulated prompt tokens with 0 failures; TTFT grew 1.23s → 7.50s (6.1× warm baseline), documented as the expected Qwen3-Next recurrent-prefill trade-off.
• Full quality, thinking force-on, standard recipe:
  • reasoning_effort=low: 131/150 pass@1, 138/150 pass@3.
  • reasoning_effort=medium: 126/150 pass@1, 133/150 pass@3.

Cross-links

• Issue: [compose] Qwen3.8-27B DFLASH15 fast TP2 on asymmetric PCIe #1081
• Related: DFlash2 drafter for Qwen3.8-27B (multi4 INT4) — 6-piece vllm#52816 port + memory findings #1064 (DFlash2 drafter), [bug] vllm/qwen38-27b-dual-fast - speculative decoding stops working after some time dragging TPS into the 20s #1052 (built-in MTP collapse)

Caveats

• Numbers are from PCIe x16/x4 WSL2 without NVLink; lane topology and host state can move prefill and decode. Re-run the canonical recipe on the target rig before comparing TPS.
• The model's architectural context ceiling is 262K. The PR's 244,320-token value is a host-constrained measured envelope for this desktop-driven WSL rig at .85; .90 OOMs during soak here. Headless rigs with additional VRAM headroom may reach the full 262,144 ceiling; that headless case is not measured in this PR.
• The profile remains incubating and requires maintainer review before promotion.
• SPEC_N=0 remains the mitigation path for speculative-decode exposure; the compose's tested default is DFLASH15.
• The v1 target intentionally retains MTP artifacts for exact reproduction; a headless v2 target is out of scope.

Validation commands

bash scripts/tests/test-compose-registry-disk.sh
URL=http://localhost:8113 MODEL=qwen3.8-27b bash scripts/verify-full.sh
URL=http://localhost:8113 MODEL=qwen3.8-27b bash scripts/verify-stress.sh
URL=http://localhost:8113 MODEL=qwen3.8-27b bash scripts/soak-test.sh --continuous
URL=http://localhost:8113 MODEL=qwen3.8-27b \
  CONTAINER=vllm-qwen38-27b-dual-dflash15-fast \
  bash scripts/report.sh --full

Full report artifact: /home/cristian/ab-quickwins/dspark-wsl/dflash15-report-full-20260821.md.

Commit: ee06b714 (feat/qwen3.8: add incubating DFLASH15 fast TP2 profile).

[cristianflorett]

Full rig report

Redacted full report.sh --full artifact: https://gist.github.com/cristianflorett/f36ad221f727e585723d4c10afbf57da

• Overall: PASS, report.sh exit 0
• verify-full: PASS
• verify-stress: PASS; ceiling ladder fillable to 224,840 tokens (92% of 244,320)
• soak-continuous: PASS; 0 MiB growth, 0 silent-empty turns, 100% TPS retention
• canonical report bench (post-soak diagnostic): 106.35 / 175.34 decode tok/s narrative/code; 1336.73 / 1104.18 prefill tok/s at 10K/90K
• agentic ramp: 12 turns to 35,512 accumulated prompt tokens, no failures; TTFT 1.23s → 7.50s (6.1× warm baseline), consistent with the Qwen3-Next recurrent-prefill trade-off

The fresh-boot two-run headline used in BENCHMARKS.md remains ~93.69 / ~163.82 decode tok/s; the report bench was run after the full validation/soak sequence and is labeled separately above.