Skip to content

Commit 99c2f4a

Browse files
committed
k3 B200: carry spec-decode-under-PP into the container as a patch
Upstream refuses this outright -- vllm/v1/worker/gpu/spec_decode/dspark/utils.py on main still raises NotImplementedError when the PP group is larger than one, and SpeculativeConfig hands the draft the target's pipeline_parallel_size, which asks K3DSparkForCausalLM for a SupportsPP it does not declare. That is the error all seven of our earlier PP2 arms died on. The support lives in our vLLM fork as 2111011d33 (adopting vllm-project/vllm#50514), 540423f2c7 and 503820ebdd: the draft stops inheriting PP size and runs whole on the last stage, aux hidden states relay between stages as IntermediateTensors behind a Kimi-K3-only opt-in, the last stage loads the real embedding table instead of aliasing a PPMissingLayer, and the draft loader drops fastsafetensors under PP because its collective runs over group.WORLD. Applied with patch(1) after a --dry-run, and verified afterwards that the refusal is gone and the draft's PP size is pinned to 1. A mismatch fails the job rather than running a half-patched engine; the fork's base is 38a466e7b6 and the image is 5894fdf98, so drift will eventually break it and should be loud when it does. Any arm using this must ship with a GSM8K run. The adopting commit records that draft tokens previously failed to reach earlier stages and the first rank embedded PLACEHOLDER_TOKEN_ID(-1) -- silent corruption that ignore_eos perf arms cannot see.
1 parent 5805859 commit 99c2f4a

2 files changed

Lines changed: 721 additions & 0 deletions

File tree

0 commit comments

Comments
 (0)