Incubate DeepSeek V4 WNA16 runtime on SM86

[Whamp]

Status

Experimental incubation branch. This is not a request to merge into vLLM main.

The branch centralizes the vLLM changes used by the technically working DeepSeek-V4-Flash-0731 WNA16 service on four RTX 3090 GPUs. It is based on the pinned SM8x DeepSeek V4 work in haosdent/vllm@12810046, not current vLLM main.

The runtime path loads and serves the model correctly. A source-confirmed DeepSeek V4 tool-generation boundary bug explained the prior zero-patch DeepSWE degeneration: after one complete DSML tool block, generation could continue into additional assistant/tool blocks in the same response. The tested artifact remains withdrawn from general coding-agent use because one repaired SuperJSON run passed only 65/80 feature tests and is not broad quality evidence. This PR remains open as runtime incubation, not as a general artifact endorsement.

Related work

This is related to, but separate from, noonghunna/club-3090#940:

• that PR incubates a llama.cpp fast-prefill path for an Antirez GGUF;
• this PR incubates a vLLM path for WNA16 safetensors, Humming MoE, and the SM86 DeepSeek V4 attention runtime.

The closest upstream loader work is vllm-project/vllm#48918. This branch does not replace or duplicate that PR as an upstream submission. It carries the additional DeepSeek V4 and SM86 integration needed by the tested runtime.

What this branch adds

1. Symmetric W2A16 compressed-tensors MoE loading through Humming.
2. Separate WNA16 schemas for fused gate/up (w13) and down (w2) projections.
3. A CUDA numerical oracle for W2, group-128, BF16 indexed MoE.
4. Hybrid DeepSeek V4 loading that preserves native block-FP8 non-expert linears.
5. Composite quantization dispatch that delegates only routed experts to compressed-tensors WNA16/Humming.
6. Production forwarding of the owning layer into Humming kernel setup.
7. An SM86 sparse-decode fallback when split-K exceeds available shared memory.
8. Runtime-bounded RoPE cache materialization while preserving the model's YaRN frequency span.
9. Preservation of model SwiGLU alpha, beta, and clamp semantics through the Humming quantization bridge, plus corrected N/K group-shape metadata.
10. A DeepSeek V4 ChatCompletion boundary that stops generation after one complete outer DSML tool-call block while preserving multiple invokes inside that block and caller stop semantics.

The exact branch tree is d315879bf0ead5088b1dded20045a4ddc25b3e4e at commit 9a2ffbb4534400064e645cb4fef8ab2f2a987f11.

Artifact

The tested model artifact is public at:

• hampsonw/DeepSeek-V4-Flash-0731-WNA16
• tested revision: 75d9286c37f3037f3ab390cfbc10747466eac714

It uses imatrix-weighted W2A16 routed experts, preserves the checkpoint's native FP8 representation for non-routed weights, and omits MTP. The artifact is now marked withdrawn because it failed coding-agent quality testing. It remains public only for reproduction and research.

Measured server60 result

Hardware: 4× RTX 3090, PCIe only, tensor parallelism 4, existing 230 W/card limits unchanged.

Selected profile:

• context: 215,000 tokens
• max_num_seqs: 4
• zero CPU weight offload
• decode: 60.79 tokens/s mean
• cache-busted prefill: 968.975 tokens/s
• exact retrieval from a 204,900-token prompt
• concurrency 2/4 aggregate decode: 65.47/89.94 tokens/s
• zero serving-process swap after a clean restart
• no post-warm VRAM growth in the acceptance run

The profile is usable but has a narrow measured reserve of roughly 141–142 MiB per GPU. Concurrency 4 was validated with short requests, not four simultaneous 215K requests.

Validation completed

• The branch reconstructs the exact previously tested runtime tree plus the source-confirmed Humming activation-semantics fix.
• Focused compressed-tensors, Humming, DeepSeek FP8, sparse-decode, and RoPE tests pass.
• The Humming quant-config regressions fail before and pass after the fix; 16 focused WNA16 tests and selected repository hooks pass.
• All seven threshold-crossing WNA16 CUDA oracle variants passed on RTX 3090/SM86 after the fix. The run covered W2/W4 with projection-specific group sizes and generated 56 inspected cubins.
• The W2/group-128/BF16 indexed-MoE numerical oracle passed on A100 and RTX 3090/SM86.
• Generated Humming cubins were inspected for the target architecture.
• The model loaded all 45 shards and dispatched the intended Humming WNA16 path.
• Long-context retrieval, deterministic generation, tool/reasoning parser canaries, prefill, decode, and short-request concurrency passed on server60.

Quality evaluation

The earlier benchlocal quick run was only smoke evidence. The first DeepSWE runs entered a severe repetitive tool-use loop and produced no patch, both with concurrent serving and with one worker. A separate 120.36 GB projection-sensitive candidate failed the same way. The SwiGLU/GroupShape repair was numerically correct but did not stop the loop.

Controlled raw-token replays then isolated a separate integration defect. DeepSeek V4 emitted a valid outer </｜DSML｜tool_calls> close and continued generating additional assistant/tool blocks because the adapter did not register that marker as a generation stop. The exact stage-4 replay produced 11 calls at a 512-token cap under both breakable CUDA graphs and --enforce-eager; parser-free raw completions reproduced the same sequence. Prefix-cache reuse and parallel_tool_calls were not causal. Adding the outer marker as an active-tool request stop reduced that replay to one valid call in 48 tokens while preserving a valid two-invoke block.

The exact one-worker SuperJSON cell was then repeated on RTX 3090 with the repaired adapter, Pi 0.84.1, maximum reasoning, temperature 1.0, top-p 0.95, and the same withdrawn artifact. It completed normally in 2,704 seconds with no watchdog event, timeout, OOM, or verifier failure. It produced a 19,226-byte patch over 149 bounded tool turns, passed all 116 existing tests, and passed 65/80 feature tests (partial reward 0.923469). The 15 misses are ordinary incomplete frames-mode implementation, not runtime or tool-loop failures.

This causally repairs the prior zero-patch degeneration for the tested task. It does not establish broad coding-agent quality: 65/80 feature tests trails the stronger reference results, and the 149-turn trajectory is expensive. The artifact therefore remains withdrawn pending broader quality evaluation or replacement-quant work.

Known limits

• Experimental fork-on-fork lineage; not rebased onto current vLLM main.
• Exact validation is limited to the pinned artifact and four RTX 3090 GPUs.
• The tested runtime uses a custom image and Humming JIT kernels.
• Custom all-reduce is disabled for the PCIe-only topology.
• The 215K profile has little free VRAM despite passing repeated runtime checks.
• The tested WNA16 artifact failed coding-agent quality testing and is withdrawn.
• The DSML generation-boundary repair fixes the tested zero-patch degeneration, but the repaired run passed only 65/80 feature tests and used 149 turns; broad coding-agent quality remains unproven.
• The branch validates a runtime path, not the quality of future artifacts.

AI assistance

AI assistance was used for implementation, investigation, testing, and drafting. I reviewed the experimental direction and retain responsibility for the branch.

[Whamp]

Quality update: the controlled one-worker, no-concurrency DeepSWE rerun showed the same severe degeneration as the concurrent run. Concurrency is ruled out. The current WNA16 artifact is withdrawn, while this draft remains open because the runtime path itself loaded and served correctly and may support replacement quants.