[CI] main2main vllm 0814 cdc4824 - #14131
Open
zhangxinyuehfad wants to merge 13 commits into
Open
Conversation
Contributor
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request updates the tracked commit hash for the vllm main branch within the CI configuration. This ensures that the continuous integration environment remains aligned with the latest verified state of the repository. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Ignored Files
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request updates the verified vLLM commit hash in .github/vllm-main-verified.commit to b1b752042f622c692d5019c3ea122f2f7ee9d6ac. The reviewer provided a suggestion for the PR title and summary to align with the repository's style guide.
Contributor
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
zhangxinyuehfad
changed the title
zhangxinyuehfad
force-pushed
the
zxy_main2main_0814
branch
2 times, most recently
from
August 18, 2026 06:13
0c564e1 to
dcdd284
Compare
zhangxinyuehfad
changed the title
zhangxinyuehfad
force-pushed
the
zxy_main2main_0814
branch
from
August 18, 2026 08:11
f3876d7 to
0809f17
Compare
zhangxinyuehfad
requested review from
realliujiaxu and
zzzzwwjj
as code owners
August 19, 2026 09:19
zhangxinyuehfad
force-pushed
the
zxy_main2main_0814
branch
3 times, most recently
from
August 19, 2026 11:05
8c25f77 to
22d3a6b
Compare
zhangxinyuehfad
force-pushed
the
zxy_main2main_0814
branch
from
August 19, 2026 11:14
22d3a6b to
8c25f77
Compare
zhangxinyuehfad
force-pushed
the
zxy_main2main_0814
branch
from
August 19, 2026 11:17
8c25f77 to
1106b82
Compare
Contributor
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
zhangxinyuehfad
force-pushed
the
zxy_main2main_0814
branch
from
August 20, 2026 03:51
1106b82 to
ec33a35
Compare
zhangxinyuehfad
force-pushed
the
zxy_main2main_0814
branch
from
August 21, 2026 03:34
9ce3600 to
ccd5ffe
Compare
zhangxinyuehfad
added
ready-all
Collaborator
Author
|
/e2e tests/e2e/pull_request/four_card/_310p/test_moe_model_310p.py::test_qwen3_moe_tp2_w8a8 |
zhangxinyuehfad
force-pushed
the
zxy_main2main_0814
branch
from
August 21, 2026 07:39
9b93705 to
a0825b1
Compare
Contributor
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
zhangxinyuehfad
force-pushed
the
zxy_main2main_0814
branch
from
August 21, 2026 12:21
a0825b1 to
acb4ec2
Compare
Signed-off-by: hfadzxy <starmoon_zhang@163.com>
vLLM #50685 removed _all_gather_hidden_and_residual from
�llm.model_executor.models.qwen3_next and inlined the
sequence-parallel all-gather/reduce-scatter logic into the decoder
layer, breaking the unconditional import at module load.
Gate the symbol import, the _ascend_all_gather_hidden_and_residual
monkey-patch, and the MTP gather path with �llm_version_is('0.27.1'):
- v0.27.1: keep the original monkey-patch behavior unchanged
- main: skip the removed helper and use the new upstream inline flow
( ensor_model_parallel_all_gather after norm)
Signed-off-by: hfadzxy <starmoon_zhang@163.com>
….1')
vLLM main (cdc4824a21) added max_query_len: int | None = None to the
InputBatch dataclass (vllm#47808), making the child
AscendInputBatch's non-default seq_lens_np field follow a default
argument, which raises TypeError at class definition time.
Use vllm_version_is('0.27.1') to conditionally give seq_lens_np a
default value on main only, satisfying Python dataclass field ordering
while keeping v0.27.1 behavior unchanged.
Signed-off-by: hfadzxy <starmoon_zhang@163.com>
… compat
vLLM main (cdc4824a21) added
um_prefill_lookahead: int = 0 to
get_kv_cache_coordinator (vllm#50062), breaking the patched function
in vllm-ascend which rejected the unexpected keyword argument.
Refactor into a shared _build_kv_cache_coordinator impl with
um_prefill_lookahead: int | None = None, and two version-specific
wrappers gated by �llm_version_is('0.27.1'):
- v0.27.1: signature without num_prefill_lookahead (not forwarded)
- main: signature with num_prefill_lookahead (forwarded to upstream)
Signed-off-by: hfadzxy <starmoon_zhang@163.com>
Adapt unit tests to vLLM main (cdc4824a21) while keeping v0.27.1
behavior, gated by vllm_version_is('0.27.1'):
- test_extract_hidden_states_proposer.py: vLLM #51458 removed PIN_MEMORY
from extract_hidden_states.py; gate the patch to v0.27.1 only
- test_patch_structured_output.py: vLLM main's _create_grammar passes
stop_token_ids kwarg and reads sampling_params.all_stop_token_ids;
version-gate FakeBackend.compile_grammar signature and mock field
- test_routed_experts.py: vLLM main's RoutedExperts.expert_map reads
quant_method.moe_kernel; version-gate the quant_method mock
Signed-off-by: hfadzxy <starmoon_zhang@163.com>
…LM main
Adapt vllm-ascend to vLLM main (cdc4824a21) while keeping v0.27.1
behavior, using vllm_version_is('0.27.1') gates.
Upstream breaks:
- vllm#47808 ([Spec Decode] DSpark confidence-scheduled verification)
added context_len to execute_model, varlen_decode to CudaGraphManager,
max_query_len to InputBatch.make_dummy, draft_tokens to
sort_batch_req_ids, and confidence_head to Qwen3DSparkModel
- vllm#51865 ([Bugfix][MRV2] Require all requests to be decoding for
uniform-decode dispatch) added batch_req_state to prepare_inputs and
has_prefill to InputBatch
Changes:
- model_runner.py: execute_model accepts context_len; graph_manager_wrapper
factory accepts varlen_decode; prepare_inputs accepts batch_req_state and
passes has_prefill; sort_batch_req_ids passes draft_tokens
- aclgraph_utils.py: ModelAclGraphManager.__init__ accepts varlen_decode
- input_batch.py: AscendInputBatch.make_dummy accepts max_query_len
- qwen3_dspark.py: delegate confidence_head to upstream load_weights
Signed-off-by: hfadzxy <starmoon_zhang@163.com>
Upstream breaks: - vllm#49436: added TEMPORAL_TILES constexpr to postprocess_mamba_fused_kernel - vllm#51838: removed DeepseekV2MoE gate branch, always passes router_logits=hidden_states - vllm#51458: removed PIN_MEMORY from extract_hidden_states.py Changes: - postprocess.py: single TEMPORAL_TILES-aware kernel, 3D grid tile_idx, split chained and into nested ifs, temporal copy tiling - fused_moe.py: version-gate is_internal_router with vllm_version_is - test_patch_mamba_utils_source.py: reflect single-kernel layout - test_extract_hidden_states_proposer.py: version-gate PIN_MEMORY fixture and torch.zeros pin_memory patch for CPU UT Signed-off-by: hfadzxy <starmoon_zhang@163.com>
vLLM main (cdc4824a21) added
um_prefill_lookahead to
get_kv_cache_coordinator and self.num_reprefillable_tokens to the
KVCacheCoordinator base class (vllm#50062). The upstream cache_blocks()
now reads self.num_reprefillable_tokens, but AscendHybridKVCacheCoordinator
overrides __init__ without calling super().__init__(), so the attribute
was never set and cache_blocks raised AttributeError.
Define version-specific AscendHybridKVCacheCoordinator.__init__ and
_build_kv_cache_coordinator gated by vllm_version_is('0.27.1'):
- v0.27.1: unchanged (upstream cache_blocks does not use the attribute)
- main: accept num_prefill_lookahead and set num_reprefillable_tokens
Signed-off-by: hfadzxy <starmoon_zhang@163.com>
…erts
vLLM main (cdc4824a21) removed the is_internal_router branch in both
DeepseekV2MoE.forward and Qwen3MoeSparseMoeBlock.forward (vllm#51838),
always passing router_logits=hidden_states. AscendMoERunner only
recomputed router_logits via the gate when ascend_shared_experts was
present; for models without shared experts (e.g. Qwen3-MoE) the wrong
shaped hidden_states was used as router_logits, producing invalid
topk_ids and EP all_to_all_single split-size mismatches.
Define version-specific _forward_impl gated by vllm_version_is('0.27.1'):
- v0.27.1: unchanged, no shared-expert path does not recompute
- main: recompute router_logits via gate in the shared-experts-None path
too, falling back to gate.weight when weight_fp32 is absent
Signed-off-by: hfadzxy <starmoon_zhang@163.com>
Signed-off-by: hfadzxy <starmoon_zhang@163.com>
Upstream PR #51734 (dc5101fb1b, Aug 10) rewrote FusedInputNorm.forward
to use a broadcast multiply-add (x * weight + bias) instead of
F.batch_norm, removing the running_mean/running_var buffers. That commit
is included in the target 16cfe728, where the old patch crashed with
AttributeError: 'FusedInputNorm' object has no attribute 'running_mean'.
Gate install_patch() with vllm_version_is('0.27.1') so newer versions
use the upstream multiply-add forward as-is. On v0.27.1 FusedInputNorm
does not exist, so the import still fails gracefully under
contextlib.suppress.
Signed-off-by: hfadzxy <starmoon_zhang@163.com>
Signed-off-by: hfadzxy <starmoon_zhang@163.com>
zhangxinyuehfad
force-pushed
the
zxy_main2main_0814
branch
4 times, most recently
from
August 21, 2026 15:11
639b174 to
55f3eb6
Compare
vLLM main (cdc4824a21, vllm#51838) removed the gate branch in Qwen3MoeSparseMoeBlock.forward and always passes router_logits=hidden_states. AscendMoERunner._forward_impl recomputes router_logits as fp32 via F.linear(hidden_states_fp32, gate.weight.to(torch.float32)). On 310P the gate weight is stored in NZ format, so the recomputed fp32 tensor may be in a non-ND format, and npu_moe_gating_top_k_softmax fails with error 161002 (expected x(DT_FLOAT, ND)). v0.27.1 passes bf16 router_logits directly and works, so only cast to fp32 for non-0.27.1 versions to keep behavior compatible. Signed-off-by: hfadzxy <starmoon_zhang@163.com>
zhangxinyuehfad
force-pushed
the
zxy_main2main_0814
branch
from
August 22, 2026 04:01
55f3eb6 to
2187978
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it?
The PR adapts vllm-ascend for compatibility with the latest vLLM main (commit cdc4824a).
.github/vllm-main-verified.commit58d3918tocdc4824a21tests/e2e/pull_request/two_card/lora/test_qwen3moe_lora.py@wait_until_npu_memory_free()decorator totest_qwen3moe_lora_epandtest_qwen3moe_lora_multi_id_eptests/ut/kv_offload/test_native_cpu_offload.pyOffloadingParallelConfigconstructor addeddata_parallel_sizeanddata_parallel_rank_localparamsvllm_version_is("0.27.1"): else branch addsdata_parallel_size=1, data_parallel_rank_local=NonetoOffloadingParallelConfigtests/ut/ops/test_routed_experts.pyis_internal_routerbranch fromDeepseekV2MoE;RoutedExperts.expert_mapreadsquant_method.moe_kernelrouted_experts.quant_method = SimpleNamespace(moe_kernel=None)on mock when not 0.27.1tests/ut/patch/platform/test_patch_structured_output.py_create_grammarpassesstop_token_idskwarg and readssampling_params.all_stop_token_idsFakeBackend.compile_grammarsignature (0.27.1: nostop_token_ids; main: accepts**kwargs);make_requestsetssampling_params.all_stop_token_ids = Noneon maintests/ut/patch/worker/test_patch_mamba_utils_source.pyTEMPORAL_TILESconstexpr topostprocess_mamba_fused_kernel_postprocess_kernels()recursively collects both version-gated kernel definitions;_selected_kernel_source()selects by version; main branch assertsTEMPORAL_TILES,tile_idx,if tile_idx == 0:presenttests/ut/spec_decode/test_extract_hidden_states_proposer.pyPIN_MEMORYfromextract_hidden_states.py_no_pin_memoryfixture version-gated: 0.27.1 patchesPIN_MEMORY=False; main patchestorch.zerosto strippin_memorykwargvllm_ascend/_310p/fused_moe/grouped_topk_router.py_forward_implrecomputesrouter_logitsas fp32, which may be in FRACTAL_NZ format on 310Prouter_logitstotorch.float16beforenpu_moe_gating_top_k_softmax(DT_FLOAT16 + ND required)vllm_ascend/__init__.pyos.getenv("VLLM_VERSION")version gate from triton gluon/_aggregatestubs after rebase; stubs now applied unconditionally at module loadvllm_ascend/attention/dsa_v1.pyblock_sizetostorage_block_sizein DSA metadata builderself.block_sizetoself.storage_block_sizeinbuild_dspark_swa_indicescallvllm_ascend/models/deepseek_v4/model.py_mtp_hidden_bufferonly for EAGLE/draft methods, not MTP-only_mtp_hidden_bufferallocation gated onget_pp_group().is_last_rank and needs_mtp_hidden_states(EAGLE or draft model, not MTP)vllm_ascend/models/qwen3_dspark.pyload_weightsalready managesconfidence_head__init__gatesconfidence_headcreation withvllm_version_is("0.27.1");load_weightson main delegates directly tosuper().load_weights(all_weights)vllm_ascend/ops/fused_moe/fused_moe.pyis_internal_routerbranch fromDeepseekV2MoE.forward, always passesrouter_logits=hidden_statesis_internal_routerproperty version-gated: 0.27.1 checkshasattr(gate, "weight_fp32"); main checks onlyself.gate is not None. Entire_forward_implduplicated: main branch recomputesrouter_logitsvia gate even in no-shared-experts path, falls back togate.weight.to(torch.float32)whenweight_fp32absentvllm_ascend/ops/triton/mamba/postprocess.pyTEMPORAL_TILESconstexpr parameterTEMPORAL_TILES; main branch uses 3D grid, temporal copy partitioned across CTAs, conv state copies only ontile_idx == 0,num_accepted_tokensstore split into nested ifsvllm_ascend/patch/__init__.pypatch_vision.py(FusedInputNorm eps) andpatch_distributed.py(GroupCoordinator/HCCL registry) patchesvllm_ascend/patch/platform/patch_kv_cache_coordinator.pyget_kv_cache_coordinatoraddednum_prefill_lookahead: int = 0param;KVCacheCoordinatorbase class addedself.num_reprefillable_tokensAscendHybridKVCacheCoordinator.__init__,_build_kv_cache_coordinator,get_kv_cache_coordinatorall version-gated: main branch addsnum_prefill_lookaheadparam and setsself.num_reprefillable_tokensvllm_ascend/patch/platform/patch_vision.pyFusedInputNorm.forwardto use multiply-add instead ofF.batch_norm, removing theeps>0requirementvllm_version_is("0.27.1"): replacesFusedInputNorm.forwardto useeps=1e-5instead ofeps=0.0for NPUF.batch_normcompatibilityvllm_ascend/patch/worker/patch_distributed.pyGroupCoordinator.__init__addeduse_all2all: bool = FalseparamGroupCoordinatorwithGroupCoordinatorPatchacceptinguse_all2all; uses HCCL PG registry for process group deduplication; wrapsdestroy_distributed_environmentto clear registryvllm_ascend/patch/worker/patch_qwen3_5.py_all_gather_hidden_and_residualfromqwen3_next; SP all-gather inlined into decoder layervllm_version_is("0.27.1");qwen3_5_mtp_forward: 0.27.1 uses_all_gather_hidden_and_residual, main usestensor_model_parallel_all_gathervllm_ascend/worker/v2/aclgraph_utils.pyvarlen_decodetoCudaGraphManagerModelAclGraphManager.__init__version-gated: main branch addsvarlen_decode: bool = Falseparametervllm_ascend/worker/v2/input_batch.pyInputBatchaddedmax_query_len: int | None = Nonefieldseq_lens_npfield version-gated: 0.27.1 has no default, main branch= None;make_dummyclassmethod also version-gated: main branch addsmax_query_lenparametervllm_ascend/worker/v2/model_runner.pyexecute_modeladdedcontext_len;prepare_inputsaddedbatch_req_state/has_prefill;sort_batch_req_idsaddeddraft_tokens;graph_manager_wrapperaddedvarlen_decodeexecute_model: main passescontext_len;prepare_inputs: fully duplicated, main acceptsbatch_req_state, passeshas_prefill, updatedsort_batch_req_idssignature;graph_manager_wrapper: main addsvarlen_decode; module-levelBatchReqStateimport gated onnot vllm_version_is("0.27.1")Does this PR introduce any user-facing change?
How was this patch tested?