Skip to content

Revert "[ROCm][Perf] Kimi-K3 Fused kernels for KDA prefill" - #53294

Merged
vllm-bot merged 2 commits into
vllm-project:mainfrom
khluu:revert-rocm-kimi-k3-gfx950
Aug 21, 2026
Merged

Revert "[ROCm][Perf] Kimi-K3 Fused kernels for KDA prefill"#53294
vllm-bot merged 2 commits into
vllm-project:mainfrom
khluu:revert-rocm-kimi-k3-gfx950

Conversation

@khluu

@khluu khluu commented Aug 21, 2026

Copy link
Copy Markdown
Member

Why

This reverts #52606 because it deterministically breaks the AMD CI image build on every architecture set that includes gfx90a:

csrc/libtorch_stable/kimi_k3/fused_kda_chunk_kernel_rocm.hip:800:18:
error: '__builtin_amdgcn_mfma_f32_16x16x32_bf16' needs target feature gfx950-insts

The merged commit 463aa5e3 is the first affected main commit. Main builds #85050 through #85055 reproduce the same compile failure; downstream AMD jobs cannot start.

The exact PR-head Buildkite #84808 selected AMD: build test image and artifacts and failed this identical error on all three attempts. This is deterministic code evidence rather than runner noise.

Duplicate-work check

Searched open PRs for 52606, gfx950-insts fused_kda_chunk_kernel_rocm, and Kimi K3 ROCm revert; no existing revert or equivalent fix was found.

Validation

  • git diff HEAD^ --check
  • Verified this commit is the exact inverse of merge commit 463aa5e3 with git apply --check --reverse
  • Signed exact-head Buildkite #85061 at be59e0b95257: AMD: build test image and artifacts passed on buildkite-01a02547-d188-4c46-bfbb-711ca04a2d34-sk26s from 17:06:34–17:24:16 UTC (17m41s), including the multi-architecture ROCm compile and image smoke. The overall build is red only from unrelated CPU/Ascend jobs; the exact AMD gate passed.

No model evaluation is applicable because this is an emergency rollback of a change that cannot compile in the supported multi-architecture AMD CI image.

Merge gate

Keep this draft until human line-by-line review. The incident remains open after pre-merge validation and closes only after merge plus an exact post-merge main AMD image pass.

AI assistance

AI assistance was used to investigate the CI failure, prepare the revert, and draft this PR. The human submitter must review every changed line and can explain and defend the change end-to-end.

@khluu

khluu commented Aug 21, 2026

Copy link
Copy Markdown
Member Author

/ci run

@github-actions

Copy link
Copy Markdown

✅ Triggered Buildkite CI #85058 for commit 8f035cce9763.

@mergify mergify Bot added k3 rocm Related to AMD ROCm labels Aug 21, 2026
@github-project-automation github-project-automation Bot moved this to Todo in AMD Aug 21, 2026
@khluu

khluu commented Aug 21, 2026

Copy link
Copy Markdown
Member Author

Exact pre-merge AMD image validation passed at this PR head. Buildkite #85058 ran AMD: build test image and artifacts at 8f035cce97633d257bedd0c8b99716eac246b622 on aus-turin-sr06-01-3 from 16:50:55–16:57:59 UTC (7m04s), exited 0, and completed AMD image smoke OK. This directly covers the deterministic gfx950-builtin/gfx90a compile failure from #52606. The PR stays draft pending human line-by-line review; the incident stays unresolved until merge plus an exact post-merge main AMD image pass.

…ect#52606)"

This reverts commit 463aa5e.

Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
@khluu

khluu commented Aug 21, 2026

Copy link
Copy Markdown
Member Author

/ci run

@khluu
khluu force-pushed the revert-rocm-kimi-k3-gfx950 branch from 8f035cc to be59e0b Compare August 21, 2026 17:02
@github-actions

Copy link
Copy Markdown

✅ Triggered Buildkite CI #85061 for commit be59e0b95257.

@khluu

khluu commented Aug 21, 2026

Copy link
Copy Markdown
Member Author

Signed-head exact rerun is Buildkite #85061. The gating AMD: build test image and artifacts job is running at be59e0b952578bb21aaaf87c2b84305542b61f19; awaiting terminal result.

@khluu

khluu commented Aug 21, 2026

Copy link
Copy Markdown
Member Author

Strict signed-head gate is green: Buildkite #85061 AMD: build test image and artifacts passed at exact head be59e0b952578bb21aaaf87c2b84305542b61f19 from 17:06:34–17:24:16 UTC. The overall build is red only from unrelated CPU/Ascend jobs; the exact AMD gate passed.

@yewentao256 yewentao256 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's revert this to make main green

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@AndreasKaratzas

Copy link
Copy Markdown
Member

/ci run

@github-actions

Copy link
Copy Markdown

✅ Triggered Buildkite CI #85097 for commit 553d0b4e3ef4.

@vllm-bot
vllm-bot merged commit 592e06f into vllm-project:main Aug 21, 2026
18 of 36 checks passed
@github-project-automation github-project-automation Bot moved this from Todo to Done in AMD Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants