[ROCm] Add TheRock preview docker updates, Keep Python 3.12 and Ubuntu 22.04 by rasmith · Pull Request #49925 · vllm-project/vllm

rasmith · 2026-07-27T02:32:21Z

Purpose

This PR adds to using The Rock 7.14 with wheels provided by The Rock as a preview version while keeping Python 3.12 and Ubuntu 22.04 as-is.

Test Plan

Full CI runs.

Test Result

Current persistently failing groups that need to be addressed are:

MI355
Language Models Tests (Standard)
Multi-Modal Models (Extended Generation 3)

Keeping track in this BK build:https://buildkite.com/vllm/amd-ci/builds/12222/list

Essential Elements of an Effective PR Description Checklist

[X ] The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
[X ] The test plan, such as providing test command.
[ X] The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

tjtanaa · 2026-08-19T16:05:47Z

+           export HIP_CLANG_PATH=/opt/sccache-wrappers \
+           && sccache --show-stats; \
+       fi \
+    && PREBUILD_KERNELS=1 AITER_USE_SYSTEM_TRITON=1 GPU_ARCHS=${AITER_ROCM_ARCH} python3 setup.py bdist_wheel --dist-dir=dist \


Should aiter also be pip installable?

I'd recommend we push that to a follow-up ... we haven't throughly tested the AITER prebuilt wheels yet

I'm trying it today. @tjtanaa @Rohan138.

I'll put it through CI tonight.

Can we do some one do some additional testing?

It seems to work. I am running the full CI right now. We will need to do some additional testing. @tjtanaa @Rohan138 You can get the images from the CI and do additional testing if you want: https://buildkite.com/vllm/amd-ci/builds/12262/list

Rohan138 · 2026-08-19T19:22:38Z

+        | grep -E '^(torch|triton|torchaudio|torchvision)==' > /etc/rocm-constraints.txt \
+    && echo "ROCm constraints:" && cat /etc/rocm-constraints.txt \
+    && test "$(wc -l < /etc/rocm-constraints.txt)" -eq 4 \
+    && python3 -c "import torch; assert torch.version.hip, f'non-ROCm torch: {torch.__version__}'; print(f'torch={torch.__version__}')" \


I assume this came from debugging the torch/triton compatibility issues, can we remove this comment+rocm_constraints+torch/triton assert and print now?

I'm working on this part at the moment. I"ll keep the comments short.

Simplification done, will post soon.

Rohan138 · 2026-08-19T19:23:45Z

+    && TRITON_VERSION=$(python3 -c "from importlib.metadata import version; print(version('triton'))") \
+    && sed -i "s/^Requires-Dist: triton==.*/Requires-Dist: triton==${TRITON_VERSION}/" \
+        ${SITE_PACKAGES}/torch-*.dist-info/METADATA \
+    # Editing METADATA stales its RECORD hash; restamp so wheel verifiers pass.


Do we need any of this after the sed, or was the METADATA also edited for debug purposes?

torch pins triton==3.7.1, so this prevents it from resolving it to a version of triton we don't want. It also prevents other resolvers from seeing the pin as unsatisfiable and then installing triton 2.13 with CUDA.

I still need to revisit this to see if a constraints file will work.

I have a simplification, but the source builds don't add the METADATA and RECORD information that cause unpreventable reinstalls, so we do need to patch that in-place.

Rohan138 · 2026-08-19T19:26:56Z

+RUN --mount=type=bind,from=debs,src=/app/debs,target=/install \
+    pip install $(ls /install/*.whl | grep -v '/triton-') \
+    && pip install --no-deps --force-reinstall /install/triton-*.whl \
+    # Point torch's pin at the triton we built. Left unsatisfied, downstream


What's the downstream resolver here? Is it a subsequent pip install either in this file or in Dockerfile.rock?

There's a lot of resolver issues with this Triton installation. Found this out by trying to get a Triton 3.6.0 and it repeatedly got swapped out for 3.7.1, I think at least at 5 different place, most (maybe all) in Dockerfile.rock. If you're not careful, you will lose the Triton you wanted. I had to use constraints files.

OK, I have a simplification that I will be uploading shortly.

Rohan138 · 2026-08-19T19:33:31Z

+    && curl -sS https://bootstrap.pypa.io/get-pip.py | python${PYTHON_VERSION} \
+    && python3 --version && python3 -m pip --version
+
+RUN pip install -U packaging 'cmake<4' ninja wheel 'setuptools<80' pybind11 Cython


Can we copy over the pybind pin to 3.0.4 and the new Triton commit from https://github.com/vllm-project/vllm/pull/52819/changes? 52819 will be merged first, just waiting on CI

Yes, will do.

@Rohan138 What do you mean by "copy over pybind pin to 3.0.4"?

Rohan138 · 2026-08-19T19:34:17Z

+ARG COMMON_WORKDIR=/app
+ARG BASE_IMAGE=rocm/vllm-dev:base
+ARG CI_BASE_IMAGE=rocm/vllm-dev:ci_base
+ARG ROCM_TRITON_KERNELS_COMMIT=0f380657dbf3ee86eb57558ff71df24f03b5d4e7


Out of curiosity, why did we pin TRITON_KERNELS to this specific commit?

I don't know. This was the PR: #51721

hmm ... @AndreasKaratzas @micah-wil this won't hold up this PR, but I'm not sure if we should pin triton_kernels in our docker image for CI?

Rohan138 · 2026-08-19T19:38:12Z

+ARG FA_REPO="https://github.com/Dao-AILab/flash-attention.git"
+ARG AITER_BRANCH="v0.1.19"
+ARG AITER_REPO="https://github.com/ROCm/aiter.git"
+ARG MORI_BRANCH="f95cd5e0ca7b730f92f9ffdfb358484d6965d9f7" # TheRock compatible v1.1.0


Do we have a MORI PR up for this change?

I was working on one, but there's this PR: ROCm/mori#541. Also that change was already in MORI, but not MORI 1.1.0, maybe we could bump, but might be best to wait for the next version which should have support for The Rock.. However, the PR I linked is pretty much tip of tree.

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

Rohan138 · 2026-08-21T15:14:41Z

+ARG TRITON_REPO="https://github.com/ROCm/triton.git"
+ARG FA_BRANCH="0e60e394"
+ARG FA_REPO="https://github.com/Dao-AILab/flash-attention.git"
+ARG AITER_BRANCH="v0.1.19.post2"


Let's revert to AITER 0.1.19 to align with upstream, I'll bump both dockerfiles to 0.1.20 as part of #52826

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

rasmith added 29 commits July 22, 2026 16:08

first pass

8e29411

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

add triton back

1f74131

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

document library issues

a0c5b3e

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

check for wheel existance

cf32cd5

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

versioning todos and updates

016bbe5

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

update pip extra url

bb50962

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

add option to build custom triton

a00fc87

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

add torchcodec update

4a3c431

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

pin versions and remove --system

64c333b

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

use 3.7.1 and update torch deps

fb52dc6

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

don't use s3 buckets

b668698

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

use system python, install device arches, prevent nvidia installs

a176de9

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

use upstream sscache config

fa86a65

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

update sccache to align with upstream

d8e46c5

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

update sccache to align with upstream

7efe9ff

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

update sccache to align with upstream

e839eb6

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

update sccache to align with upstream

784c2b6

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

use upstream aiter build

94fd363

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

merge main

28a1aec

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

alignment

4b30f2e

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

upstream alignment

ccf6c29

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

upstream alignment

8ba499e

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

nixl wheel

52b81e5

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

nixl wheel -manylinux too old

b562f42

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

update tensoeizer

2ecec8d

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

312 version

3042958

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

revert setup.py

52b3edc

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

remove ubuntu jammy stuff

d5b5970

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

cleanup

16379e6

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

mergify Bot added the ci/build label Jul 27, 2026

resolve merge conflict

68db0cf

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

mergify Bot removed the needs-rebase label Aug 12, 2026

rasmith added 6 commits August 12, 2026 22:08

use .rock_base / .rock, make a build/rock.txt file

1b17405

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

revert dockerfile

532a312

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

update base image

198523d

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

Merge branch 'rock-312' of github.com:rasmith/vllm into rock-312

3008c94

update wheel url

eff102d

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

update rock_base

50e5dcb

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

Rohan138 mentioned this pull request Aug 14, 2026

[ROCm]: Drop pybind11 from Dockerfile.rocm to prevent version mismatch #52400

Merged

4 tasks

rasmith added 3 commits August 18, 2026 13:33

Merge branch 'vllm-project:main' into rock-312

299fe8b

add 3.7.1 + K3 commits + update python path and wheelurls

e63702e

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

Merge branch 'rock-312' of github.com:rasmith/vllm into rock-312

b3c77cf

rasmith changed the title ~~[ROCm] Switch to the Rock, Keep Python 3.12 and Ubuntu 22.04~~ [ROCm] Add to TheRock preview docker updates, Keep Python 3.12 and Ubuntu 22.04 Aug 19, 2026

rasmith changed the title ~~[ROCm] Add to TheRock preview docker updates, Keep Python 3.12 and Ubuntu 22.04~~ [ROCm] Add TheRock preview docker updates, Keep Python 3.12 and Ubuntu 22.04 Aug 19, 2026

tjtanaa reviewed Aug 19, 2026

View reviewed changes

Rohan138 reviewed Aug 19, 2026

View reviewed changes

rasmith added 2 commits August 20, 2026 02:31

use aiter wheel, simplify triton install, use pybind 3.0.4

badbd82

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

typos fix

4d9d308

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

rasmith requested a review from khluu as a code owner August 20, 2026 02:57

rasmith added 4 commits August 19, 2026 21:57

Merge branch 'main' into rock-312

588c5b2

add kmod

0b6537f

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

fix llvmobjdump path

21b1088

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

back to building aiter from source

caeca59

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

Rohan138 reviewed Aug 21, 2026

View reviewed changes

aiter -> 0.1.19

b03ba4f

Signed-off-by: Randall Smith <Randall.Smith@amd.com>

Uh oh!

Uh oh!

Conversation

rasmith commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Rohan138 Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rasmith Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rasmith Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Rohan138 Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

rasmith commented Jul 27, 2026 •

edited

Loading

Rohan138 Aug 19, 2026 •

edited

Loading

rasmith Aug 19, 2026 •

edited

Loading

rasmith Aug 19, 2026 •

edited

Loading

Rohan138 Aug 21, 2026 •

edited

Loading