Skip to content

[XPU][UT] Fix OOM and skip graph case - #49287

Merged
jikunshang merged 8 commits into
vllm-project:mainfrom
mayuyuace:qiming/fix_oom
Aug 18, 2026
Merged

[XPU][UT] Fix OOM and skip graph case#49287
jikunshang merged 8 commits into
vllm-project:mainfrom
mayuyuace:qiming/fix_oom

Conversation

@mayuyuace

@mayuyuace mayuyuace commented Jul 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Fix UT on XPU:

  1. test_mamba_cache_cg_padding can only run with graph enabled. Otherwise, UT will fall into an infinite loop.
  2. To address the issue of insufficient VRAM caused by consecutive use cases (where there is insufficient free memory upon engine startup), the approach is to reuse the existing ROCm VRAM waiting mechanism and provide an appropriate default gpu_memory_utilization for the XPU.

@mayuyuace
Fix OOM and skip graph case
fe186d6 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
claude[bot]
claude Bot reviewed Jul 21, 2026
View reviewed changes

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added the intel-gpu Related to Intel GPU label Jul 21, 2026
@jikunshang

jikunshang commented Jul 21, 2026

Copy link
Copy Markdown
Member

@chaojun-zhang for the second part, I feel you should already fix it.

@mayuyuace

mayuyuace commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

@chaojun-zhang for the second part, I feel you should already fix it.

I tested the latest vllm, but problem is not fixed yet.

@mayuyuace mayuyuace closed this Aug 5, 2026
@mayuyuace
mayuyuace deleted the qiming/fix_oom branch August 5, 2026 06:32
@mayuyuace
mayuyuace restored the qiming/fix_oom branch August 5, 2026 06:33
@mayuyuace mayuyuace reopened this Aug 5, 2026
@jikunshang

jikunshang commented Aug 12, 2026

Copy link
Copy Markdown
Member

cc @chaojun-zhang @zhenwei-intel can you take a look?

@zhenwei-intel

zhenwei-intel commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

sycl graph cannot work with flash attn kernel

flash attn is supported by sycl graph in pytorch2.13

@mayuyuace

mayuyuace commented Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

sycl graph cannot work with flash attn kernel

flash attn is supported by sycl graph in pytorch2.13

Thanks, I will try it with torch 2.13.

@mayuyuace
Merge branch 'main' into qiming/fix_oom
cf3a149
@mayuyuace
skip ut if graph is disabled
85925ce 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
@mayuyuace

mayuyuace commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

sycl graph cannot work with flash attn kernel

flash attn is supported by sycl graph in pytorch2.13

Verified.
Change the skip condition to if graph is disabled; otherwise, it will result in an infinite loop.

chaojun-zhang
chaojun-zhang reviewed Aug 13, 2026
View reviewed changes
Comment thread tests/conftest.py Outdated
torch._dynamo.reset()
cleanup_dist_env_and_memory()
self._wait_for_rocm_memory_release(gpu_memory_utilization)
self._wait_for_xpu_memory_release(gpu_memory_utilization)

@chaojun-zhang chaojun-zhang Aug 13, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can merge _wait_for_rocm_memory_release and _wait_for_xpu_memory_release to single function _wait_for_memory_release

@mayuyuace mayuyuace Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merged.

chaojun-zhang
chaojun-zhang reviewed Aug 13, 2026
View reviewed changes
Comment thread tests/conftest.py Outdated
# previous engine shuts down, so wait before constructing LLM.
from tests.utils import wait_for_xpu_memory_to_settle

wait_for_xpu_memory_to_settle(threshold_ratio=1.0 - gpu_memory_utilization)

@chaojun-zhang chaojun-zhang Aug 13, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we merge wait_for_xpu_memory_to_settle and wait_for_rocm_memory_to_settle to single function wait_for_memory_to_settle ?

@mayuyuace mayuyuace Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merged.

@mayuyuace
merge wait function
f13394a 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
@mergify mergify Bot added multi-modality Related to multi-modality (#4194) mistral Related to Mistral models kv-connector labels Aug 13, 2026
@mergify

mergify Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @mayuyuace.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Aug 13, 2026
@mayuyuace
Merge branch 'main' into qiming/fix_oom
8a5b1d9 
Signed-off-by: Qiming Zhang <qiming1.zhang@intel.com>
@mergify mergify Bot removed the needs-rebase label Aug 14, 2026
@jikunshang

jikunshang commented Aug 15, 2026

Copy link
Copy Markdown
Member

/ci run

@github-actions

github-actions Bot commented Aug 15, 2026

Copy link
Copy Markdown

✅ Triggered Buildkite CI #84025 for commit cd874ca8aa02.

@mayuyuace

mayuyuace commented Aug 17, 2026

Copy link
Copy Markdown
Contributor Author

/ci run

@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown

✅ Triggered Buildkite CI #84131 for commit 9f3cc3befcb8.

@jikunshang

jikunshang commented Aug 18, 2026

Copy link
Copy Markdown
Member

/ci retry

@github-actions

github-actions Bot commented Aug 18, 2026

Copy link
Copy Markdown

✅ Queued 1 failed job(s) for retry in Buildkite CI #84131.

@jikunshang
jikunshang merged commit f4b161d into vllm-project:main Aug 18, 2026
69 checks passed
@mayuyuace
mayuyuace deleted the qiming/fix_oom branch August 20, 2026 07:13
zyp2014 pushed a commit to zyp2014/vllm that referenced this pull request Aug 21, 2026
@mayuyuace @jikunshang zhangyp26
[XPU][UT] Fix OOM and skip graph case (vllm-project#49287)
ff7a057 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Signed-off-by: Qiming Zhang <qiming1.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
wyettzeng pushed a commit to wyettzeng/vllm that referenced this pull request Aug 21, 2026
@mayuyuace @jikunshang @wyettzeng
[XPU][UT] Fix OOM and skip graph case (vllm-project#49287)
3319edb 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Signed-off-by: Qiming Zhang <qiming1.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Wyett <wyettzeng@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@claude claude[bot] claude[bot] left review comments
@jikunshang jikunshang jikunshang approved these changes
@tdoublep tdoublep Awaiting requested review from tdoublep tdoublep is a code owner
@tomeras91 tomeras91 Awaiting requested review from tomeras91 tomeras91 is a code owner
@noooop noooop Awaiting requested review from noooop noooop is a code owner
@ApostaC ApostaC Awaiting requested review from ApostaC ApostaC is a code owner
@orozery orozery Awaiting requested review from orozery orozery is a code owner
@ivanium ivanium Awaiting requested review from ivanium ivanium is a code owner
@DarkLight1337 DarkLight1337 Awaiting requested review from DarkLight1337 DarkLight1337 is a code owner
@ywang96 ywang96 Awaiting requested review from ywang96 ywang96 is a code owner
@AndreasKaratzas AndreasKaratzas Awaiting requested review from AndreasKaratzas AndreasKaratzas is a code owner
@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat
@aarnphm aarnphm Awaiting requested review from aarnphm
@NickLucche NickLucche Awaiting requested review from NickLucche
+2 more reviewers
@chaojun-zhang chaojun-zhang chaojun-zhang approved these changes
@zhenwei-intel zhenwei-intel zhenwei-intel approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

intel-gpu Related to Intel GPU kv-connector mistral Related to Mistral models multi-modality Related to multi-modality (#4194)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@mayuyuace @jikunshang @zhenwei-intel @chaojun-zhang