[Bug] Gemma-4-31B-it TP=8 hangs in the vision path on AMD/ROCm

[victrgao]

Checklist

• I searched related issues but found no solution.
• The bug persists in the latest version.
• Issues without environment info and a minimal reproducible demo are hard to resolve and may receive no feedback.
• If this is not a bug report but a general question, please start a discussion at https://github.com/sgl-project/sglang/discussions. Otherwise, it will be closed.
• Please use English. Otherwise, it will be closed.

Describe the bug

[Bug] Gemma-4-31B-it TP=8 hangs in the vision path on AMD/ROCm

When running Gemma-4-31B-it with TP=8 and hybrid SWA enabled on AMD/ROCm, all TP ranks initialize and load the model successfully, but the server never becomes healthy.

Observed behavior

• TP0–TP7 complete distributed initialization.
• Model weights and KV cache are loaded successfully.
• /model_info returns HTTP 200.
• /health continuously returns HTTP 503.
• After several minutes, scheduler watchdog timeouts occur.

The watchdog stack traces show multiple TP ranks blocked in:

rocr::core::BusyWaitSignal::WaitRelaxed
resolve_precomputed_max_seqlen (vision.py:237)
get_image_feature (gemma4_mm.py:443)
general_mm_embed_routine (mm_utils.py:1260)
run_scheduler_process (scheduler.py:4639)

TP0 also shows:

set_full_to_swa_mapping (allocator/swa.py:345)

The server eventually receives SIGQUIT and is killed.

Configuration

• Model: Gemma-4-31B-it
• TP/PP/EP: 8/1/1
• AMD GPU server with 8 GPUs

An earlier run with only HIP_VISIBLE_DEVICES=0 failed with invalid device ordinal. After exposing all eight GPUs, that error disappeared, but the TP=8 run still hung in the vision/SWA execution path and failed readiness.

Expected behavior

The server should complete startup and /health should return HTTP 200.

Questions

1. Is TP=8 supported for Gemma 4 hybrid models on AMD/ROCm?
2. Is the hang in gemma4_mm.py / vision.py a known issue?
3. Could hybrid SWA allocation or TP=8 synchronization cause this ROCr wait?
4. Are any additional startup or multimodal configuration options required?

Full server log attached.

Reproduction

python3 -m sglang.launch_server
--model-path gemma-4-31B-it/
--host 0.0.0.0
--port 30000
--tp 8
--context-length 16384
--max-running-requests 8
--mem-fraction-static 0.90
--attention-backend triton
--kv-cache-dtype fp8_e5m2
--disable-radix-cache
--chunked-prefill-size -1
--disable-shared-experts-fusion
--disable-cuda-graph

python3 -m sglang.bench_serving
--backend sglang
--base-url http://127.0.0.1:30000
--model gemma-4-31B-it/
--dataset-name random-ids
--random-input-len 1024
--random-output-len 1024
--random-range-ratio 1.0
--max-concurrency 8
--request-rate inf
--seed 0
--extra-request-body '{"temperature":0}'
--tokenize-prompt
--num-prompts 16

Environment

Python: 3.12.3 (main, Jun 19 2026, 12:46:00) [GCC 13.3.0]
ROCM available: True
GPU 0,1,2,3,4,5,6,7: AMD Radeon AI PRO R9700
GPU 0,1,2,3,4,5,6,7 Compute Capability: 12.0
ROCM_HOME: /opt/venv/lib/python3.12/site-packages/_rocm_sdk_devel
HIPCC: HIP version: 7.14.60850-0000000
ROCM Driver Version: 6.19.14.31400000
PyTorch: 2.11.0+rocm7.14.0
sglang: 0.5.17.dev20+g6d2a5996e
sglang-kernel: 0.4.5
flashinfer_python: Module Not Found
flashinfer_cubin: Module Not Found
flashinfer_jit_cache: Module Not Found
triton: 3.7.1+git0263a6a6.rocm7.14.0
transformers: 5.12.1
torchao: 0.9.0
numpy: 2.3.5
aiohttp: 3.14.3
fastapi: 0.140.0
huggingface_hub: 1.24.0
interegular: 0.3.3
modelscope: 1.39.0
orjson: 3.11.9
outlines: 0.1.11
packaging: 26.2
psutil: 7.2.2
pydantic: 2.13.4
python-multipart: 0.0.32
pyzmq: 27.1.0
uvicorn: 0.51.0
uvloop: 0.22.1
vllm: 0.21.0+rocm714
xgrammar: 0.2.1
openai: 2.6.1
tiktoken: 0.13.0
anthropic: 0.120.0
litellm: Module Not Found
torchcodec: Module Not Found
AMD Topology:

============================ ROCm System Management Interface ============================
=============================== Link Type between two GPUs ===============================
GPU0 GPU1 GPU2 GPU3 GPU4 GPU5 GPU6 GPU7
GPU0 0 PCIE PCIE PCIE PCIE PCIE PCIE PCIE
GPU1 PCIE 0 PCIE PCIE PCIE PCIE PCIE PCIE
GPU2 PCIE PCIE 0 PCIE PCIE PCIE PCIE PCIE
GPU3 PCIE PCIE PCIE PCIE PCIE PCIE PCIE PCIE
GPU4 PCIE PCIE PCIE PCIE 0 PCIE PCIE PCIE
GPU5 PCIE PCIE PCIE PCIE PCIE 0 PCIE PCIE
GPU6 PCIE PCIE PCIE PCIE PCIE PCIE 0 PCIE
GPU7 PCIE PCIE PCIE PCIE PCIE PCIE 0
================================== End of ROCm SMI Log ===================================