Skip to content

[Bugfix] Restore multimodal support on the plain "vllm" throughput backend - #52168

Open
mganczarenko wants to merge 1 commit into
vllm-project:mainfrom
mganczarenko:fix/throughput-vllm-backend-multimodal
Open

[Bugfix] Restore multimodal support on the plain "vllm" throughput backend#52168
mganczarenko wants to merge 1 commit into
vllm-project:mainfrom
mganczarenko:fix/throughput-vllm-backend-multimodal

Conversation

@mganczarenko

@mganczarenko mganczarenko commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Purpose

vllm bench throughput --backend vllm crashes on any multimodal dataset (e.g. ASR/Whisper) with:

ValueError: Multi-modal content is not supported on backend 'vllm'; use one of [].

Regression from #50981. get_requests() computed the multimodal-backend allowlist from the backend already selected instead of from what the backend can actually handle:

mm_backends = ("vllm-chat",) if args.backend == "vllm-chat" else ()

This is circular — it only lets the already-selected backend through the gate, and only if it's literally "vllm-chat". So --backend vllm always computes mm_backends=() and self-rejects, even though _run_vllm_requests (the function that executes --backend vllm requests) already
forwards multi_modal_data straight into LLM.generate — it was always multimodal-capable, same as vllm-chat. Only hf (asserts prompts must be str) and mii (text-only) genuinely can't carry multimodal content, so the allowlist should be static rather than derived from the chosen
backend:

  mm_backends = ("vllm", "vllm-chat")

Test Plan

  • Added test_get_requests_allows_multimodal_on_plain_vllm_backend to tests/benchmarks/test_throughput_cli.py, mirroring the existing test_get_requests_resolves_mmvu pattern: stubs ASRDataset (network-free), parses --backend vllm --dataset-name hf --dataset-path
    openslr/librispeech_asr, asserts get_requests() resolves without raising.
  • Ran the exact vllm bench throughput command from the bug report end-to-end on real hardware (Intel XPU B70) before and after the fix.
  • ruff check / ruff format on both changed files.

Test Result

  • New test: fails with the reported ValueError against pre-fix code, passes against post-fix code.
  • python -m pytest tests/benchmarks/test_throughput_cli.py -k "allows_multimodal_on_plain_vllm_backend or resolves_mmvu" → 2 passed.
  • End-to-end on B70: pre-fix reproduces the ValueError verbatim; post-fix completes cleanly (Throughput: 20.96 requests/s, 5386.74 total tokens/s).
  • ruff check: all checks passed. ruff format --diff: no changes.

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

…ckend

  get_requests() only let multimodal content through "vllm-chat", even
  though the plain "vllm" backend also forwards multi_modal_data straight
  to LLM.generate. --backend vllm always computed an empty allowlist and
  self-rejected (e.g. ASR/Whisper datasets). Regression from vllm-project#50981.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added performance Performance-related issues bug Something isn't working labels Aug 13, 2026
@DarkLight1337

Copy link
Copy Markdown
Member

cc @JaredforReal

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working performance Performance-related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants