Skip to content

[Bugfix] Resolve B12X modules before Dynamo tracing - #53326

Open
lukealonso wants to merge 1 commit into
vllm-project:mainfrom
local-inference-lab:dev/b12x-dynamo-import
Open

[Bugfix] Resolve B12X modules before Dynamo tracing#53326
lukealonso wants to merge 1 commit into
vllm-project:mainfrom
local-inference-lab:dev/b12x-dynamo-import

Conversation

@lukealonso

@lukealonso lukealonso commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Purpose

Resolve optional B12X modules before TorchDynamo traces model forwards. The B12X accessors previously used functools.cache around importlib.import_module; Dynamo unwraps cached functions and can trace the import during the memory-profile forward. Import attempts now happen when vllm.utils.b12x loads, while runtime accessors perform dictionary lookups only.

This allows B12X linear kernels to run with vLLM model compilation enabled instead of requiring compilation mode NONE.

Duplicate-work checks found no open issue or PR addressing the B12X/Dynamo import path. AI assistance was used to diagnose, implement, and validate this change; the submitter reviewed the resulting diff.

Test Plan

pre-commit run --files \
  vllm/utils/b12x.py \
  tests/model_executor/kernels/test_b12x_linear.py

.venv/bin/python -m pytest -q \
  tests/model_executor/kernels/test_b12x_linear.py::test_b12x_module_lookup_is_dynamo_safe

The regression test calls the B12X accessor from a torch.compile(backend="eager", fullgraph=True) function.

Test Result

  • Scoped pre-commit hooks: passed.
  • Targeted fullgraph Dynamo regression test: 1 passed.
  • Model evaluation was not run because this changes import timing only; kernel selection, tensor operations, and model outputs are unchanged.

Import optional B12X modules when the utility module loads so compiled forwards only perform dictionary lookups. Add a fullgraph regression test for the accessor path.

Assisted-by: OpenAI Codex

Signed-off-by: Luke Alonso <lalonso@gmail.com>
@mergify mergify Bot added the bug Something isn't working label Aug 21, 2026
@lukealonso
lukealonso marked this pull request as ready for review August 21, 2026 22:22

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@eugr

eugr commented Aug 21, 2026

Copy link
Copy Markdown

@mgoin - can we please merge this one? Fixes vLLM crashing with an exception after model loading.

@mgoin
mgoin enabled auto-merge (squash) August 21, 2026 22:34
@github-actions github-actions Bot added the ready ONLY add when PR is ready to merge/full CI is needed label Aug 21, 2026
@mgoin

mgoin commented Aug 21, 2026

Copy link
Copy Markdown
Member

/ci run

@github-actions

Copy link
Copy Markdown

✅ Triggered Buildkite CI #85125 for commit 7faa72985478.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants