[Attention] Fix FlashInfer SM12x prefill with sinks - #52148
Merged
vllm-bot merged 3 commits intoAug 13, 2026
Conversation
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
randomvariable
added a commit
to randomvariable/vllm
that referenced
this pull request
Aug 13, 2026
Replaces the XQA SM12x revert (vllm-project#51987) with the proper fix from vllm-project#52148: when a model has attention sinks on SM12x, route prefill through BatchAttentionWithAttentionSinkWrapper instead of the plain BatchPrefillWithPagedKVCacheWrapper. The original XQA decode path (vllm-project#49718) is restored.
…iguration Signed-off-by: Andrii Skliar <askliar@nvidia.com>
askliar
marked this pull request as ready for review
August 13, 2026 12:38
askliar
requested review from
mgoin,
pavanimajety and
vadiklyutiy
as code owners
August 13, 2026 12:38
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
LucasWilkinson
enabled auto-merge (squash)
August 13, 2026 13:44
github-actions
Bot
added
the
ready
mgoin
approved these changes
Aug 13, 2026
Member
|
/ci run |
|
✅ Triggered Buildkite CI #83752 for commit |
Contributor
|
Validated on DGX Spark (GB10, sm121), same GPU and job, only the patch or backend flag changed.
Non-sink controls, no regression.
Nemotron 3.5 Lightning NVFP4 with a DSpark-3 drafter (hybrid mamba plus attention), 200 items:
|
Member
|
dgx spark test passed, merging https://buildkite.com/vllm/ci/builds/83752#019ffbb2-b776-476f-bdd8-128988ec7298 |
Alessandra005
pushed a commit
to Alessandra005/vllm
that referenced
this pull request
Aug 17, 2026
Signed-off-by: Andrii Skliar <askliar@nvidia.com> Co-authored-by: Andrii Skliar <askliar@nvidia.com> Co-authored-by: Michael Goin <mgoin64@gmail.com> Signed-off-by: Alessandra005 <aurib032@fiu.edu>
16 tasks
zyp2014
pushed a commit
to zyp2014/vllm
that referenced
this pull request
Aug 21, 2026
Signed-off-by: Andrii Skliar <askliar@nvidia.com> Co-authored-by: Andrii Skliar <askliar@nvidia.com> Co-authored-by: Michael Goin <mgoin64@gmail.com>
wyettzeng
pushed a commit
to wyettzeng/vllm
that referenced
this pull request
Aug 21, 2026
Signed-off-by: Andrii Skliar <askliar@nvidia.com> Co-authored-by: Andrii Skliar <askliar@nvidia.com> Co-authored-by: Michael Goin <mgoin64@gmail.com> Signed-off-by: Wyett <wyettzeng@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Use FlashInfer's sink-aware paged prefill wrapper on SM12x when attention sinks are enabled. The generic FA2 prefill path accepts a
sinksargument but does not apply it, so #49718 can use XQA for decode while producing incorrect prefill output.The wrapper is specialized with the active dtypes, head dimensions, sliding window, and softmax scale. DCP, NVFP4, SM90/SM100, and sink-free paths are unchanged.
This is not a duplicate of the automatic revert #51987: that PR removes SM12x XQA support, while this change preserves XQA and fixes the prefill path for sink-attention models on SM12x .
Validation
pytest tests/v1/attention/test_attention_backends.py::test_flashinfer_native_prefill_with_sinks -qpassing