[ModelRunner V2] Speculative Decoding NGram GPU Implementations - #40704

Open

PatchouliTIS wants to merge 79 commits into

vllm-project:mainfrom

PatchouliTIS:patchy/async_ngram_v2_pr

Commits on Apr 20, 2026

async ngram gpu debug
PatchouliTaisa
committed
58aa569

Commits on Apr 21, 2026

complete async ngram v2
PatchouliTaisa
committed
4bfb745
remove overcomments
PatchouliTaisa
committed
02185ac

Commits on Apr 23, 2026

set minimal values in FP32
PatchouliTaisa
committed
242fb82
Merge branch 'main' of github.com:vllm-project/vllm into patchy/async_ngram_v2_pr
PatchouliTaisa
committed
8569425
remove debug
PatchouliTaisa
committed
b4f1db9

Commits on Apr 24, 2026

reorg codes
PatchouliTaisa
committed
f22224a
Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
3437791

Commits on Apr 25, 2026

Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
324fe81

Commits on Apr 28, 2026

Merge branch 'main' of github.com:vllm-project/vllm into patchy/async_ngram_v2_pr
PatchouliTaisa
committed
95dffa8
fix argmax
PatchouliTaisa
committed
3af6547
Merge branch 'patchy/async_ngram_v2_pr' of https://github.com/PatchouliTIS/vllm into patchy/async_ngram_v2_pr
PatchouliTaisa
committed
2527013

Commits on May 6, 2026

fix merge conflicts and revert changes in gumbel
PatchouliTaisa
committed
199b525
Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
480614b

Commits on May 9, 2026

Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
81a1915

Commits on May 10, 2026

Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
7d5deb7

Commits on May 11, 2026

Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
00b071a
test added for ngram gpu
PatchouliTaisa
committed
a285ef6

Commits on May 12, 2026

fix return value
PatchouliTaisa
committed
fcdab55

Commits on May 13, 2026

add tests in yaml and change return values into tuple
PatchouliTaisa
committed
3374d6f
add tests in yaml and change return values into tuple
PatchouliTaisa
committed
34fa287
merge conflicts
PatchouliTaisa
committed
b0eb7ad
Merge branch 'main' of github.com:vllm-project/vllm into patchy/async_ngram_v2_pr
PatchouliTaisa
committed
aabb68f
fix test error
PatchouliTaisa
committed
a9101c7

Commits on May 14, 2026

inputs params fixed
PatchouliTaisa
committed
5986d2b
fix(test): Add required InitVar fields to SchedulerConfig in ngram_gpu test helper
PatchouliTaisa
committed
07641bc
Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
b906e89
Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
29b8a6c

Commits on May 15, 2026

merge conflicts fixed
PatchouliTaisa
committed
a880d05
Merge branch 'patchy/async_ngram_v2_pr' of github.com:PatchouliTIS/vllm into patchy/async_ngram_v2_pr
PatchouliTaisa
committed
7a7ea14
modify vllm config to pass ngram_gpu
PatchouliTaisa
committed
29cb65e
fix method signature
PatchouliTaisa
committed
98cc7da

Commits on May 16, 2026

Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
9d6fd07

Commits on May 18, 2026

Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
4824b4a

Commits on May 19, 2026

Merge branch 'main' of github.com:vllm-project/vllm into patchy/async_ngram_v2_pr
PatchouliTaisa
committed
430e18a
format fixed
PatchouliTaisa
committed
445322d

Commits on May 25, 2026

Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
1474690

Commits on May 26, 2026

fix gpu utils drop bug
PatchouliTaisa
committed
98a0809
Merge branch 'main' of github.com:vllm-project/vllm into patchy/async_ngram_v2_pr
PatchouliTaisa
committed
4ea6d81

Commits on Jun 2, 2026

Commits on Jun 3, 2026

merge conflicts fixed
PatchouliTaisa
committed
37cc9c5
Merge branch 'patchy/async_ngram_v2_pr' of https://github.com/PatchouliTIS/vllm into patchy/async_ngram_v2_pr
PatchouliTaisa
committed
cb77a14
duplicated codes removed
PatchouliTaisa
committed
5228bd6

Commits on Jun 4, 2026

merge conflicts resolved
PatchouliTaisa
committed
eb4dc4e
Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
2d0bab6
triton kernel for ngram gpu
PatchouliTaisa
committed
af498ea
Merge branch 'patchy/async_ngram_v2_pr' of https://github.com/PatchouliTIS/vllm into patchy/async_ngram_v2_pr
PatchouliTaisa
committed
e66fbb8

Commits on Jun 11, 2026

return values fixed
PatchouliTaisa
committed
6dd58ad
fix merge conflicts
PatchouliTaisa
committed
2754fe3
Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
709e437
config bug fixed
PatchouliTaisa
committed
3e20869
Merge branch 'patchy/async_ngram_v2_pr' of https://github.com/PatchouliTIS/vllm into patchy/async_ngram_v2_pr
PatchouliTaisa
committed
172b2bd

Commits on Jun 12, 2026

return type fixed
PatchouliTaisa
committed
4033877
Merge branch 'main' of github.com:vllm-project/vllm into patchy/async_ngram_v2_pr
PatchouliTaisa
committed
52e79a8

Commits on Jun 15, 2026

merge conflicts fixed
PatchouliTaisa
committed
2d5f8f9
Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
4702930

Commits on Jun 28, 2026

Merge branch 'main' of github.com:vllm-project/vllm into patchy/async_ngram_v2_pr
PatchouliTaisa
committed
028e77a

Commits on Jul 2, 2026

merge conflicts fixed
PatchouliTaisa
committed
6097cba

Commits on Jul 3, 2026

merge conflicts fixed
PatchouliTaisa
committed
e9805e6

Commits on Jul 7, 2026

Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
80205e7

Commits on Aug 1, 2026

merge conflicts fixed
PatchouliTIS
committed
ad4bd48
test vllm_runner updated
PatchouliTIS
committed
66e1ade

Commits on Aug 3, 2026

Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
919a826

Commits on Aug 7, 2026

resolve merge conflicts
PatchouliTIS
committed
4ae1b28

Commits on Aug 13, 2026

Commits on Aug 15, 2026

Commits on Aug 17, 2026

Merge remote-tracking branch 'origin/main' into patchy/async_ngram_v2_pr
njhill
committed
ee72c7a

Commits on Aug 18, 2026

Commits on Aug 19, 2026

Merge remote-tracking branch 'origin/main' into patchy/async_ngram_v2_pr
njhill
committed
7e5703a

Commits on Aug 20, 2026

Merge branch 'main' into patchy/async_ngram_v2_pr
PatchouliTIS
authored
ffeadf6