[Bugfix] Declare SupportsEagle3 on KimiLinearForCausalLM - #52171
Merged
Conversation
The multimodal KimiK3ForConditionalGeneration declares SupportsEagle3, but the
text-only KimiLinearForCausalLM does not - even though both serve the same
inner KimiLinearModel, which already inherits EagleModelMixin and implements
the aux-hidden-state tap machinery. As a result, serving a text-only Kimi-K3
checkpoint with EAGLE3-family speculative decoding (e.g. dspark) is rejected
at startup:
RuntimeError: Model does not support EAGLE3 interface
Adding the interface to the class bases is sufficient: the protocol's default
set_aux_hidden_state_layers / get_eagle3_default_aux_hidden_state_layers
delegate to self.model, which satisfies the EagleModelMixin requirement.
Verified by serving a text-only Kimi-K3 checkpoint with a dspark draft on
8x RTX 3090 with an equivalent runtime patch.
Assisted-By: Claude
Signed-off-by: Nick Iusiumbeli <nickuspro@gmail.com>
nickus
requested review from
AndreasKaratzas,
DarkLight1337 and
ywang96
as code owners
August 13, 2026 13:55
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
DarkLight1337
approved these changes
Aug 13, 2026
DarkLight1337
added
the
verified
DarkLight1337
enabled auto-merge (squash)
August 13, 2026 14:06
Member
|
/ci run |
github-actions
Bot
added
the
ready
|
✅ Triggered Buildkite CI #83739 for commit |
Collaborator
|
How can 8× RTX 3090 serve kimi k3? cc @DarkLight1337 |
Member
|
I don't think you got enough VRAM for that |
Alessandra005
pushed a commit
to Alessandra005/vllm
that referenced
this pull request
Aug 17, 2026
…t#52171) Signed-off-by: Nick Iusiumbeli <nickuspro@gmail.com> Signed-off-by: Alessandra005 <aurib032@fiu.edu>
zyp2014
pushed a commit
to zyp2014/vllm
that referenced
this pull request
Aug 21, 2026
…t#52171) Signed-off-by: Nick Iusiumbeli <nickuspro@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
KimiK3ForConditionalGeneration(multimodal) declaresSupportsEagle3; the text-onlyKimiLinearForCausalLMdoes not — even though both serve the same innerKimiLinearModel, which already inheritsEagleModelMixinand implements the aux-hidden-state tap machinery. Serving a text-only Kimi-K3 checkpoint with EAGLE3-family speculative decoding (e.g.dspark) therefore dies at startup:Adding the interface to the class bases is the whole fix: the protocol's default
set_aux_hidden_state_layers/get_eagle3_default_aux_hidden_state_layersdelegate toself.model, which satisfies theirEagleModelMixinrequirement.Test plan
tests/models/kimi_k3/test_eagle3.pygainstest_kimi_linear_advertises_eagle3_support, mirroring the existing multimodal assertion; it fails onmainand passes with this change.dsparkdraft on 8× RTX 3090 using an equivalent patch (that is how we hit the error).Assisted-By: Claude