fix: surface clear error for missing parameters in LinearBase weight loader - #53118
fix: surface clear error for missing parameters in LinearBase weight loader#53118Tejas-Raj01 wants to merge 1 commit into
Conversation
…loader Signed-off-by: Tejas-Raj01 <rajtejas.xyz@gmail.com>
|
I had the same fix sitting in my fork (found while debugging GPTQ MoE loading on Intel XPU) and am closing it rather than opening a competing PR — yours resolves #53107 and got here first. Two things from my side that may be useful. A real checkpoint that reproduces it, if you want something beyond the dummy-layer test: Its That is arguably a second bug worth its own fix — the draft model silently loses its quantization — but your change is what turns it from an opaque One difference in approach, take it or leave it. I removed the _OPTIONAL_CHECKPOINT_TENSORS = frozenset(("bias",))The reason I went that way: I also have 129 lines of tests for these two call sites that ran on real hardware (Intel B70 and NVIDIA H200, 3 failed → 5 passed on both, identical split, which is what convinced me |
|
Hi @afierka-intel, thanks for stopping by and sharing your findings! That explanation regarding the top-level Your point about Also, regarding the failing |
Resolves #53107
Purpose
When
LinearBase.load_weightsencounters an unexpected tensor in a checkpoint (such as a quantization scale on an unquantized layer), the parameter lookup falls back toself(the module itself). This previously allowed the loading loop to proceed until it crashed several frames later inside the weight loader with a confusingAttributeError: 'MergedColumnParallelLinear' object has no attribute 'data'.This PR fixes the issue by explicitly verifying that the resolved
paramis atorch.nn.Parameter. If it is not, the loader now intercepts it and raises a clear, descriptiveValueErrorindicating exactly which parameter is missing and what type was found instead.Test Plan
tests/model_executor/layers/test_linear_missing_param.pythat utilizes a dummy layer and binds the vLLMload_weightsmethod to safely test the loading loop without distributed GPU state.pytest tests/model_executor/layers/test_linear_missing_param.py -vto verify the new error handling.Test Result
ValueError: dummy_layer: cannot load 'unexpected_scale' — no such parameter, got DummyLayer instead...) rather than a crypticAttributeError.Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.