Your current environment
0.27.1
env - doesn't matter for this
🐛 Describe the bug
Loading a checkpoint whose scale tensor has the wrong shape fails with a bare AssertionError — no parameter name, no shapes, no hint what to fix.
File "vllm/model_executor/parameter.py", line 94, in _assert_and_load
assert self.data.shape == loaded_weight.shape or self._is_1d_and_scalar(
AssertionError
Repro: compressed-tensors FP8, strategy: channel, weight_scale written as [out] instead of [out, 1]. The checkpoint is at fault, but the error says nothing about which parameter or which shapes collide. It took a debugger to find; the message below would have made it a one-line fix.
vLLM reports the analogous case well elsewhere — a missing parameter raises ValueError: There is no module or parameter named 'fc.weight_scale' in ... and lists the available names.
Second concern: assert is stripped under python -O. self.data.copy_(loaded_weight) then runs unchecked, and copy_ broadcasts — a 1-element per-tensor scale would be silently spread across all channels instead of rejected.
Patch (no signature change; the parameter does not carry its name, so shapes only):
def _assert_and_load(self, loaded_weight: torch.Tensor):
- assert self.data.shape == loaded_weight.shape or self._is_1d_and_scalar(
- loaded_weight
- )
+ if self.data.shape != loaded_weight.shape and not self._is_1d_and_scalar(
+ loaded_weight
+ ):
+ raise ValueError(
+ f"weight shape mismatch: parameter has {tuple(self.data.shape)}, "
+ f"checkpoint has {tuple(loaded_weight.shape)}"
+ )
self.data.copy_(loaded_weight)
Before submitting a new issue...
Your current environment
0.27.1
env - doesn't matter for this
🐛 Describe the bug
Loading a checkpoint whose scale tensor has the wrong shape fails with a bare AssertionError — no parameter name, no shapes, no hint what to fix.
File "vllm/model_executor/parameter.py", line 94, in _assert_and_load
assert self.data.shape == loaded_weight.shape or self._is_1d_and_scalar(
AssertionError
Repro: compressed-tensors FP8, strategy: channel, weight_scale written as [out] instead of [out, 1]. The checkpoint is at fault, but the error says nothing about which parameter or which shapes collide. It took a debugger to find; the message below would have made it a one-line fix.
vLLM reports the analogous case well elsewhere — a missing parameter raises ValueError: There is no module or parameter named 'fc.weight_scale' in ... and lists the available names.
Second concern: assert is stripped under python -O. self.data.copy_(loaded_weight) then runs unchecked, and copy_ broadcasts — a 1-element per-tensor scale would be silently spread across all channels instead of rejected.
Patch (no signature change; the parameter does not carry its name, so shapes only):
Before submitting a new issue...