Cannot load quantized Multimodal_audio model using whisper.load_model("quantized-model-path)

[Bakht-Ullah]

I quantized whisper medium model using the GPTQ method and saved it. When I want to load the model it through the following error.

RuntimeError: Error(s) in loading state_dict for Whisper:
Missing key(s) in state_dict: "encoder.blocks.0.attn.query.weight", "encoder.blocks.0.attn.key.weight", "encoder.blocks.0.attn.value.weight", "encoder.blocks.0.attn.out.weight", "encoder.blocks.0.mlp.0.weight", "encoder.blocks.0.mlp.2.weight", "encoder.blocks.1.attn.query.weight", "encoder.blocks.1.attn.key.weight", "encoder.blocks.1.attn.value.weight", "encoder.blocks.1.attn.out.weight", "encoder.blocks.1.mlp.0.weight", "encoder.blocks.1.mlp.2.weight", "encoder.blocks.2.attn.query.weight", "encoder.blocks.2.attn.key.weight", "encoder.blocks.2.attn.value.weight", "encoder.blocks.2.attn.out.weight", "encoder.blocks.2.mlp.0.weight", "encoder.blocks.2.mlp.2.weight", "encoder.blocks.3.attn.query.weight", "encoder.blocks.3.attn.key.weight", "encoder.blocks.3.attn.value.weight", "encoder.blocks.3.attn.out.weight", "encoder.blocks.3.mlp.0.weight", "encoder.blocks.3.mlp.2.weight", "encoder.blocks.4.attn.query.weight", "encoder.blocks.4.attn.key.weight", "encoder.blocks.4.attn.value.weight", "encoder.blocks.4.attn.out.weight", "encoder.blocks.4.mlp.0.weight", "encoder.blocks.4.mlp.2.weight", "encoder.blocks.5.attn.query.weight", "encoder.blocks.5.attn.key.weight", "encoder.blocks.5.attn.value.weight", "encoder.blocks.5.attn.out.weight", "encoder.blocks.5.mlp.0.weight", "encoder.blocks.5.mlp.2.weight", "encoder.blocks.6.attn.query.weight", "encoder.blocks.6.attn.key.weight", "encoder.blocks.6.attn.value.weight", "encoder.blocks.6.attn.out.weight", "encoder.blocks.6.mlp.0.weight", "...
Unexpected key(s) in state_dict: "encoder.blocks.0.attn.query.weight_scale", "encoder.blocks.0.attn.query.weight_packed", "encoder.blocks.0.attn.query.weight_shape", "encoder.blocks.0.attn.key.weight_scale", "encoder.blocks.0.attn.key.weight_packed", "encoder.blocks.0.attn.key.weight_shape", "encoder.blocks.0.attn.value.weight_scale", "encoder.blocks.0.attn.value.weight_packed", "encoder.blocks.0.attn.value.weight_shape", "encoder.blocks.0.attn.out.weight_scale", "encoder.blocks.0.attn.out.weight_packed", "encoder.blocks.0.attn.out.weight_shape", "encoder.blocks.0.mlp.0.weight_scale", "encoder.blocks.0.mlp.0.weight_packed", "encoder.blocks.0.mlp.0.weight_shape", "encoder.blocks.0.mlp.2.weight_scale", "encoder.blocks.0.mlp.2.weight_packed", "encoder.blocks.0.mlp.2.weight_shape", "encoder.blocks.1.attn.query.weight_scale", "encoder.blocks.1.attn.query.weight_packed", "encoder.blocks.1.attn.query.weight_shape", "encoder.blocks.1.attn.key.weight_scale", "encoder.blocks.1.attn.key.weight_packed", "encoder.blocks.1.attn.key.weight_shape", "encoder.blocks.1.attn.value.weight_scale", "encoder.blocks.1.attn.value.weight_packed", "encoder.blocks.1.attn.value.weight_shape", "encoder.blocks.1.attn.out.weight_scale", "encoder.blocks.1.attn.out.weight_packed", "encoder.blocks.1.attn.out.weight_shape", "encoder.blocks.1.mlp.0.weight_scale", "encoder.blocks.1.mlp.0.weight_packed", "encoder.blocks.1.mlp.0.weight_shape", "encoder.blocks.1.mlp.2.weight_scale", "encoder.blocks.1.mlp.2.weight...

[brian-dellabetta]

@rahul-tuli is this the error you resolved with this PR into huggingface transformers? If so, @Bakht-Ullah I think you'll have to install transformers from source until a new version is released