Return empty sparsity config if targets and ignores are empty (#1115)

This PR fixes an issue where a sparsity configuration could end up being empty under certain conditions. Specifically, if the global sparsity is greater than 0.05, but no individual layer has a sparsity greater than 0.5, we end up with an empty sparsity config. To address this, we now ensure that an empty sparsity config is not added in such cases --- - To see the specific tasks where the Asana app for GitHub is being used, see below: - https://app.asana.com/0/0/1209272443107638 Signed-off-by: Rahul Tuli <[email protected]> Co-authored-by: Dipika Sikka <[email protected]>
vllm-project · Jan 29, 2025 · 8de50a2 · 8de50a2
1 parent 8764291
commit 8de50a2
Showing 1 changed file with 5 additions and 0 deletions.
diff --git a/src/llmcompressor/transformers/compression/sparsity_config.py b/src/llmcompressor/transformers/compression/sparsity_config.py
@@ -130,6 +130,11 @@ def from_pretrained(
             sparsity_threshold=SparsityConfigMetadata.SPARSITY_THRESHOLD,
         )
 
+        if not (targets or ignores):
+            # no sparsity config
+            # needed if targets/ignores are empty
+            return None
+
         return SparsityCompressionConfig.load_from_registry(
             format,
             global_sparsity=global_sparsity,