Replace LayerCompressor with HooksMixin

[kylesayrs]

Purpose

• Remove layer compressor to decouple modifiers from data pipelines
• Reduce abstractions
• Support VLMs with SparseGPT and Wanda

Prerequisites

• Extend remove_hooks to remove subsets #1021
• Extend disable_hooks to keep subsets #1023
• GPTQModifier Nits and Code Clarity #1068
• VLM: Model Tracing Guide #1030

Changes

Interface/ Features

• SparseGPT and Wanda now both support VLM architectures
• Added sequential_targets to match GPTQ and made targets an alias
• Support hessian offloading for SparseGPT
• Add customized _LinAlgError for SparseGPT

Implementations

• Changed implementation styles of SparseGPTModifier and WandaPruningModifier to match GPTQModifier
• Removed LayerCompressor, ModuleCompressionWrapper, SparseGptWrapper, and WandaWrapper
• Shared implementations between SparseGPT and Wanda are implemented by the SparsityModifierMixin
• Removed lines blocking allow_tf32
  • Maybe @rahul-tuli knows why this was originally implemented, potentially to avoid hardware issues?
  • This change was only present for wanda. Given that all other modifiers do not have this change, I see no reason why it should stay
• Updated sparsegpt tests to reflect new implementation

Tests

• Updated obcq tests to reflect new implementations
• Removed test_sgpt_defaults.py since this test doesn't test anything new or novel about this modifier

Testing

• grep -r "LayerCompressor\|ModuleCompressionWrapper\|SparseGptWrapper\|WandaWrapper" src/ examples/ tests/
• Modified test_invalid_layerwise_recipes_raise_exceptions and test_successful_layerwise_recipe pass
• llama3_8b_2of4.py passes and was evaluated with both SparseGPT and Wanda

Potential Follow ups

• Add module targets and ignore to SparseGPT and Wanda

Regression Testing

The hessian, row scalar, and compressed weight values were confirmed to be unchanged in the case that of one calibration sample. The final evaluations are different, which is likely due to numerical imprecision (dividing by int vs torch.int), different pipelines (different subgraph partitions => different imprecision from cpu offloading, potentially different module arguments).

Evaluation

Models were compressed using examples/sparse_2of4_quantization_fp8/llama3_8b_2of4.py

sparsegpt

Main

hf (pretrained=/home/ksayers/llm-compressor/old_Llama-3.2-1B-Instruct2of4-sparse,dtype=bfloat16,add_bos_token=True), gen_kwargs: (None), limit: None, num_fewshot: 5, batch_size: 1                                                           
|  Tasks   |Version|Filter|n-shot|Metric|   |Value |   |Stderr|                                                        
|----------|------:|------|-----:|------|---|-----:|---|-----:|                                                        
|winogrande|      1|none  |     5|acc   |?  |0.5391|?  | 0.014|

Branch

hf (pretrained=/home/ksayers/llm-compressor/new_Llama-3.2-1B-Instruct2of4-sparse,dtype=bfloat16,add_bos_token=True), gen_kwargs: (None), limit: None, num_fewshot: 5, batch_size: 1
|  Tasks   |Version|Filter|n-shot|Metric|   |Value|   |Stderr|
|----------|------:|------|-----:|------|---|----:|---|-----:|
|winogrande|      1|none  |     5|acc   |?  |0.547|?  | 0.014|

To test wanda, the SparseGPTModifier was replaced with the WandaPruningModifier

wanda

Main

hf (pretrained=/home/kyle/old_llm-compressor/Llama-3.2-1B-Instruct2of4-sparse,dtype=bfloat16,add_bos_token=True), gen_kwargs: (None), limit: None, num_fewshot: 5, batch_size: 1
|  Tasks   |Version|Filter|n-shot|Metric|   |Value|   |Stderr|
|----------|------:|------|-----:|------|---|----:|---|-----:|
|winogrande|      1|none  |     5|acc   |↑  |0.532|±  | 0.014|

Branch

hf (pretrained=/home/kyle/llm-compressor/Llama-3.2-1B-Instruct2of4-sparse,dtype=bfloat16,add_bos_token=True), gen_kwargs: (None), limit: None, num_fewshot: 5, batch_size: 1
|  Tasks   |Version|Filter|n-shot|Metric|   |Value |   |Stderr|
|----------|------:|------|-----:|------|---|-----:|---|-----:|
|winogrande|      1|none  |     5|acc   |↑  |0.5414|±  | 0.014|

[github-actions]

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

[kylesayrs]

Looks like adding this change fixed the test

weights = torch.rand(10, 4)
    if is_24:
        weights = _make_24_sparse(weights)
    else:
        weights[0, :] = torch.ones(4, )  # guarantee not 24 sparse

The most likely explanation is that this test randomly fails, and that this PR happened to be unlucky

[dsikka]

Purpose

• Remove layer compressor to decouple modifiers from data pipelines
• Reduce abstractions
• Support VLMs with SparseGPT and Wanda

Prerequisites

• Extend remove_hooks to remove subsets #1021
• Extend disable_hooks to keep subsets #1023
• GPTQModifier Nits and Code Clarity #1068
• VLM: Model Tracing Guide #1030

Changes

Interface/ Features

• SparseGPT and Wanda now both support VLM architectures
• Added sequential_targets to match GPTQ and made targets an alias
• Support hessian offloading for SparseGPT
• Add customized _LinAlgError for SparseGPT

Implementations

• Changed implementation styles of SparseGPTModifier and WandaPruningModifier to match GPTQModifier

• Removed LayerCompressor, ModuleCompressionWrapper, SparseGptWrapper, and WandaWrapper

• Shared implementations between SparseGPT and Wanda are implemented by the SparsityModifierMixin

• Removed lines blocking allow_tf32

  • Maybe @rahul-tuli knows why this was originally implemented, potentially to avoid hardware issues?
  • This change was only present for wanda. Given that all other modifiers do not have this change, I see no reason why it should stay

• Updated sparsegpt tests to reflect new implementation

Tests

• Updated obcq tests to reflect new implementations
• Removed test_sgpt_defaults.py since this test doesn't test anything new or novel about this modifier

Testing

• grep -r "LayerCompressor\|ModuleCompressionWrapper\|SparseGptWrapper\|WandaWrapper" src/ examples/ tests/
• Modified test_invalid_layerwise_recipes_raise_exceptions and test_successful_layerwise_recipe pass
• llama3_8b_2of4.py passes and was evaluated with both SparseGPT and Wanda

Potential Follow ups

• Add module targets and ignore to SparseGPT and Wanda

Regression Testing

The hessian, row scalar, and compressed weight values were confirmed to be unchanged in the case that of one calibration sample. The final evaluations are different, which is likely due to numerical imprecision (dividing by int vs torch.int), different pipelines (different subgraph partitions => different imprecision from cpu offloading, potentially different module arguments).

Evaluation

Models were compressed using examples/sparse_2of4_quantization_fp8/llama3_8b_2of4.py

sparsegpt
To test wanda, the SparseGPTModifier was replaced with the WandaPruningModifier

wanda

In terms of regression testing being different, do you mind posting the values you're now seeing?

[dsikka]

This looks good overall. Happy to see the wrapper go.

My only concern is the discrepancy you're regression testing.
Would be good to get a sense of how much variation we're seeing with these changes.

[kylesayrs]

In terms of regression testing being different, do you mind posting the values you're now seeing?

You can see the values by expanding the collapsables in the PR

[kylesayrs]

The test failure on 5fb18d9 seems to be ephemeral, as it wasn't able to be replicated locally and was fixed by the subsequent merge commit

[dsikka]

The test failure on 5fb18d9 seems to be ephemeral, as it wasn't able to be replicated locally and was fixed by the subsequent merge commit

auto-merge is enabled. should merge in once testing is finished