LLM-Barber: Block-Aware Rebuilder for Sparsity Mask in One-Shot for Large Language Models

Yupeng Su; Ziyi Guan; Xiaoqun Liu; Tianlai Jin; Dongkuan Wu; Zhengfei Chen; Graziano Chesi; Ngai Wong; Hao Yu

doi:10.1109/ICCAD66269.2025.11240704

Method

LLM-Barber starts from any pruned model and rebuilds the mask of every Self-Attention and MLP block in one pass. The weights keep their values and the sparsity stays the same.

Kept
Pruned
Score
Grown back
Trimmed

Starting mask

Swap ratio

\alpha

50%

More options

Sparsity

Pairs formed per

Score

Block error against $\boldsymbol{\alpha}$

Too many swaps hurt again

Gain of each candidate pair

A few pairs carry most of the gain

A toy SwiGLU MLP block, computed live in your browser with the rules of the released code. The three steps below take it apart.

Measure the error at the block

Layer-wise pruners fit each projection to its own output, $\|\mat{W}\mat{X} - (\mat{W} \odot \mat{M})\mat{X}\|_2^2$ , and the small biases add up from layer to layer. LLM-Barber takes one error at the output of the whole block, so the masks of all its projections are judged together:

\class{c-e}{\mat{E}} = \|\mathrm{Block}(\mat{W}, \mat{X}) - \mathrm{Block}(\mat{W} \odot \mat{M}, \mat{X})\|_2^2

Error measured per

The forward pass reaches each block's output, and the gradient of

\mat{E}

flows back to all of its projections, whose masks are then rebuilt. Each rebuilt block feeds the next: one forward pass for the model and one local backward pass per block.

Score every weight with its gradient

At the dense model $\mat{E}$ is zero and its gradient vanishes, which is why methods such as SparseGPT turn to the Hessian. After pruning $\mat{E}$ is no longer zero, so the first-order term is enough. Pruning or regrowing a weight changes it by $|\Delta\mat{W}_{ij}| = |\mat{W}_{ij}|$ , which gives the score:

\Delta\class{c-e}{\mat{E}} \approx \frac{\partial \class{c-e}{\mat{E}}}{\partial \mat{W}} \cdot \Delta\mat{W}

\Rightarrow\quad \class{c-s}{\mat{S}}_{ij} = |\class{c-w}{\mat{W}}_{ij}| \cdot \left|\frac{\partial \class{c-e}{\mat{E}}}{\partial \mat{W}_{ij}}\right|

Weight $\mat{W}$

Gradient $|\partial\mat{E}/\partial\mat{W}|$

Score $\mat{S}$

$\mat{W}_{11} = 1$ is one of the smallest weights, but its gradient is the largest, so it scores highest. $\mat{W}_{22} = 11$ , one of the two largest weights, scores 1.1. Point at a cell to read it.

The worked example of Figure 2b in the paper.

Swap weights in pairs

Pruned weights are sorted by $\mat{S}$ from high to low and kept weights from low to high, then paired in order. Each swap regrows one weight and prunes another, so the sparsity never changes. Only a fraction $\alpha$ of the pairs with a positive gain is swapped:

N = \alpha \cdot |\{ i : \class{c-s}{\mat{S}}^{\text{grow}}_i - \class{c-s}{\mat{S}}^{\text{prune}}_i > 0 \}|

Pairs are formed within a group: an output row, an input column, a matrix or the whole block. The best $\alpha$ sits where the outlier pairs end, about 10% after Magnitude and 1% after Wanda on LLaMA-7B.

Starting mask $\mat{M}_{\mathbf{i}}$

Grow pruned weights, highest score first

Prune kept weights, lowest score first

Rebuilt mask $\mat{M}_{\mathbf{r}}$

Swap ratio

\alpha

The same example. Each column is one pair: the weight to grow on top, the weight to prune below, the gain between them. At

\alpha = 50\%

the two largest gains are swapped, as in the paper.

Results

LLaMA and OPT models from 6.7B to 13B parameters, pruned to 50% by Magnitude, SparseGPT or Wanda and then rebuilt, with 128 segments of 2048 tokens from C4 for calibration.

WikiText-2 perplexity

Every starting mask gets better

205.5→10.98LLaMA3-8B, from a magnitude mask

7of 7models improve from every starting mask

≈ 30minfor a 7B to 13B model on one A100

Starting mask

Sparsity pattern

One pass

One pass beats iterative regrowth

DSnoT also rebuilds masks without training, but prunes and regrows weights over many cycles with a fixed iteration limit, layer by layer. LLM-Barber rebuilds each block once and recovers far more from a poor start.

LLaMA3-8B at 50% unstructured sparsity, WikiText-2 perplexity (Table V of the paper). The dashed line is the dense model.

Zero-shot

Accuracy follows the perplexity

Mean accuracy over BoolQ, RTE, HellaSwag, ARC-e, ARC-c and OpenBookQA. From a magnitude mask, LLaMA3-8B goes from 36.57 to 52.58; the dense model has 62.79.

Mean zero-shot accuracy at 50% unstructured sparsity (Table IV of the paper). The dark tick is the dense model.

More results

Choosing $\boldsymbol{\alpha}$ where the outliers end

Top: the gain of each candidate pair, largest first. Bottom: WikiText-2 perplexity of LLaMA-7B against $\alpha$ , for four ways of grouping the pairs. The dashed line marks the best $\alpha$ , which follows where the outlier pairs end: 10% after Magnitude, 1% after Wanda.

Figure 3 of the paper: importance score of the mask rebuilding pairs and WikiText-2 perplexity against the mask rebuilding ratio for Magnitude, SparseGPT and Wanda starting masks on LLaMA-7B.

Higher sparsity, LLaMA3-8B

Starting mask
With LLM-Barber

Zero-shot accuracy at 50% sparsity

Bold marks the best pruned result per task.

One pass against DSnoT, LLaMA3-8B

Grouping of pairs, WikiText-2 perplexity

Score used for swaps, WikiText-2 perplexity

With APTQ on FPGA (EdgeLLM, Qwen2-7B)

With APTQ on GPU, tokens/s

Code

The repository rebuilds masks from Magnitude, SparseGPT or Wanda on LLaMA and OPT models, and has the scripts for every experiment in the paper.

Set up

conda create -n barber python=3.10
conda activate barber
pip3 install torch torchvision torchaudio
pip3 install -r requirements.txt

Rebuild a Wanda mask on LLaMA-7B

python main.py --model huggyllama/llama-7b \
  --prune_method wanda \
  --sparsity_ratio 0.5 --sparsity_type unstructured \
  --prune_barber --prune_granularity output1 --threshold 0.01 \
  --save_ppl out/ppl

--threshold is the swap ratio $\alpha$ (the paper's mask rebuilding ratio). Commands for every experiment are in script/.

Citation

BibTeX

@inproceedings{su2025llmbarber,
  title     = {{LLM-Barber}: Block-Aware Rebuilder for Sparsity Mask
               in One-Shot for Large Language Models},
  author    = {Su, Yupeng and Guan, Ziyi and Liu, Xiaoqun and Jin, Tianlai and
               Wu, Dongkuan and Chen, Zhengfei and Chesi, Graziano and
               Wong, Ngai and Yu, Hao},
  booktitle = {2025 IEEE/ACM International Conference On Computer Aided Design (ICCAD)},
  pages     = {1--9},
  year      = {2025},
  doi       = {10.1109/ICCAD66269.2025.11240704}
}