Physics-based pruning beats heuristics for LLM compression

Multiverse AI has published research demonstrating that treating LLM block removal as a constrained binary optimization problem significantly outperforms traditional heuristics. By modeling transformer blocks as spins in an Ising glass, the method accounts for pairwise interactions between layers using a Hessian matrix derived from a small calibration dataset. In tests on Llama-3.3-70B-Instruct, this physics-based approach achieved nearly 23 percentage points higher MMLU scores than competing block-removal methods at 50% compression. The technique leverages classical and quantum-inspired solvers to find optimal pruning configurations without requiring expensive retraining for every candidate.