Pruning Neural Networks (Based on Concepts from Statistical Physics such as Free Energy and Entropy), 2025

Project Supervisor: Sergey Koltsov

Participants: Anton Surkov, Ksenia Kupitman, Vera Ignatenko

This project focuses on the research and development of neural network pruning algorithms utilizing functions from statistical physics, such as free energy and entropy. Neural network pruning reduces the number of parameters in a network, thereby decreasing computational load and memory requirements for storage and inference, while maintaining an acceptable level of accuracy. In the first stage, the project explores existing pruning techniques, including unstructured weight magnitude-based pruning and structured pruning algorithms. The pruning process is analyzed by means of free energy. In the second stage, the project aims to develop novel structured pruning algorithms based on entropy or other functions  from statistical physics.