Neural networks are now widely deployed, but models are often massive and over-parameterized, which contributes to increasing their energy consumption and training costs. For example, GPT-4 has 1.76 trillion parameters and the energy consumption associated with using GPT-4o is estimated at around 391 GWh in 2025. Furthermore, during the inference (usage) phase, only a fraction of the parameters are actually used during calculations. In this context,frugal AI can be a way to “do more with less”: reducing infrastructure costs, lowering energy consumption, and decreasing latency, which is ideal for embedded systems.
Pruning applied to neural networks is an algorithmic method that involves removing parameters deemed unimportant (weights, neurons, etc.) from a model to make it lighter without significantly altering its performance. Pruning a model can help reduce its inference time and energy consumption. It is therefore suitable for real-time applications and facilitates the deployment of complex models on embedded devices.
Depending on the parameters removed, there are two types of pruning strategies, each with their own specificities :
In the case of unstructured pruning, individual connection weights or even neurons are removed, regardless of their position in the model. Unstructured pruning allows for high-granularity pruning and precise optimization of the model architecture. Unstructured pruning adapts to various network architectures and can achieve high compression rates. Unstructured pruning creates zeros that are scattered in an irregular and disordered way within the weights, which requires specialized hardware to process the calculations; otherwise, this pruning will not automatically shorten inference time.
With structured pruning, entire structural elements of the model are removed, such as layers of neurons. This method effectively reduces size while maintaining an orderly network structure. Structured pruning is also compatible with standard computing hardware. It offers more predictable and consistent compression rates because it targets entire components. Compression rates are generally lower than those allowed for unstructured pruning, and the granularity is also coarser.
Various methods exist to determine which parameters to remove: magnitude-based pruning, gradient-based pruning…
Figure: Difference between structured and unstructured pruning (Zenon)
When and how often should pruning be used?
The pruning process can be performed during a model's training phase or post-training. Lightening the model while it is being trained allows its parameters to be adapted gradually without significantly altering its performance. However, its operational complexity is higher and can increase training time. Post-training pruning is a simpler process to implement and is compatible with already trained models, but it can lead to a greater loss in performance. In both cases, pruning can be applied in a single pass (one-shot pruning) or iteratively, by alternating several cycles of removal and retraining to gradually reach the desired level of compression. Pruning can also take place during model inference : certain weights are ignored during inference depending on the input received, without being permanently removed. Dynamic pruning therefore improves execution efficiency by reducing the number of calculations performed and data transfers, but has no effect on the model's memory footprint.
The entire challenge of pruning lies in finding a balance between model compression and maintaining a certain level of performance. Finally, beyond pruning, other algorithmic methods to make AI frugal or more efficient exist and can be utilized: quantization, distillation, fine-tuning.