Extreme compression of deep neural networks, up to full binarization, dramatically reduces memory footprint and arithmetic complexity, facilitating deployment on constrained edge hardware with field-programmable gate arrays (FPGAs) and microcontrollers.
Although combining binarization with pruning promises additional efficiency gains, existing pruning strategies are ill-suited to binarized representations and rarely translate into meaningful hardware savings.
We introduce a PyTorch-based, research-oriented framework
We introduce a PyTorch-based, research-oriented framework that incorporates freezing and pruning mechanisms for designing and optimizing binarized neural networks.
The framework enables rapid and reproducible evaluation of state-of-the-art approaches and the fast prototyping of new ones.
Leveraging this framework
Leveraging this framework, we propose a novel pruning method that accounts for the relative importance of learned parameters across abstraction levels.
Such a global weighting mechanism consistently achieves a superior trade-off between model accuracy and pruning rate, achieving a 70% pruning rate on VGG11 with constant accuracy, while state-of-the-art results reach only 41% in the binarized setting.