Kỹ thuật nén thông tin trong mạng học sâu

Despite the great success of deep learning in a wide range of fields, it usually requires a huge amount of parameters to be trained and thereby demands intensive computation, storage, and energy resources. The situation gets even worse for model inference on resource-limited mobile devices. Model compression is an important category of methods to enable on-device inference. As a well-known approach for model compression, network pruning has shown its huge potential in reducing the storage and energy consumption of deep learning by sparsifying deep neural network parameters. In this chapter, we provide a brief introduction on model compression techniques and present a layerwise network pruning method as a representative example. Specifically, we present a nonconvex and nonsmooth log-sum minimization approach to improve the network compression ratio, followed by developing an iteratively reweighted minimization algorithm. Simulation results demonstrate its advantage over convex methods.