‎

Paper accepted for publication at ICCV workshops

01.10.2023|15:43 Uhr

Our paper: “MGiaD: Multigrid in all dimensions. Efficiency and robustness by weight sharing and coarsening in resolution and channel dimensions” by Antonia van Betteray, Matthias Rottmann and Karsten Kahl has been accepted for publication and poster presentation for the workshop on resource efficient deep learning for computer vision at ICCV23.

Current state-of-the-art deep neural networks for image classification consist of 10-100 million learnable parameters, i.e. weights. Despite their high classification accuracy, these networks are highly overparameterized. The complexity of the number of weights can be considered as a function of the number of channels, the spatial extent of the input, and the number of layers of the network. Due to the use of convolutional layers, the scaling of the weight complexity is usually linear with respect to the resolution dimensions, but remains quadratic with respect to the number of channels. Active research in recent years on the use of multigrid-inspired ideas in deep neural networks has shown that, on the one hand, a significant number of weights can be saved by appropriate weight sharing and, on the other hand, that a hierarchical structure in the channel dimension can improve the weight complexity to linear. Exploiting these findings, we introduce an architecture that establishes multigrid structures in all relevant dimensions, contributing to a drastically improved accuracy-parameter trade-off. Our experiments show that this structured reduction in the number of weights reduces overparameterization and additionally improves performance over state-of-the-art ResNet architectures on typical image classification benchmarks.

The paper is available at: openaccess.thecvf.com/content/ICCV2023W/RCV/papers/van_Betteray_MGiaD_Multigrid_in_all_Dimensions._Efficiency_and_Robustness_by_Weight_ICCVW_2023_paper.pdf