论文标题
使用CNN测试表示代表成本理论的预测
Testing predictions of representation cost theory with CNNs
论文作者
论文摘要
人们普遍认为，受过训练的卷积神经网络（CNN）对不同频率信号的敏感性不同。特别是，许多实证研究已经证明了CNN对低频信号的敏感性。在这项工作中，我们通过理论和实验表明，这种观察到的灵敏度是自然图像的频率分布的结果，自然图像的频率分布是其大多数功率集中在低到中间频率上的结果。我们的理论分析依赖于频率空间中CNN层的表示，该想法以前已用于加速计算和研究网络培训算法的隐式偏见，但据我们所知，尚未将其应用于模型鲁棒性领域。
It is widely acknowledged that trained convolutional neural networks (CNNs) have different levels of sensitivity to signals of different frequency. In particular, a number of empirical studies have documented CNNs sensitivity to low-frequency signals. In this work we show with theory and experiments that this observed sensitivity is a consequence of the frequency distribution of natural images, which is known to have most of its power concentrated in low-to-mid frequencies. Our theoretical analysis relies on representations of the layers of a CNN in frequency space, an idea that has previously been used to accelerate computations and study implicit bias of network training algorithms, but to the best of our knowledge has not been applied in the domain of model robustness.
