论文标题
标签感知的神经切线内核：迈向更好的概括和局部弹性
Label-Aware Neural Tangent Kernel: Toward Better Generalization and Local Elasticity
论文作者
论文摘要
作为建模训练过多散热性神经网络（NNS）动力学的一种流行方法，众所周知，神经切线内核（NTK）的概括能力落后于现实世界中的NNS。该性能差距部分是由于\ textIt {label aStic} ntk的性质，它的性质将所得的内核不像\ textit {局部弹性}作为nns〜 \ citep {he2019local}。在本文中，我们从\ emph {label-sevareness}的角度介绍了一种新颖的方法，以减少NTK的这一差距。具体而言，我们提出了两个标签感知的内核，它们都是标签 - 敏捷部分的叠加，以及使用Hoeffding分解的标签依赖性复杂性的标签感知部分的层次结构。通过理论和经验证据，我们表明，用所提出的内核训练的模型可以更好地模拟NNS，以概括能力和局部弹性。
As a popular approach to modeling the dynamics of training overparametrized neural networks (NNs), the neural tangent kernels (NTK) are known to fall behind real-world NNs in generalization ability. This performance gap is in part due to the \textit{label agnostic} nature of the NTK, which renders the resulting kernel not as \textit{locally elastic} as NNs~\citep{he2019local}. In this paper, we introduce a novel approach from the perspective of \emph{label-awareness} to reduce this gap for the NTK. Specifically, we propose two label-aware kernels that are each a superimposition of a label-agnostic part and a hierarchy of label-aware parts with increasing complexity of label dependence, using the Hoeffding decomposition. Through both theoretical and empirical evidence, we show that the models trained with the proposed kernels better simulate NNs in terms of generalization ability and local elasticity.
