论文标题
向多个专家学习：长尾分类的自定进度知识蒸馏
Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification
论文作者
论文摘要
在实际情况下，数据倾向于表现出长尾分布，这增加了训练深网的困难。在本文中，我们提出了一个新颖的自定进度知识蒸馏框架，称为多个专家（LFME）学习。我们的方法的灵感来自于观察到的，即接受分布不平衡子集训练的网络通常比共同训练的对应物获得更好的性能。我们将这些模型称为“专家”，而拟议的LFME框架汇总了来自多个“专家”的知识，以学习统一的学生模型。具体而言，所提出的框架涉及两个级别的自适应学习时间表：自定进定的专家选择和课程实例选择，以便将知识自适应地转移到“学生”中。我们进行了广泛的实验，并证明与最先进的方法相比，我们的方法能够实现出色的性能。我们还表明，我们的方法可以轻松地插入最先进的长尾分类算法中，以进一步改进。
In real-world scenarios, data tends to exhibit a long-tailed distribution, which increases the difficulty of training deep networks. In this paper, we propose a novel self-paced knowledge distillation framework, termed Learning From Multiple Experts (LFME). Our method is inspired by the observation that networks trained on less imbalanced subsets of the distribution often yield better performances than their jointly-trained counterparts. We refer to these models as 'Experts', and the proposed LFME framework aggregates the knowledge from multiple 'Experts' to learn a unified student model. Specifically, the proposed framework involves two levels of adaptive learning schedules: Self-paced Expert Selection and Curriculum Instance Selection, so that the knowledge is adaptively transferred to the 'Student'. We conduct extensive experiments and demonstrate that our method is able to achieve superior performances compared to state-of-the-art methods. We also show that our method can be easily plugged into state-of-the-art long-tailed classification algorithms for further improvements.
