论文标题
一种新颖的自我知识蒸馏方法，采用暹罗代表学习以进行行动识别
A Novel Self-Knowledge Distillation Approach with Siamese Representation Learning for Action Recognition
论文作者
论文摘要
知识蒸馏是从重型网络（教师）到小型网络（学生）的有效转移，以提高学生的表现。已经提出了自我知识蒸馏，即知识蒸馏的特殊情况，可以在保留学生的表现的同时删除大型教师网络培训过程。本文通过暹罗表示学习介绍了一种新型的自我知识蒸馏方法，该方法将来自给定样本的两个不同观点的两个表示矢量之间的差异最小化。我们提出的方法SKD-SRL使用了软标签蒸馏和表示向量的相似性。因此，SKD-SRL可以在同一数据点的各种视图中生成更一致的预测和表示。我们的基准已在各种标准数据集上进行了评估。实验结果表明，与现有的监督学习和知识蒸馏方法相比，SKD-SRL显着提高了准确性。
Knowledge distillation is an effective transfer of knowledge from a heavy network (teacher) to a small network (student) to boost students' performance. Self-knowledge distillation, the special case of knowledge distillation, has been proposed to remove the large teacher network training process while preserving the student's performance. This paper introduces a novel Self-knowledge distillation approach via Siamese representation learning, which minimizes the difference between two representation vectors of the two different views from a given sample. Our proposed method, SKD-SRL, utilizes both soft label distillation and the similarity of representation vectors. Therefore, SKD-SRL can generate more consistent predictions and representations in various views of the same data point. Our benchmark has been evaluated on various standard datasets. The experimental results have shown that SKD-SRL significantly improves the accuracy compared to existing supervised learning and knowledge distillation methods regardless of the networks.
