论文标题
深层分类器擅长非平凡概括吗？
Are Deep Sequence Classifiers Good at Non-Trivial Generalization?
论文作者
论文摘要
序列分类的深度学习模型的最新进展极大地提高了它们的分类准确性，特别是当有大型培训集时。但是，几项作品表明，在某些情况下，这些模型的预测校准较差。在这项工作中，我们研究了二进制序列分类问题，并通过提出问题来研究模型校准：深度学习模型是否能够学习基础目标类别分布？我们专注于稀疏序列分类，即目标类是罕见的问题，并比较了三个深度学习序列分类模型。我们开发了一项评估，以衡量分类器学习目标类别分布的能力。此外，我们的评估仅通过仅压缩训练序列而与正确的模型概括实现的性能来实现良好的性能。我们的结果表明，在这种二进制环境中，深度学习模型确实能够以非平凡的方式学习基础类别的分布，即通过适当的概括数据压缩。
Recent advances in deep learning models for sequence classification have greatly improved their classification accuracy, specially when large training sets are available. However, several works have suggested that under some settings the predictions made by these models are poorly calibrated. In this work we study binary sequence classification problems and we look at model calibration from a different perspective by asking the question: Are deep learning models capable of learning the underlying target class distribution? We focus on sparse sequence classification, that is problems in which the target class is rare and compare three deep learning sequence classification models. We develop an evaluation that measures how well a classifier is learning the target class distribution. In addition, our evaluation disentangles good performance achieved by mere compression of the training sequences versus performance achieved by proper model generalization. Our results suggest that in this binary setting the deep-learning models are indeed able to learn the underlying class distribution in a non-trivial manner, i.e. by proper generalization beyond data compression.
