论文标题
群集和调子：提高文本分类中的冷启动性能
Cluster & Tune: Boost Cold Start Performance in Text Classification
论文作者
论文摘要
在实际情况下，当标记的数据稀缺时，文本分类任务通常从冷启动开始。在这种情况下，针对目标分类任务的微调预训练模型的常见实践很容易产生性能不佳。我们建议一种通过在预训练和微调阶段之间添加中间的无监督分类任务来提高此类模型的性能的方法。作为一个中间任务，我们执行聚类并训练预训练的模型，以预测群集标签。我们在各种数据集上测试了这一假设，并表明此附加的分类阶段可以显着提高性能，主要用于局部分类任务，而可用于微调的标记实例数量仅为几十个至几百个。
In real-world scenarios, a text classification task often begins with a cold start, when labeled data is scarce. In such cases, the common practice of fine-tuning pre-trained models, such as BERT, for a target classification task, is prone to produce poor performance. We suggest a method to boost the performance of such models by adding an intermediate unsupervised classification task, between the pre-training and fine-tuning phases. As such an intermediate task, we perform clustering and train the pre-trained model on predicting the cluster labels. We test this hypothesis on various data sets, and show that this additional classification phase can significantly improve performance, mainly for topical classification tasks, when the number of labeled instances available for fine-tuning is only a couple of dozen to a few hundred.
