论文标题
中级任务培训对域适应和跨语性转移学习的（在）有效性
The (In)Effectiveness of Intermediate Task Training For Domain Adaptation and Cross-Lingual Transfer Learning
论文作者
论文摘要
从大型语言模型（LLM）转移学习已成为一种强大的技术，可以实现基于知识的多个任务的微调，对不同领域甚至语言的模型的改编。但是，这仍然是一个悬而未决的问题，是否以及何时转移学习会起作用，即导致正转移或负转移。在本文中，我们分析了三个自然语言处理（NLP）任务的知识转移 - 文本分类，情感分析和句子相似性，使用三个LLMS- BERT，ROBERTA和XLNET-通过在域名和跨语言适应任务的目标数据集中进行微调，并在不进行Intermed Medical DataSES上进行较大的数据培训。我们的实验表明，没有中间任务培训的微调可以为大多数任务带来更好的性能，而更广泛的任务可能需要先进的中级任务培训步骤。我们希望这项工作将作为向NLP从业人员转移学习的指南。
Transfer learning from large language models (LLMs) has emerged as a powerful technique to enable knowledge-based fine-tuning for a number of tasks, adaptation of models for different domains and even languages. However, it remains an open question, if and when transfer learning will work, i.e. leading to positive or negative transfer. In this paper, we analyze the knowledge transfer across three natural language processing (NLP) tasks - text classification, sentimental analysis, and sentence similarity, using three LLMs - BERT, RoBERTa, and XLNet - and analyzing their performance, by fine-tuning on target datasets for domain and cross-lingual adaptation tasks, with and without an intermediate task training on a larger dataset. Our experiments showed that fine-tuning without an intermediate task training can lead to a better performance for most tasks, while more generalized tasks might necessitate a preceding intermediate task training step. We hope that this work will act as a guide on transfer learning to NLP practitioners.
