论文标题
关于强大自然语言推断的模型 - 反应式偏见策略的实证研究
An Empirical Study on Model-agnostic Debiasing Strategies for Robust Natural Language Inference
论文作者
论文摘要
先前关于自然语言推断（NLI）辩论的工作主要是针对一个或几个已知偏见的目标，同时不一定会使模型更强大。在本文中，我们专注于模型 - 不足的辩论策略，并探索如何（或可能）使NLI模型对多次不同的对抗性攻击进行健全，同时保持甚至增强模型的通用能力。首先，我们基于盛行的神经NLI模型，包括各种对抗数据集中的预审计的模型。然后，我们尝试通过修改专家（MOE）集合方法的混合物来应对明显的已知偏见，并表明同时减轻多个NLI偏见是不繁琐的，并且该模型级集合方法优于MOE集合方法。我们还执行数据扩展，包括文本交换，单词替换和释义，并证明其在同时对抗各种（尽管不是全部）对抗攻击的效率。最后，我们研究了几种合并异质训练数据（135万）并执行模型结合的方法，这些方法直接但有效地增强了NLI模型。
The prior work on natural language inference (NLI) debiasing mainly targets at one or few known biases while not necessarily making the models more robust. In this paper, we focus on the model-agnostic debiasing strategies and explore how to (or is it possible to) make the NLI models robust to multiple distinct adversarial attacks while keeping or even strengthening the models' generalization power. We firstly benchmark prevailing neural NLI models including pretrained ones on various adversarial datasets. We then try to combat distinct known biases by modifying a mixture of experts (MoE) ensemble method and show that it's nontrivial to mitigate multiple NLI biases at the same time, and that model-level ensemble method outperforms MoE ensemble method. We also perform data augmentation including text swap, word substitution and paraphrase and prove its efficiency in combating various (though not all) adversarial attacks at the same time. Finally, we investigate several methods to merge heterogeneous training data (1.35M) and perform model ensembling, which are straightforward but effective to strengthen NLI models.
