论文标题
（QA）$^2 $：通过可疑假设回答问题
(QA)$^2$: Question Answering with Questionable Assumptions
论文作者
论文摘要
自然发生的信息寻求问题通常包含可疑的假设 - 虚假或无法验证的假设。包含可疑假设的问题是具有挑战性的，因为它们需要一个独特的答案策略，该策略偏离了典型的信息寻求信息问题。例如，“玛丽·库里（Marie Curie）什么时候发现铀？”这个问题。不用解决“玛丽·库里发现铀”的错误假设，不能作为典型的“当”问题回答。在这项工作中，我们建议（QA）$^2 $（问题回答可疑假设），这是一个开放域评估数据集，该数据集由天然发生的搜索引擎查询组成，可能或可能不会包含可疑假设。要在（QA）$^2 $上取得成功，系统必须能够检测出可疑的假设，并且还能够为典型的寻求信息的问题和具有可疑假设的问题提供足够的回答。通过（QA）$^2 $对端到端质量检查的人类评估者的可接受性，我们发现当前的模型确实在处理可疑假设方面很难，为进步留下了实质性的余地。
Naturally occurring information-seeking questions often contain questionable assumptions -- assumptions that are false or unverifiable. Questions containing questionable assumptions are challenging because they require a distinct answer strategy that deviates from typical answers for information-seeking questions. For instance, the question "When did Marie Curie discover Uranium?" cannot be answered as a typical "when" question without addressing the false assumption "Marie Curie discovered Uranium". In this work, we propose (QA)$^2$ (Question Answering with Questionable Assumptions), an open-domain evaluation dataset consisting of naturally occurring search engine queries that may or may not contain questionable assumptions. To be successful on (QA)$^2$, systems must be able to detect questionable assumptions and also be able to produce adequate responses for both typical information-seeking questions and ones with questionable assumptions. Through human rater acceptability on end-to-end QA with (QA)$^2$, we find that current models do struggle with handling questionable assumptions, leaving substantial headroom for progress.
