HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering

HotpotQA is a question-answering dataset designed to push systems toward complex reasoning and explainable answers, addressing the failure of prior datasets to do so. It contains 113k Wikipedia-based question-answer pairs with four key features: questions require finding and reasoning over multiple supporting documents; they are diverse and not constrained to any pre-existing knowledge base or schema; each provides sentence-level supporting facts for strong supervision; and it introduces factoid comparison questions.

The supporting facts let QA systems reason with strong supervision and produce explainable predictions, while the comparison questions test a model's ability to extract relevant facts and compare them. Experiments demonstrate that HotpotQA is challenging for the latest QA systems, and that access to the supporting facts enables models to improve performance and make their predictions explainable, making the dataset a widely used multi-hop reasoning benchmark.