TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension

TriviaQA is a large-scale reading comprehension dataset containing over 650K question-answer-evidence triples, built from 95K question-answer pairs authored by trivia enthusiasts and independently collected evidence documents that average six per question. These evidence documents supply high-quality distant supervision for training and evaluating systems that answer the questions.

Analysis shows TriviaQA has relatively complex, compositional questions with considerable syntactic and lexical variability between questions and answer-evidence sentences, and it requires more cross-sentence reasoning than other recent datasets. Two baselines, a feature-based classifier and a state-of-the-art neural network, reach only 23% and 40% accuracy versus 80% for humans, marking the dataset as a challenging testbed for future study.