Lost in the Middle: How Language Models Use Long Contexts

This paper investigates how well language models actually use long input contexts, a question that had received little study even as models gained the ability to accept longer inputs. The authors analyze performance on two tasks that require identifying relevant information within the input: multi-document question answering and key-value retrieval. By systematically varying where the relevant information is placed within the context, they measure how sensitive models are to position.

They find that performance can degrade significantly depending on the position of the relevant information, indicating that current models do not robustly use information across long contexts. Performance is often highest when the relevant content appears at the beginning or end of the input and degrades substantially when models must retrieve information from the middle, even for models explicitly built for long contexts. This lost-in-the-middle finding provides a better understanding of context usage and introduces new evaluation protocols for future long-context models.