Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing

The paper is a systematic survey of what the authors call prompt-based learning, a paradigm that departs from traditional supervised learning. Instead of directly modeling P(y|x), it leverages language models that assign probabilities to text: an input is transformed by a template into a textual prompt containing unfilled slots, the language model probabilistically fills those slots, and the final answer is derived from the resulting string. The authors introduce a unified mathematical notation intended to cover a broad range of existing work.

The framework's appeal is that a single language model pre-trained on massive unlabeled text can be adapted to new tasks simply by designing prompting functions, supporting few-shot and even zero-shot learning. To make the area accessible, the survey organizes prior research along dimensions such as the choice of pre-trained model, the form of the prompt, and the tuning strategy, and it releases additional resources including the NLPedia-Pretrain website with a continually updated paper list.