Self-Refine: Iterative Refinement with Self-Feedback

Self-Refine is a test-time approach for improving the initial outputs of large language models through iterative feedback and refinement, modeled on how humans revise their writing. The same LLM generates an initial output, produces feedback on that output, and uses the feedback to refine itself, repeating the loop; the method requires no supervised training data, additional training, or reinforcement learning, using a single model as generator, refiner, and feedback provider.

Evaluated across 7 diverse tasks ranging from dialog response generation to mathematical reasoning with GPT-3.5, ChatGPT, and GPT-4, Self-Refine's outputs were preferred by both humans and automatic metrics over conventional one-step generation, improving task performance by about 20% absolute on average. The work demonstrated that even state-of-the-art models like GPT-4 can be further improved at test time using a simple, standalone approach.