Self-Consistency Improves Chain of Thought Reasoning in Language Models

The paper introduces self-consistency, a decoding strategy for chain-of-thought prompting with pre-trained large language models. Instead of greedily decoding a single reasoning path, it first samples a diverse set of candidate reasoning paths and then selects the most consistent final answer by marginalizing over the sampled paths. The method rests on the intuition that a complex reasoning problem typically admits several different ways of thinking that all lead to its unique correct answer.

In extensive empirical evaluation, self-consistency boosted chain-of-thought performance by a striking margin across a range of popular arithmetic and commonsense reasoning benchmarks, with gains of +17.9% on GSM8K, +11.0% on SVAMP, +12.2% on AQuA, +6.4% on StrategyQA, and +3.9% on ARC-challenge. This mattered because it showed a simple, model-agnostic change to decoding could substantially strengthen large language model reasoning without any additional training or fine-tuning.