Qwen2.5 Technical Report

The Qwen2.5 technical report introduces a comprehensive series of large language models improved substantially in both pre-training and post-training. On the pre-training side, the high-quality dataset was scaled from the previous 7 trillion tokens to 18 trillion tokens, strengthening common sense, expert knowledge, and reasoning. On the post-training side, the team applied supervised finetuning with over 1 million samples along with multi-stage reinforcement learning, enhancing human preference alignment and notably improving long text generation, structured-data analysis, and instruction following.

Qwen2.5 is offered in many sizes to fit diverse use cases: open-weight base and instruction-tuned models (including quantized versions), plus hosted proprietary mixture-of-experts variants, Qwen2.5-Turbo and Qwen2.5-Plus. It achieves top-tier results across benchmarks in language understanding, reasoning, mathematics, coding, and alignment; the open-weight flagship Qwen2.5-72B-Instruct performs competitively with the roughly five-times-larger Llama-3-405B-Instruct, while Turbo and Plus rival GPT-4o-mini and GPT-4o with better cost-effectiveness. Qwen2.5 also underpins specialized models like Qwen2.5-Math and Qwen2.5-Coder.