Qwen Technical Report

The report introduces Qwen, the first installment of a comprehensive large language model series that spans models of varying parameter counts. It comprises the base pretrained models, called Qwen, and Qwen-Chat, chat models fine-tuned with human alignment techniques, notably Reinforcement Learning from Human Feedback (RLHF). Building on the base models, the authors also develop coding-specialized models, Code-Qwen and Code-Qwen-Chat, and mathematics-focused models, Math-Qwen-Chat.

The base language models consistently show superior performance across a wide range of downstream tasks, and the RLHF-trained chat models are highly competitive, exhibiting advanced tool-use and planning capabilities that support agent applications, performing impressively even against larger models on complex tasks such as using a code interpreter. The specialized coding and math models significantly outperform existing open-source models while falling only slightly behind proprietary systems, positioning Qwen as a strong open contribution to LLM research.