Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
Applies preference modeling and RLHF to finetune language models into helpful and harmless assistants, using iterated online updates from human feedback.
The authors apply preference modeling and reinforcement learning from human feedback (RLHF) to finetune language models into helpful and harmless assistants. This alignment training improves almost all NLP evaluations and stays compatible with skills like Python coding and summarization. They explore an iterated online mode where preference models and RL policies are updated weekly with fresh human feedback. Studying robustness, they find a roughly linear relation between RL reward and the square root of the KL divergence between the policy and its initialization.
Based on: Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback · arXiv.org
Curated by Aramai Editorial
Read summary →