On the Opportunities and Risks of Foundation Models

The report argues that AI is undergoing a paradigm shift driven by foundation models—models such as BERT, DALL-E, and GPT-3 that are trained on broad data at scale and can be adapted to a wide range of downstream tasks. It provides a thorough account spanning their capabilities (language, vision, robotics, reasoning, human interaction), technical principles (model architectures, training procedures, data, systems, security, evaluation, theory), applications (law, healthcare, education), and societal impact (inequity, misuse, economic and environmental impact, legal and ethical considerations).

A central theme is that although foundation models rest on standard deep learning and transfer learning, their scale produces new emergent capabilities, and their effectiveness across many tasks incentivizes homogenization. Homogenization offers powerful leverage but is risky, because defects in the foundation model are inherited by all adapted downstream models. The authors stress that we still lack a clear understanding of how these models work, when they fail, and what they can do, calling for deep interdisciplinary collaboration given their sociotechnical nature.