Sparks of Artificial General Intelligence: Early experiments with GPT-4

The authors investigate an early version of OpenAI's GPT-4 while it was still under active development, a model trained using an unprecedented scale of compute and data. Rather than relying only on formal benchmarks, they probe the model qualitatively across a wide range of domains and difficult, novel tasks in order to characterize its capabilities, positioning GPT-4 within a new cohort of large language models alongside systems such as ChatGPT and Google's PaLM.

They report that GPT-4 can solve novel and difficult problems spanning mathematics, coding, vision, medicine, law, and psychology without needing any special prompting, with performance often strikingly close to human level and frequently far surpassing prior models like ChatGPT. On this basis they contend that GPT-4 could reasonably be viewed as an early, though still incomplete, version of an artificial general intelligence system, while placing special emphasis on its limitations and the challenges ahead, including the possible need for a new paradigm that moves beyond next-word prediction.