HuggingFace's Transformers: State-of-the-art Natural Language Processing

This paper introduces Transformers, an open-source library that packages state-of-the-art Transformer architectures under a single, unified API. The authors frame recent natural language processing progress as driven by two advances: Transformer architectures that allow higher-capacity models, and pretraining that makes it possible to use that capacity effectively across a wide variety of tasks. The library is backed by a curated collection of pretrained models that are made by and available to the community.

Transformers is designed simultaneously to be extensible for researchers, simple for practitioners, and fast and robust in industrial deployments, lowering the barrier to using modern pretrained Transformer models. By opening up these advances to the wider machine learning community through carefully engineered, reusable components, the library became widely adopted infrastructure for applying and sharing pretrained NLP models.