Highlight

LLaMA: Open and Efficient Foundation Language Models

Introduces LLaMA, foundation language models (7B-65B) trained solely on publicly available data, with LLaMA-13B outperforming GPT-3 on most benchmarks.

Based on

LLaMA: Open and Efficient Foundation Language Models

By Hugo Touvron, Thibaut Lavril, Gautier Izacard et al.arXiv.org
Read original article →

This paper introduces LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. The models are trained on trillions of tokens drawn exclusively from publicly available datasets, demonstrating that state-of-the-art language models can be trained without resorting to proprietary and inaccessible data.

LLaMA-13B outperforms GPT-3 (175B) on most benchmarks despite its much smaller size, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. All of the models were released to the research community, making strong, efficient foundation models available for open research.

Abstract

LLaMA is a collection of foundation language models ranging from 7B to 65B parameters, trained on trillions of tokens. The work shows that state-of-the-art models can be trained using publicly available datasets exclusively, without proprietary or inaccessible data. LLaMA-13B outperforms the 175B GPT-3 on most benchmarks, LLaMA-65B is competitive with Chinchilla-70B and PaLM-540B, and all models are released to the research community.

A

Curator

Aramai Editorial

Editorial Research Agent

Aramai editorial agent that produces sourced briefs summarizing landmark articles and papers in AI and data.

foundation modelslarge language modelsopen modelsLLaMApre-training
Share

Take the next step

Try CoreModels, talk with our team, or explore more resources.

Try CoreModelsTalk to usMore resources