Skip to main content
AITier

Llama 3.1 Instruct 405B

MetaLlamaOpen WeightLlama 3.1 Community License

Description

Llama 3.1 405B Instruct is a large language model optimized for multilingual dialogue use cases. It outperforms many available open source and closed chat models on common industry benchmarks. The model supports 8 languages and has a 128K token context length.

Release Date
2024-07-23
Parameters
405.0B
Context Length
—
Modalities
text

Capability Radar

32
general
22
coding
23
reasoning
34
scienceest.
70
agents
0
multimodal

Science uses a reasoning proxy when dedicated science benchmarks are unavailable.

Rankings

Domain#RankScoreSource
Code Ranking296
25.0
AA
General Ranking289
37.0
AA
Math Reasoning303
20.0
AA
Reasoning5
92.0
LS
Science293
36.0
AA

Benchmark Scores (LLM Stats)

Biology

GPQA50.7%SR

Code

HumanEval89.0%SR
Gorilla Benchmark API Bench35.3%SR

Finance

MMLU (CoT)88.6%SR
MMLU87.3%SR
MMLU-Pro73.3%SR

General

ARC-C96.9%SR
MBPP EvalPlus88.6%SR
IFEval88.6%SR
BFCL88.5%SR
Multipl-E HumanEval75.2%SR
Multipl-E MBPP65.7%SR
Nexus58.7%SR

Math

GSM8k96.8%SR
Multilingual MGSM (CoT)91.6%SR
DROP84.8%SR
MATH73.8%SR

Reasoning

API-Bank92.0%SR

AA Evaluation Indices

Intelligence Index
17.4
Coding Index
14.5
Math Index
3.0
Mmlu Pro
0.7
Math 500
0.7
Gpqa
0.5
Ifbench
0.4
Livecodebench
0.3
Scicode
0.3
Lcr
0.2
Aime
0.2
Tau2
0.2
Terminalbench Hard
0.1
Hle
0.0
Aime 25
0.0

LLM Stats Category Scores

Structured Output
90
Instruction Following
90
Math
90
Finance
80
General
80
Healthcare
80
Language
80
Legal
80
Reasoning
80
Tool Calling
70
Code
60
Biology
50
Chemistry
50
Physics
50

Pricing

Input Price$2.75 / 1M tokens
Output Price$6.5 / 1M tokens
Blended Price (3:1)$3.688 / 1M tokens

Speed

Tokens/sec31.5 tokens/s
Time to First Token0.69s
Time to Answer0.69s

Available Providers

(LS internal units)

No provider data available

External Sources

LLM Stats