Llama 3.1 Instruct 405B
MetaLlamaOpen WeightLlama 3.1 Community License
Description
Llama 3.1 405B Instruct is a large language model optimized for multilingual dialogue use cases. It outperforms many available open source and closed chat models on common industry benchmarks. The model supports 8 languages and has a 128K token context length.
Release Date
2024-07-23
Parameters
405.0B
Context Length
—
Modalities
text
Capability Radar
27
general
30
coding
23
reasoning
34
scienceest.
70
agents
0
multimodal
Science uses a reasoning proxy when dedicated science benchmarks are unavailable.
Rankings
| Domain | #Rank | Score | Source |
|---|---|---|---|
| Code Ranking | 323 | 25.0 | AA |
| General Ranking | 327 | 33.0 | AA |
| Math Reasoning | 303 | 20.0 | AA |
| Reasoning | 5 | 92.0 | LS |
| Science | 318 | 35.0 | AA |
Benchmark Scores (LLM Stats)
Biology
GPQA
50.7%SR
Code
HumanEval
89.0%SR
Gorilla Benchmark API Bench
35.3%SR
Finance
MMLU (CoT)
88.6%SR
MMLU
87.3%SR
MMLU-Pro
73.3%SR
General
ARC-C
96.9%SR
MBPP EvalPlus
88.6%SR
IFEval
88.6%SR
BFCL
88.5%SR
Multipl-E HumanEval
75.2%SR
Multipl-E MBPP
65.7%SR
Nexus
58.7%SR
Math
GSM8k
96.8%SR
Multilingual MGSM (CoT)
91.6%SR
DROP
84.8%SR
MATH
73.8%SR
Reasoning
API-Bank
92.0%SR
AA Evaluation Indices
Intelligence Index8.5
Math Index3.0
Mmlu Pro0.7
Math 5000.7
Gpqa0.5
Ifbench0.4
Livecodebench0.3
Scicode0.3
Lcr0.2
Aime0.2
Tau20.2
Terminalbench Hard0.1
Hle0.0
Aime 250.0
LLM Stats Category Scores
Instruction Following90
Math90
Structured Output90
Language80
Legal80
Reasoning80
Finance80
General80
Healthcare80
Tool Calling70
Code60
Physics50
Biology50
Chemistry50
Pricing
Input Price$2.75 / 1M tokens
Output Price$6.5 / 1M tokens
Blended Price (3:1)$3.688 / 1M tokens
Speed
Tokens/sec60.6
Time to First Token0.68s
Time to Answer0.68s
Provider Price Ranking
Provider Price Ranking
7 providers
Cheapest: Kilo GatewayMost Expensive: Azure
ProviderInputOutput
1Kilo GatewayCheapest
$1
$1
2Venice AI
$1.1
$3
3MetaPRIMARY
$2.75
$6.5
4Synthetic
$3
$3
5Abacus
$3.5
$3.5
6Azure Cognitive Services
$5.33
$16
7Azure
$5.33
$16
Compare pricing across different API providers for this model.