Skip to main content
AITier

Llama 3.1 Nemotron Instruct 70B

NVIDIALlamaOpen WeightLlama 3.1 Community License

Description

A large language model customized by NVIDIA to improve the helpfulness of LLM generated responses. It is a fine-tuned version of Llama 3.1 70B Instruct. The model was trained using RLHF (REINFORCE) with HelpSteer2-Preference prompts.

Release Date
2024-10-15
Parameters
70.0B
Context Length
131K
Modalities
text

Capability Radar

29
general
14
coding
27
reasoning
30
scienceest.
0
agents
0
multimodal

Science uses a reasoning proxy when dedicated science benchmarks are unavailable.

Rankings

Domain#RankScoreSource
Code Ranking391
14.0
AA
General Ranking355
31.0
AA
Math Reasoning282
26.0
AA
Reasoning18
86.0
LS
Science346
31.0
AA

Benchmark Scores (LLM Stats)

Communication

MT-Bench0.09 / 100SR

Finance

MMLU Chat80.6%SR
MMLU80.2%SR
TruthfulQA58.6%SR

General

Instruct HumanEval73.8%SR
ARC-C69.2%SR

Language

Winogrande84.5%SR
XLSum English31.6%SR

Math

GSM8k91.4%SR
GSM8K Chat81.9%SR

Reasoning

HellaSwag85.6%SR

AA Evaluation Indices

Intelligence Index
13.4
Math Index
11.0
Coding Index
10.8
Math 500
0.7
Mmlu Pro
0.7
Gpqa
0.5
Ifbench
0.3
Aime
0.2
Scicode
0.2
Tau2
0.2
Livecodebench
0.2
Aime 25
0.1
Lcr
0.1
Hle
0.0
Terminalbench Hard
0.0

LLM Stats Category Scores

Math
90
Language
80
Finance
70
Healthcare
70
Legal
70
Reasoning
70
General
50
Communication
10
Creativity
10
Roleplay
10

Pricing

Input Price$1.2 / 1M tokens
Output Price$1.2 / 1M tokens
Blended Price (3:1)$1.2 / 1M tokens

Speed

Tokens/sec38.1 tokens/s
Time to First Token0.34s
Time to Answer0.34s

Available Providers

(LS internal units)

No provider data available

External Sources

LLM Stats