Skip to main content
AITier

Gemma 3 12B Instruct

GoogleGemmaOpen WeightGemma · Commercial OK

Description

Gemma 3 12B is a 12-billion-parameter vision-language model from Google, handling text and image input and generating text output. It features a 128K context window, multilingual support, and open weights. Suitable for question answering, summarization, reasoning, and image understanding tasks.

Release Date
2025-03-12
Parameters
12.0B
Context Length
131K
Modalities
image, text

Capability Radar

20
general
15
coding
31
reasoning
23
scienceest.
26
agents
80
multimodal

Science uses a reasoning proxy when dedicated science benchmarks are unavailable.

Rankings

Domain#RankScoreSource
Code Ranking459
8.0
AA
General Ranking421
24.0
AA
Math Reasoning253
33.0
AA
Multimodal Ranking46
76.0
LS
Reasoning96
48.0
LS
Science431
22.0
AA

Benchmark Scores (LLM Stats)

Biology

GPQA40.9%SR

Code

HumanEval85.4%SR
LiveCodeBench24.6%SR

Factuality

FACTS Grounding75.8%SR
SimpleQA6.3%SR

Finance

MMLU-Pro60.6%SR

General

IFEval88.9%SR
Natural2Code80.7%SR
MBPP0.73 / 100SR
Global-MMLU-Lite69.5%SR
MMMU (val)59.6%SR
BIG-Bench Extra Hard16.3%SR

Image To Text

DocVQA87.1%SR
VQAv2 (val)71.6%SR
TextVQA67.7%SR

Language

BIG-Bench Hard85.7%SR
WMT24++51.6%SR
ECLeKTic10.3%SR

Math

GSM8k94.4%SR
MATH83.8%SR
MathVista-Mini62.9%SR
HiddenMath54.5%SR

Multimodal

AI2D84.2%SR
ChartQA75.7%SR
InfoVQA64.9%SR

Reasoning

Bird-SQL (dev)47.9%SR

AA Evaluation Indices

Math Index
18.3
Intelligence Index
3.4
Math 500
0.9
Mmlu Pro
0.6
Ifbench
0.4
Gpqa
0.3
Aime
0.2
Aime 25
0.2
Scicode
0.2
Livecodebench
0.1
Tau2
0.1
Lcr
0.1
Hle
0.0
Terminalbench Hard
0.0

LLM Stats Category Scores

Instruction Following
90
Structured Output
90
Image To Text
80
Grounding
80
Math
70
Multimodal
70
Vision
70
Legal
60
Reasoning
60
Finance
60
Healthcare
60
Code
60
Language
50
General
50
Physics
40
Factuality
40
Biology
40
Chemistry
40

Pricing

Input Price$0.09 / 1M tokens
Output Price$0.29 / 1M tokens
Blended Price (3:1)$0.14 / 1M tokens

Speed

Tokens/sec0.0
Time to First Token0.00s
Time to Answer0.00s

Provider Price Ranking

Provider Price Ranking

6 providers

Cheapest: ChutesMost Expensive: Cloudflare AI Gateway
ProviderInputOutput
1ChutesCheapest
$0.03
$0.1
2Helicone
$0.05
$0.1
3Amazon Bedrock
$0.05
$0.1
4GooglePRIMARY
$0.09
$0.29
5NanoGPT
$0.272
$0.272
6Cloudflare AI Gateway
$0.35
$0.56

Compare pricing across different API providers for this model.

External Sources

LLM StatsArtificial Analysis