Gemma 3 4B Instruct
GoogleGemmaOpen WeightGemma · Commercial OK
Description
Gemma 3 4B is a 4-billion-parameter vision-language model from Google, handling text and image input and generating text output. It features a 128K context window, multilingual support, and open weights. Suitable for question answering, summarization, reasoning, and image understanding tasks.
Release Date
2025-03-12
Parameters
4.0B
Context Length
131K
Modalities
image, text
Capability Radar
16
general
6
coding
22
reasoning
22
science
17
agents
70
multimodal
Rankings
| Domain | #Rank | Score | Source |
|---|---|---|---|
| Code Ranking | 616 | 5.0 | AA |
| General Ranking | 631 | 15.0 | AA |
| Multimodal Ranking | 133 | 32.0 | LS |
| Science | 623 | 15.0 | AA |
Benchmark Scores (LLM Stats)
(LLM Stats (zeroeval))Chat
IFEvalGoogle Research (2023)
90.2%SR
Factuality
SimpleQA
4.0%SR
General
Global-MMLU-Lite
54.5%SR
Language
WMT24++
46.8%SR
MMLU-Pro
43.6%SR
ECLeKTic
4.6%SR
Math
GSM8k
89.2%SR
MATH
75.6%SR
MathVista-Mini
50.0%SR
HiddenMath
43.0%SR
Reasoning
BIG-Bench Hard
72.2%SR
HumanEvalOpenAI (2021)
71.3%SR
Natural2Code
70.3%SR
FACTS Grounding
70.1%SR
ChartQAMasry et al. (2022)
68.8%SR
MBPP
0.63 / 100SR
Bird-SQL (dev)
36.3%SR
GPQANYU + Cohere + Anthropic (2023)
30.8%SR
LiveCodeBench
12.6%SR
BIG-Bench Extra Hard
11.0%SR
Vision
DocVQADocVQA (2020)
75.8%SR
AI2D
74.8%SR
VQAv2 (val)
62.4%SR
TextVQA
57.8%SR
InfoVQA
50.0%SR
MMMU (val)
48.8%SR
AA Evaluation Indices
(Artificial Analysis)Math 500(OpenAI (2024), subset of Hendrycks et al. MATH (2021))76.6
Mmlu Pro(TIGER-Lab (Univ. of Waterloo, Toronto, CMU, 2024))41.7
Gpqa(NYU + Cohere + Anthropic (2023))29.1
Ifbench(Google Research (2023))28.3
Math Index(Artificial Analysis)12.7
Aime 25(MAA (Mathematical Association of America))12.7
Livecodebench(UC Berkeley + MIT + Cornell (2024))11.2
Lcr(Artificial Analysis)6.7
Aime(MAA (Mathematical Association of America))6.3
Hle(Center for AI Safety + Scale AI (2025))5.3
Tau2(Sierra + U Toronto + Vector Institute (2025))5.0
Intelligence Index(Artificial Analysis)4.8
Coding Index(Artificial Analysis)2.7
Terminalbench Hard(Stanford × Laude Institute (2026))0.8
Tau Banking0.4
Terminalbench V2 10.4
LLM Stats Category Scores
(LLM Stats (zeroeval))Chat90
Instruction Following90
Structured Output90
Image To Text70
Grounding70
Math60
Multimodal60
Vision60
Reasoning50
General50
Healthcare50
Language40
Legal40
Factuality40
Finance40
Code40
Physics30
Biology30
Chemistry30
Pricing
Input PriceFree
Output PriceFree
Blended Price (3:1)Free
Speed
Tokens/sec0.0
Time to First Token0.00s
Time to Answer0.00s
Provider Price Ranking
Provider Price Ranking
6 providers
Cheapest: DeepInfraMost Expensive: Kilo Gateway
ProviderInputOutput
1DeepInfraCheapest
$0
$0
2Merge Gateway
$0.04
$0.08
3OpenRouter
$0.05
$0.1
4Hugging Face
$0.05
$0.1
5Deep Infra
$0.05
$0.1
6Kilo Gateway
$0.05
$0.1
Compare pricing across different API providers for this model.