Gemini 1.5 Pro (Sep '24)
GoogleGeminiProprietary
Description
Gemini 1.5 Pro is a mid-size multimodal model optimized for a wide range of reasoning tasks. It can process large amounts of data at once, including 2 hours of video, 19 hours of audio, codebases with 60,000 lines of code, or 2,000 pages of text.
Release Date
2024-09-24
Parameters
—
Context Length
—
Modalities
image, text
Capability Radar
28
general
27
coding
50
reasoning
42
science
43
agents
80
multimodal
Rankings
| Domain | #Rank | Score | Source |
|---|---|---|---|
| Code Ranking | 424 | 31.0 | AA |
| General Ranking | 421 | 33.0 | AA |
| Multimodal Ranking | 92 | 48.0 | LS |
| Science | 451 | 32.0 | AA |
Benchmark Scores (LLM Stats)
(LLM Stats (zeroeval))General
MMLU
85.9%SR
Language
FLEURS
93.3%SR
MMLU-Pro
75.8%SR
WMT23
75.1%SR
Long Context
MRCR
82.6%SR
Math
GSM8k
90.8%SR
MGSM
87.5%SR
MATH
86.5%SR
MathVista
68.1%SR
FunctionalMATH
64.6%SR
HiddenMath
52.0%SR
AMC_2022_23
46.4%SR
Multimodal
Video-MME
78.6%SR
MMMU
65.9%SR
Vibe-Eval
53.9%SR
Physics
PhysicsFinals
63.9%SR
Reasoning
HellaSwagAI2 (2019)
93.3%SR
BIG-Bench Hard
89.2%SR
Natural2Code
85.4%SR
HumanEvalOpenAI (2021)
84.1%SR
DROP
74.9%SR
GPQANYU + Cohere + Anthropic (2023)
59.1%SR
Safety
XSTest
98.8%SR
AA Evaluation Indices
(Artificial Analysis)Math 500(OpenAI (2024), subset of Hendrycks et al. MATH (2021))87.6
Mmlu Pro(TIGER-Lab (Univ. of Waterloo, Toronto, CMU, 2024))75.0
Gpqa(NYU + Cohere + Anthropic (2023))58.9
Livecodebench(UC Berkeley + MIT + Cornell (2024))31.6
Coding Index(Artificial Analysis)23.6
Aime(MAA (Mathematical Association of America))23.0
Intelligence Index(Artificial Analysis)7.9
Hle(Center for AI Safety + Scale AI (2025))4.6
LLM Stats Category Scores
(LLM Stats (zeroeval))Safety100
Speech To Text90
Language80
Legal80
Long Context80
Math80
Reasoning80
Finance80
General80
Healthcare80
Code80
Multimodal70
Vision70
Physics60
Biology60
Chemistry60
Pricing
Input PriceFree
Output PriceFree
Blended Price (3:1)Free
Speed
Tokens/sec0.0
Time to First Token0.00s
Time to Answer0.00s
Provider Price Ranking
No provider data available