Skip to main content
AITier

Claude 3.5 Sonnet

AnthropicClaudeProprietary

Description

Claude 3.5 Sonnet is a powerful AI model with industry-leading software engineering skills. It excels in coding, planning, and problem-solving, with significant improvements in agentic coding and tool use tasks. The model includes computer use capabilities in public beta, allowing it to interact with computer interfaces like a human user.

Release Date
2024-10-22
Parameters
—
Context Length
—
Modalities
image, text

Capability Radar

80
general
70
coding
90
reasoning
60
scienceest.
60
agents
100
multimodal

Science uses a reasoning proxy when dedicated science benchmarks are unavailable.

Rankings

Domain#RankScoreSource
Agents & Tools99
18.0
LS
Multimodal Ranking1
94.0
LS

Benchmark Scores (LLM Stats)

Agents

OSWorld Extended22.0%SR
OSWorld Screenshot-only14.9%SR

Biology

GPQA67.2%SR

Code

HumanEval93.7%SR
SWE-Bench Verified49.0%SR

Communication

TAU-bench Retail69.2%SR
TAU-bench Airline46.0%SR

Finance

MMLU90.4%SR
MMLU-Pro77.6%SR

General

MMMU68.3%SR

Image To Text

DocVQA95.2%SR

Language

BIG-Bench Hard93.1%SR

Math

GSM8k96.4%SR
MGSM91.6%SR
DROP87.1%SR
MATH78.3%SR
MathVista67.7%SR

Multimodal

AI2D94.7%SR
ChartQA90.8%SR

AA Evaluation Indices

No AA evaluation data available

LLM Stats Category Scores

Image To Text
100
Language
90
Math
90
Vision
80
Finance
80
General
80
Healthcare
80
Legal
80
Multimodal
80
Reasoning
80
Biology
70
Chemistry
70
Code
70
Physics
70
Tool Calling
60
Communication
60
Frontend Development
50

Pricing

No pricing data available

Speed

No speed data available

Available Providers

(LS internal units)

No provider data available

External Sources

LLM Stats