Which LLM should you use? A living buyer's guide
Most model comparisons make a category error: they put a $0.28-per-million model and a $50-per-million model in the same list and act surprised when one “wins”. This guide works the way the decision actually works — three steps: pick your weight class, pick open or closed within it, pick the model. Numbers refresh monthly from the same collection as the Pareto Frontier report; every score links to its source.
Step 1 — Pick your weight class
Section titled “Step 1 — Pick your weight class”| Your workload | Class |
|---|---|
| Agentic coding, multi-step agents, complex reasoning — anywhere a failed run costs more than the tokens did | Heavyweight |
| Extraction, classification, summarization, chat, RAG, pipelines that run all day | Budget |
The classes are real: heavyweight models cost 4–180× more per token than budget ones, and nothing in the budget class should be benchmarked against Claude Opus.
Step 2a — Heavyweight: open or closed?
Section titled “Step 2a — Heavyweight: open or closed?”| Model | AA Index | SWE-bench Verified | SWE-bench Pro | Output $/M | Weights |
|---|---|---|---|---|---|
| Claude Fable 5 | 60 | ~95.0 | — | $50.00 | closed |
| Claude Opus 4.8 | 56 | 88.6 | 69.2 | $25.00 | closed |
| GPT-5.5 | 55 | — | 58.6 | $30.00 | closed |
| Claude Sonnet 5 | 53 | — | — | $15.00 | closed |
| GLM 5.2 | 51 | — | 62.1 | $4.40 | open |
| Gemini 3.1 Pro | 46 | 80.6 | — | $12.00 | closed |
| MiniMax M3 | 44 | 80.5 | 59.0 | $1.20 | open |
| Kimi K2.6 | 43 | 80.2 | 58.6 | $4.00 | open |
| Kimi K2.7 | 42 | — | — | $4.00 | open |
Choose closed when you need the ceiling. Nobody in the open corner beats Claude Opus 4.8 or Fable 5 at the top: 88.6–95 on SWE-bench Verified is 8–15 points clear of the best open scores, and on long-horizon agentic reliability (METR time horizons) every frontier entry is closed. If the last ten points pay for themselves, that fight is theirs — at $25–50 per million output tokens.
Choose open for everything below the ceiling. GLM 5.2 outscores GPT-5.5 on SWE-bench Pro (62.1 vs 58.6) at one-seventh the price; MiniMax M3 and Kimi K2.6 sit at statistical parity with Gemini 3.1 Pro on Verified at one-tenth and one-third of its price. And open weights carry strategic advantages no benchmark shows:
- No deprecation risk. Closed models get retired on the vendor’s schedule — OpenAI retired GPT-4o in February 2026; Anthropic’s deprecation page lists six Claude retirements in nine months. Downloaded weights cannot be retired; the model you validated is the model you run.
- Control. No closed vendor trains on paid API data by default, but with open weights policy isn’t the only protection: the same model can move to any host or your own hardware. Licenses vary — see the fine print per model in State of Open Weights.
- A competitive market. Every closed model has exactly one seller; open weights are served by competing hosts, which is why their prices keep falling (Price Tracker) and why flat-rate offers exist at all.
The Kimi paradox, or why the index misleads here: Kimi K2.6 scores a mid-table 43 on the composite index yet holds the best open SWE-bench Verified score (80.2) — it’s an agentic-coding specialist, and Artificial Analysis called it “the new leading open weights model” at launch. Usage agrees: Chinese-origin open models carry over 45% of OpenRouter’s token volume, and the heaviest token burners are coding agents. When a composite index and the market disagree, look at what the market does with the model.
Step 2b — Budget: open or closed?
Section titled “Step 2b — Budget: open or closed?”| Model | AA Index | Input $/M | Output $/M | Weights |
|---|---|---|---|---|
| Gemini 3.5 Flash | 50 | $1.50 | $9.00 | closed |
| GPT-5.4 mini | 40 | $0.75 | $4.50 | closed |
| Gemini 3 Flash* | 27 | $0.50 | $3.00 | closed |
| GPT-5 mini | 25 | $0.25 | $2.00 | closed |
| Claude 4.5 Haiku* | 24 | $1.00 | $5.00 | closed |
| DeepSeek V4 Flash | 40 | $0.14 | $0.28 | open |
| MiMo V2.5 | — | $0.105 | $0.28 | open |
* Non-reasoning variants as evaluated by Artificial Analysis; the others are reasoning variants. MiMo V2.5 price is its OpenRouter listing — no AA index entry yet.
This division isn’t close on today’s data: DeepSeek V4 Flash matches GPT-5.4 mini’s index (40 = 40) at 16× less per output token, and every closed model under $1 input scores 13–16 points below it while costing 7–18× more. The one strong closed entry, Gemini 3.5 Flash (50), costs $9/M output — twice the price of GLM 5.2 (51), a heavyweight. When a vendor’s budget model costs more than the other side’s heavyweight, the weight classes have inverted. MiMo V2.5 adds native multimodal input, a 1M context and a 56.1 SWE-bench Pro card score at $0.28/M.
Step 3 — Which model for which job
Section titled “Step 3 — Which model for which job”| The job | Reach for | Why |
|---|---|---|
| Agentic coding, tool-heavy agents | Kimi K2.7 / K2.6 | Best open SWE-bench Verified (80.2); K2.7 is the token-efficiency refresh |
| Long-horizon coding, hardest open-model work | GLM 5.2 | Top open index (51); beats GPT-5.5 on SWE-bench Pro |
| Whole-codebase / long-document / multimodal | MiniMax M3 | 1M context, image+video input, ~80 SWE-bench Verified at $1.20/M |
| High-volume pipelines, chat, extraction | DeepSeek V4 Flash | Index 40 at $0.28/M output — the budget Pareto anchor |
| Budget multimodal | MiMo V2.5 | 1M context + multimodal at the lowest price on the board |
| The absolute capability ceiling | Claude Opus 4.8 / Fable 5 | 88.6–95 SWE-bench Verified — when the last points pay for themselves |
The first five are served in our pools flat-rate — Frontier from $44.20/mo, Core from $5.94/mo, no token caps during your reserved hours.
Head-to-head: the pairs people actually compare
Section titled “Head-to-head: the pairs people actually compare”Search traffic clusters on a few specific match-ups. Here is each one, with the same sourced numbers as the tables above — they refresh monthly from the same collection, so nothing below is a frozen figure. The short version first: most “X vs Y” questions are really which job, not which is better. Where a vendor publishes no score for a cell, it shows — rather than an estimate.
Kimi vs DeepSeek
Section titled “Kimi vs DeepSeek”| Model | AA Index | SWE-bench Verified | SWE-bench Pro | Input $/M | Output $/M | Context | Weights |
|---|---|---|---|---|---|---|---|
| Kimi K2.7 | 42 | — | — | $0.95 | $4.00 | 256K | open |
| Kimi K2.6 | 43 | 80.2 | 58.6 | $0.95 | $4.00 | 256K | open |
| DeepSeek V4 Flash | 40 | — | — | $0.14 | $0.28 | 1M | open |
Different weight classes doing different jobs, not rivals for the same task. Kimi is the agentic-coding specialist — reach for it when a failed multi-step, tool-heavy run costs more than the tokens it burns; K2.6 holds the benchmark score and K2.7 is the token-efficiency refresh. DeepSeek V4 Flash is the budget workhorse: high-volume extraction, chat, RAG and pipelines that run all day, at a fraction of Kimi’s per-token cost. Pick Kimi for the hard agent loops, DeepSeek when throughput and price dominate.
Kimi K2.7, Kimi K2.6 and DeepSeek V4 Flash are all served in our pools flat-rate — no per-token metering during your reserved hours.
GLM vs Kimi
Section titled “GLM vs Kimi”| Model | AA Index | SWE-bench Verified | SWE-bench Pro | Input $/M | Output $/M | Context | Weights |
|---|---|---|---|---|---|---|---|
| GLM 5.2 | 51 | — | 62.1 | $1.40 | $4.40 | 1M | open |
| Kimi K2.6 | 43 | 80.2 | 58.6 | $0.95 | $4.00 | 256K | open |
| Kimi K2.7 | 42 | — | — | $0.95 | $4.00 | 256K | open |
Both are open heavyweight coders; the split is horizon. GLM 5.2 carries the top open AA index and a longer context window — reach for it on long-horizon, whole-repo work. Kimi holds the best open SWE-bench Verified score and is tuned for tool-heavy agent runs. Per-token pricing is close, so choose on the work rather than the sticker: GLM for the longest, hardest tasks, Kimi for multi-step agentic coding.
GLM 5.2 and both Kimi versions are on our pools flat-rate — capacity you reserve, not metered volume.
GLM vs DeepSeek
Section titled “GLM vs DeepSeek”| Model | AA Index | SWE-bench Verified | SWE-bench Pro | Input $/M | Output $/M | Context | Weights |
|---|---|---|---|---|---|---|---|
| GLM 5.2 | 51 | — | 62.1 | $1.40 | $4.40 | 1M | open |
| DeepSeek V4 Flash | 40 | — | — | $0.14 | $0.28 | 1M | open |
Weight classes again. GLM 5.2 is a heavyweight for the hardest coding and long-horizon reasoning; DeepSeek V4 Flash is the budget anchor for volume. GLM costs more per token but scores higher on the index; DeepSeek matches mid-tier closed models on the index at budget-tier pricing. Reach for GLM when the task is hard enough that the extra capability pays for itself, DeepSeek when you are running all day and per-token cost compounds.
Both GLM 5.2 and DeepSeek V4 Flash are available flat-rate on our pools — no token caps during your reserved hours.
MiniMax vs DeepSeek
Section titled “MiniMax vs DeepSeek”| Model | AA Index | SWE-bench Verified | SWE-bench Pro | Input $/M | Output $/M | Context | Weights |
|---|---|---|---|---|---|---|---|
| MiniMax M3 | 44 | 80.5 | 59.0 | $0.30 | $1.20 | 1M | open |
| DeepSeek V4 Flash | 40 | — | — | $0.14 | $0.28 | 1M | open |
Both open, both long-context, but different tiers. MiniMax M3 sits near the top of the open SWE-bench Verified scores at the lowest price in that band and adds native multimodal input — reach for it on whole-codebase, long-document or image and video work. DeepSeek V4 Flash is cheaper still and tuned for high-volume text pipelines. Choose MiniMax when you need the capability or the modalities, DeepSeek when the job is bulk text and cost is the deciding factor.
MiniMax M3 and DeepSeek V4 Flash are both served flat-rate on our pools — you buy reserved capacity, not metered tokens.
Caveats
Section titled “Caveats”Scores are as published by each vendor or independent evaluator; scaffolding differs between labs, which is why every number links to its source. The AA Intelligence Index is one composite — task-specific rankings differ (see the Kimi callout). Prices are list prices for the variants AA evaluates; open models are often cheaper on aggregators. Where a vendor publishes no score, the cell shows — rather than an estimate.
Changelog
Section titled “Changelog”- 2026-07-09 — added a Head-to-head section covering the four most-compared pairs (Kimi vs DeepSeek, GLM vs Kimi, GLM vs DeepSeek, MiniMax vs DeepSeek). Each table reads from the same monthly collection as the rest of the guide, so the numbers move with it; the verdicts are static.
- 2026-07-06 (first edition) — this guide absorbs and supersedes two blog posts (“Open-source vs proprietary LLMs in 2026” and “LLM weight classes”), whose data now lives here and refreshes monthly instead of rotting in dated posts. Baseline verdicts: heavyweight ceiling closed (Opus 4.8 / Fable 5), everything below it open (GLM 5.2, MiniMax M3, Kimi K2.6/K2.7); budget division dominated by open (DeepSeek V4 Flash, MiMo V2.5).
Companion reports: LLM Pareto Frontier (the price-intelligence map) · State of Open Weights (the served models in depth) · LLM Price Tracker (prices over time).