A benchmark for AI models on real non-coding knowledge work, graded by rubric, scored by a mix of human reviewers and AI judges, priced by no-cache tokens.
For most non-coding work, the right default is Balanced Value · one lens of four.
A blended score that rewards what production work actually needs: quality you can defend, cost you can scale, and consistency you can trust. The cost term is gated by consistency — a $0.03 run that crashes 6% of the time loses to a steady $0.09 one.
Each card shows the same eleven models scored under a different cost-aware lens. The headline is the lens leader; the strip below is the non-dominated frontier (models not beaten on both quality and price). Click a card to read what the lens optimizes for and why it matters.
The Balanced Value Index blends quality, cost efficiency and reliability, gated by consistency so cheap-but-volatile models can't game the board. For most work outside of coding it is the right default ranking. It is not the only one: switch the tabs, or set your own weights below.
M3 leads Balanced Value (76.6) and Acceptable Value (88.1) by combining the lowest cost in the suite with acceptable quality. It is not the quality leader — Claude Fable 5 takes Quality Core (93.5) with a perfect Pass@80, at 51× the price. Adding Opus 4.7 and 4.8 widens the premium tier but doesn't change the production-value answer. The right deployment is therefore a routing policy, not a single-model bet.
This routing layer is what G4 OS ships: every model above in one workspace, routed for quality, speed and cost. Download G4 OS → · Benchmark code and data on GitHub.
Switch the metric tab to re-rank · Balanced is the recommended default for everyday selection.
Quality is 72% of the final score. Cost is 23%, gated by consistency so a $0.03 run that crashes 6% of the time loses to a steady $0.09 run. Reliability adds 5%.
Two views: the range of scores each model produces across 5 runs, and the operating cost per run. The mean (open circle) and median (filled circle) markers expose each model's consistency.
Agentic work costs more than tokens. Extra steps and longer tool loops add latency, orchestration load, error surface, and user-perceived unpredictability. Drag is a real cost separable from quality and price.
The two cost-aware lenses point at different defaults on purpose. Acceptable rewards cheap reliability for volume; Excellent rewards premium quality that isn't wildly overpriced. The crossover is the trade-off.
Four facts that change which model you should reach for tomorrow — and why a blended score beats any one number.
Today's price gap between US frontier models and China-linked low-cost models is the widest the industry has ever priced. The benchmark would be naive to ignore that. We re-rank the field under two price shocks — holding quality, latency, errors, and tokens fixed.
Only MiniMax and Kimi reprice upward (~2×). The leader changes: GPT-5.4 takes #1 (Balanced 75.3), narrowly ahead of GPT-5.5 (75.2). M3 still wins Acceptable Value — but its overall production lead evaporates.
All models reprice upward; the China-linked pair rises more. MiniMax M3 ties GPT-5.4 at ~75.3. Being absolutely cheap still confers most of its advantage when every model is more expensive — but the lead is no longer unconditional.
The right operating model is therefore not a fixed bet on the cheapest provider — it's a pricing-aware routing layer that recomputes value when unit economics move. This page is a snapshot; production should run a router.
Real mean score per task (0–100) across 10 runs per model×task. Darker cell = higher score. The model's row ordering and the avg column re-rank in real time when you switch modes. All numbers come from the official 1,430-run set (Opus 4.7 / 4.8 per-task scores estimated until the audit pivot is rebuilt). The T4 Slack CX column remains the clearest separator in the suite.
"Best overall" uses task-local Quality Core. "Best cost-adjusted" uses task-local Acceptable Value. The fact that these two columns disagree on most tasks is the whole argument for routing.
Click any task to expand the full prompt, definition of done, sources and Pass@1.
The benchmark is open source, and the routing layer it argues for is a download away.
Tasks, prompts, adjudication rules and the full 1,430-run dataset live on GitHub. Audit the methodology, reproduce the numbers, or open a PR to add a model.
View on GitHub G4 OSG4 OS ships the routing layer this page argues for — it sends each task to the right model on this board, so your work gets done faster, at the quality it needs, for the lowest defensible cost.
Download G4 OS