A benchmark for AI models on real non-coding knowledge work, graded by rubric, scored by a mix of human reviewers and AI judges, priced by no-cache tokens.
For most non-coding work, the right default is Balanced Value · one lens of four.
A blended score that rewards what production work actually needs: quality you can defend, cost you can scale, and consistency you can trust. The cost term is gated by consistency — a $0.03 run that crashes 6% of the time loses to a steady $0.09 one.
Each card shows the same fourteen models scored under a different cost-aware lens. The headline is the lens leader; the strip below is the non-dominated frontier (models not beaten on both quality and price). Click a card to read what the lens optimizes for and why it matters.
The Balanced Value Index blends quality, cost efficiency and reliability, gated by consistency so cheap-but-volatile models can't game the board. For most work outside of coding it is the right default ranking. It is not the only one: switch the tabs, or set your own weights below.
Kimi K2.7 Code (88.9) and DeepSeek V4 Pro (88.6) sit at the top of Balanced Value on the same published scale. MiniMax M3 is third at 83.9, ahead of GLM 5.2 (76.2) and GPT-5.5/GPT-5.4 (75.0). GLM 5.2 has the strongest new median (91), while Claude Fable 5 remains the pure Quality Core leader. The right deployment is still a routing policy, not a single-model bet.
This routing layer is what G4 OS ships: every model above in one workspace, routed for quality, speed and cost. Download G4 OS → · Benchmark code and data on GitHub.
Switch the metric tab to re-rank · Balanced is the recommended default for everyday selection.
Quality is 72% of the final score. Cost is 23%, gated by consistency so a cheap run with a failure tail loses to a steadier model. Reliability adds 5%. All rows use the same published scale.
Two views: the range of scores each model produces across 10 runs per model-task, and the operating cost per run. The mean (open circle) and median (filled circle) markers expose each model's consistency.
Agentic work costs more than tokens. Extra steps and longer tool loops add latency, orchestration load, error surface, and user-perceived unpredictability. Drag is a real cost separable from quality and price.
The two cost-aware lenses point at different defaults on purpose. Acceptable rewards cheap reliability for volume; Excellent rewards premium quality that isn't wildly overpriced. The crossover is the trade-off.
Four facts that change which model you should reach for tomorrow — and why a blended score beats any one number.
Today's price gap between US frontier models and China-linked low-cost models is the widest the industry has ever priced. The benchmark would be naive to ignore that. We re-rank the field under two price shocks — holding quality, latency, errors, and tokens fixed.
Only MiniMax and Kimi reprice upward (~2×). Kimi keeps the lead, while MiniMax M3 sits ahead of the GPT pair on Balanced Value. The margin narrows, but low absolute cost still matters.
All models reprice upward; the low-cost pair rises more. Kimi and MiniMax remain ahead of the GPT pair because their starting cost is low enough to absorb a broad price shock.
The right operating model is therefore not a fixed bet on the cheapest provider — it's a pricing-aware routing layer that recomputes value when unit economics move. This page is a snapshot; production should run a router.
Real mean score per task (0–100) across 10 runs per model×task. Darker cell = higher score. The model's row ordering and the avg column re-rank in real time when you switch modes. All numbers come from the official 1,820-run set, including the recovered Opus 4.7 / 4.8 runs. The T4 Slack CX column remains the clearest separator in the suite.
"Best overall" uses task-local Quality Core. "Best cost-adjusted" uses task-local Acceptable Value. The fact that these two columns disagree on most tasks is the whole argument for routing.
Click any task to expand the full prompt, definition of done, sources and Pass@1.
Pick the task, up to three models, and either how often you run it or a plain number of runs. Totals use the median cost per run observed in the benchmark; tags compare your picks on the benchmark scores.
The benchmark is open source, and the routing layer it argues for is a download away.
Tasks, prompts, adjudication rules and the full 1,820-run dataset live on GitHub. Audit the methodology, reproduce the numbers, or open a PR to add a model.
View on GitHub G4 OSG4 OS ships the routing layer this page argues for — it sends each task to the right model on this board, so your work gets done faster, at the quality it needs, for the lowest defensible cost.
Download G4 OS