A 2026 launch-day benchmark for a new LLM, written from the point of view of a desktop agent that already has to live with it.

The four that survived the 2025 to 2026 transition without going stale or getting saturated are GPQA Diamond (graduate-level science Q&A, resistant to training-set contamination), Humanity's Last Exam (a broad expert-curated set held out from public dumps), SWE-Bench Verified (real GitHub issues with verified test pass/fail), and LiveCodeBench (rolling competitive-programming problems collected after each model's cutoff). Arena Elo from LMSYS sits on top as the human-preference aggregate. Vellum, llm-stats.com, BenchLM, Siliconflow, and clickrank all rank frontier launches on some subset of those numbers. The pages do their job: they tell you which model is currently smartest at clean prompt-in, prompt-out reasoning.

They score the model in isolation. A desktop agent is the model plus an SDK plus a bridge subprocess plus a persisted user preference plus a UI label. The thing that actually breaks on launch day is the seam between those parts. When Anthropic flipped Claude Opus 4.7 to GA on April 22, 2026, GPQA went up and SWE-Bench went up; the @anthropic-ai/claude-agent-sdk also renamed the Opus tier from a string containing 'opus' to the literal alias 'default'. Every shipping desktop agent that stored 'opus' in user defaults silently fell back to the next-best model that did exist in the new list. None of the four benchmarks above predict that, because none of them measure it.

It is a four-axis launch-day number Fazm tracks per frontier release: alias stability (does the SDK report the new tier under the same identifier substring as last time?), tool-payload shape (do existing tool-use frames still deserialise, or has a field been renamed?), persisted-preference migration cost (how many lines of code does it take to map yesterday's stored value to today's available list?), and binary-update requirement (can the new model appear in the picker without an App Store push?). For the Opus 4.7 GA on April 22, 2026, alias stability failed, tool-payload shape held, migration cost was seven lines of Swift, and binary-update requirement was waived because Fazm 2.4.0 had already moved the picker to a dynamic list two days earlier.

It is a static function called normalizeModelId(_:) at /Users/matthewdi/fazm/Desktop/Sources/FloatingControlBar/ShortcutSettings.swift lines 180 to 191. The body is three guarded substring checks. The first two map the legacy haiku/sonnet strings to their short aliases. The third branch, the one the 2.4.2 changelog refers to, is 'if modelId.contains("opus") { return "default" + bracketSuffix }'. The doc comment above the function reads verbatim 'ACP SDK v0.29+ uses default for Opus 4.7; migrate stored opus to match.' Seven lines, three substring branches, one preserved bracket suffix for context variants. That is the entire migration.

Fazm 2.4.0 (shipped April 20, 2026, two days before Opus 4.7 GA) moved the model picker off a static array baked into the binary and onto a JSON-RPC frame named models_available emitted by the acp-bridge subprocess. The flow is three hops. The bridge listens to whatever the ACP SDK currently reports. The bridge emits models_available to the Swift app over the agent connection. The Swift handler at ChatProvider.swift line 1016 routes the frame into ShortcutSettings.updateModels, which calls recomputeAvailableModels and, if the persisted selection no longer matches a row, falls back through normalizeModelId and then a substring upgrade. The benchmark-relevant point is that this is the only architecture that absorbs a frontier launch without a binary release. Anything else has to ship a build per frontier model.

The bridge's DEFAULT_MODEL constant at acp-bridge/src/index.ts line 1920 is currently 'claude-sonnet-4-6'. The dynamic list reported by the SDK across April and May 2026 has included haiku-4-5, sonnet-4-6, sonnet-4-6[1m] (the 1M context variant), opus-4-5, opus-4-6, opus-4-7, and the post-rename 'default' alias. The Fazm modelFamilyMap at ShortcutSettings.swift lines 159 to 164 has four rows: ('haiku', 'Scary', 'Haiku', 0), ('sonnet', 'Fast', 'Sonnet', 1), ('opus', 'Smart', 'Opus', 2), ('default', 'Smart', 'Opus', 2). The last two rows share an order index, which is why a launch-day rename inside the Opus tier does not reorder the picker pills.

Run the four-axis integration test before you run the reasoning test. First, attach the model to the agent bridge under whatever identifier the SDK reports, then send a tool-heavy prompt and inspect the tool-use frames for shape drift against the previous version. Second, write the stored preference 'old_alias', restart the app, and verify the picker migrates it without a settings dialog. Third, count the lines of code your migration required and add that to the launch's score. Fourth, only then run GPQA, SWE-Bench, LiveCodeBench, and Humanity's Last Exam on the model the bridge actually routes to. A model that scores 5 points higher on GPQA but costs you 200 lines of bridge code is a regression for a shipping agent.

It matters anywhere the model identifier is persisted across a process boundary. A scheduled-task runner that resumes after a crash and re-reads the user's preferred model has the same exposure. A CRM automation that stores 'opus' in a row of a Postgres table has the same exposure. A browser extension that caches the picked model in chrome.storage has the same exposure. The reason it shows up most loudly on a desktop agent is that the time between SDK rename and user-visible breakage is roughly two seconds (the time from app launch to the first models_available frame). On a server-side stack you usually find out hours later when alerts fire on the wrong dashboard.

Inside the agent SDK, 'default' is a stable handle that always resolves to whichever Opus tier is currently the highest-quality production model on the account. Renaming the live alias lets Anthropic swap underlying weights from Opus 4.5 to 4.6 to 4.7 to whatever ships next without forcing every consumer app to push a release per swap. The cost of that decision is borne once by every downstream app that holds a versioned legacy ID. The migration function above is the receipt: write the rewrite once, and every future swap inside the same tier is invisible to your users.

Open the floating bar with Cmd+Shift+Space and look at the pill on the right. It says Scary, Fast, or Smart. Behind those three words is a substring map that absorbed at least one frontier launch in April 2026 without anyone clicking anything. The reason you never had to repick the model after the Opus 4.7 GA is that the seven-line function rewrote your stored preference on the next launch, logged a line that reads 'ShortcutSettings: normalized selectedModel to default', and moved on. That is the integration test passing in the wild.