Two numbers were published five days apart and almost nobody set them next to each other.
OpenAI reports that GPT-6 Astra scores 72.6 percent on OSWorld, the benchmark that asks a model to sit down at a real computer and finish a real task. When OSWorld was published in 2024, its authors measured a human baseline on the same 369 tasks. Humans finished 72.36 percent of them. The best model of that moment managed 12.24 percent.
So the headline is not that Astra improved. It is that a general-purpose model has drawn level with the people it would be replacing at the specific job of operating software. That deserves more care than it has been given, in both directions.
What OSWorld actually asks for
OSWorld is 369 tasks across real desktop and web applications, including file operations and workflows that span more than one program. Each task ships with a scripted starting state and an execution-based checker, so a task is scored by inspecting the machine afterwards rather than by asking another model whether the answer looked right. That is a much harder thing to game than a chat benchmark, and it is why the 2024 numbers were so low.
The tasks are mundane on purpose. Rename a set of files according to a rule. Pull a figure out of a spreadsheet and put it in a document. Change a setting three menus deep. This is the texture of administrative work in any company, which is precisely why the score matters commercially.
The footnote in the version number
The figure OpenAI reports is on OSWorld V2-Offline, and the word offline is carrying weight. In the GUI-agent literature, offline evaluation replays recorded expert trajectories and scores predicted actions against them. Online evaluation puts the agent in a live virtual machine that reacts to whatever it does, including reacting badly. The two measure different things, and research comparing them finds they surface different dominant failure modes entirely.
The honest reading: 72.6 percent offline is not 72.6 percent against your own live Exact instance. Nobody should quote it that way, including us.
The direction, though, is real and it is corroborated by a second number. On ScreenSpot-Pro, which tests only whether a model can find and click the correct element on screen, Astra went from 76.9 to 92.7 percent without external tooling. Grounding, meaning knowing where to click, was the documented blocker in the original OSWorld analysis. That blocker has largely moved.
Why this lands harder in Nieuwegein than in San Francisco
Most Dutch mid-market automation projects do not die on the model. They die on access.
A WMS installed in 2011 whose supplier no longer employs anyone who wrote it. An Exact Globe environment where the export is a scheduled CSV and nothing else. AFAS with a connector that exists but is licensed at a price that removes the business case on its own. A supplier portal where pakbon status lives behind a login and is available to a human eye and nothing else. A school running Magister. For twenty years the honest answer to "can we automate this" was "not for a price that makes sense", and the reason was almost never the intelligence of the software.
Computer use is the one route to those systems that does not require the vendor to agree. That is the actual news in this release, and it is a bigger deal for a company running a fifteen-year-old ERP in Gelderland than for a company whose entire stack has a documented API.
The forty minutes, not the seventy-two percent
The number we would put in a business case is not the success rate. OpenAI reports average time per OSWorld task falling from about 75 minutes to about 40. A task that takes half as long costs roughly half as much to attempt, and for GUI work the attempt is most of the bill.
Set against that: a computer-use agent pays for a screenshot on every step. Astra lists at 10 dollars per million input tokens and 50 per million output, with cached reads at 1 dollar. Images are not a rounding error in that arithmetic, and a long GUI run is hundreds of images. Anyone modelling this should price a real trajectory before promising a saving, not multiply a token rate by a guess.
Where it still fails, and the failure has a name
The dominant failure mode on OSWorld has been characterised in the literature as hidden operation blindness: the agent does not perceive state it did not itself create. A dialog sitting behind the active window. A filter still applied from the previous task. A posting period already closed. The agent proceeds confidently on a screen it has misread.
That is exactly the failure mode that hurts in an ERP or a WMS. A visible failure is inexpensive. A silent one is not. An agent that cannot complete a werkbon and says so costs you a minute. An agent that files it against the wrong project because a filter was left on costs you a quarter-end.
What 72.6 percent does when you chain it
OSWorld scores whole tasks, not steps, so the number should not be compounded naively. But the design lesson survives the caveat. Three genuinely independent tasks at 72.6 percent each give you about 38 percent end to end. Five gives you 20.
The conclusion we draw is structural rather than statistical: one task per agent run, a checkable result at the end of each, and a human gate between any two that depend on each other. The chained autonomous workflow demoed on stage is not the shape this capability supports yet.
The retry gap is a design instruction
One pair of numbers in the release says more about how to build with this model than any single score. On SRE-Bench15, Astra solved 88.0 percent of tasks on the first attempt and 99.2 percent within four. The eleven points between those two figures are the whole engineering brief.
It tells you the model is close to reliable but not reliable, and that retries recover most of the gap. So budget for them, in money and in latency, and then apply the filter that follows: a task you cannot safely attempt four times is not a candidate for this pattern. Sending an email is not retryable. Posting a journal entry is not retryable. Reading a screen is.
The cutoff is April, and in this market that matters
Astra carries a knowledge cutoff of 30 April 2026 and a 1.1 million token context window. The context window gets the attention; the cutoff is the one that will bite a Dutch company, because the European rulebook moved after it. The digital omnibus reshaped parts of the AI Act during the summer, and a model trained to April simply does not know that happened.
The practical rule is unchanged but worth restating, because a larger context window makes people forget it: for anything where a current rule decides the answer, put the text in the prompt. Do not ask the model what the obligation is. Give it the article and ask it to apply it. That is true of every frontier model and it is not a criticism of this one.
Three benchmarks died this month
Astra posts 97.6 percent on FrontierMath Tier 4, 99.9 on ARC-AGI-3 and 100 on ExploitBench. Those are not results so much as obituaries. A benchmark at 99.9 percent has stopped being a measurement and become a formality, because it can no longer separate this model from the next one.
That has a practical consequence for anyone buying. The scores that still discriminate between frontier models are the messy, long-horizon, real-environment ones: computer use, agentic coding, incident response. Those are also the ones with the widest gap between a benchmark harness and your building. So the industry is losing its clean numbers precisely where its remaining numbers are least transferable, and the only honest substitute is a trial on your own data.
What we would automate with this, and what we would not
Worth doing now:
- Read-only extraction from a screen that has no export at all. The agent looks, writes down, and changes nothing.
- Polling a supplier portal for pakbon or CMR status and pushing the result somewhere structured.
- Filling a form whose result a person can verify in a single glance before it is submitted.
Not yet:
- Anything that writes to the financial administration without a second pair of eyes.
- Anything under Wkb or Wtta evidence obligations, where you have to be able to show afterwards what happened and why.
- Anything where a wrong answer looks exactly like a right one until somebody reconciles in March.
Three questions before you point an agent at your ERP
Can you see the failure? If the agent gets it wrong, does something turn red within the hour, or does a wrong value sit quietly in a field? If the answer is the second one, the task is not a candidate yet, whatever the benchmark says.
Is there a read-only version of this task? There nearly always is, and it is where the first month should go. You learn the failure modes on your own screens while the blast radius is zero.
What does the supplier contract actually permit? Several Dutch software suppliers prohibit automated access to their interface in their terms. That is a procurement question, not a technical one, and it is better asked before the build than after.
The short version
A general model has reached the human baseline on a serious computer-use benchmark, in an offline variant, with a grounding score that suggests the improvement is real rather than a harness artefact. For Dutch mid-market companies the consequence is specific: the systems that were never worth automating because they had no usable interface are now worth a second look. Not all of them, and not autonomously. The distance between an offline benchmark and your own live system is the entire engineering problem, and it is the sort of thing an audit answers in weeks rather than an argument settles.
We looked at the rest of the release, including what happened to the reasoning trace and why the identical list price produces a very different bill, in our comparison with Claude Fable 5.1.