The Open General Computer Use Agent project is advancing toward a fully open stack for generalist computer use.
OpenGCUA is an open full-stack project for computer-use agents, with OSWorld-v2 results, real-device harness, Cross-Device SDK, and seven online labs.
Key Takeaways
- Alibaba's MAI-UI team introduced OpenGCUA, an open full-stack project for building generalist computer-use agents with community contribution.
- The agent design lets models control how they observe devices, act, and organize work, using accessibility trees, APIs, shell commands, and GUIs only when needed.
- On OSWorld-v2, eight frontier models showed a 19.6-point average score gain and 53% fewer steps, with the harness, prompts, and evaluation trajectories made open-source.
- The release includes a macOS real-device app, a Cross-Device SDK preview for coordinating computers, browsers, and phones, and online interactive labs for exploring the system.
- OpenGCUA is still evolving, with next steps focused on unifying computer and browser use, improving execution efficiency, and expanding real-device benchmarks.
Meta description: Alibaba's MAI-UI team released OpenGCUA, an open full-stack project for generalist computer-use agents, with OSWorld-v2 results and a real-device lab.
Tags: OpenGCUA, computer-use agents, OSWorld-v2, Alibaba MAI-UI, Cross-Device SDK
Featured image: abstract, friendly illustration of a laptop, browser window, and phone linked by a simple control path, with no people, logos, or text.
The MAI-UI team at Alibaba has introduced OpenGCUA, an ongoing project to develop and open the full stack for generalist computer-use agents, built with the community. The first release includes a technical report, the agent design and its harness, evaluation trajectories from 8 frontier models, a real-device harness for macOS, a Cross-Device SDK preview, and an interactive lab that can be tried online.
The project's central idea is that a computer-use agent should do more than take a screenshot and click. In the OpenGCUA design, the agent decides for itself how to observe a device, how to act, and how to organize its work. It can read the accessibility tree, a structured description of app elements, call APIs, run shell commands, and use the graphical interface only when that is necessary. The release presents GPT-6 Astra's computer-use system as a typical example of this approach.
A Design That Lets the Model Control the Workflow
The release describes two linked changes. First, GUI actions are written as code and combined with accessibility, API, and shell operations in a single program. Loops, branches, and intermediate results can then shape what happens next, rather than leaving every action as a separate click or keystroke.
Second, the team moves workflow decisions from fixed harness rules toward the model. In this framing, the harness provides the mechanism, while the model decides when to use it. A concrete example given in the release is that the agent can request a screenshot only when it needs one, instead of being forced to follow a predetermined observation cycle.
Eight Models Gain on OSWorld-v2
The first release reports results on OSWorld-v2 from 8 frontier models, including GPT-6 Astra, Claude Opus 5, and Gemini 3.8 Flash. Across those models, the average score rises by 19.6 points, while the number of steps falls by 53%. The release says the gap between the strongest and weakest model shrinks by more than half.
The team also says the OpenGCUA harness implementation, prompts, and all evaluation trajectories are open-sourced. For readers, that means the result is not presented only as a headline score: the inputs and the path the agent took can be examined, replayed, or compared against other setups.
Real Devices and One Program Across Several Screens
Beyond benchmark environments, the release includes a macOS menu-bar app intended to bring the agent onto a user's own computer. The team says it solved and documented a range of problems that only appear on real devices.
A separate Cross-Device SDK preview connects computers, browsers, and phones through one programming interface, allowing a single program to coordinate several devices within one task. In the real-device tests described by the project, Qwen3.8-Max uses 19.4 times as many reasoning tokens as GPT-6 Astra. The release also says that on real devices, the gap between models is even larger.
Seven Questions, Seven Labs
The online interactive lab is the most direct way to inspect the release. It is organized around seven questions from the report, with one lab for each. Visitors can take a real Mac apart in 3D, compare four ways to act side by side to see what code execution changes, break down challenges unique to real devices, run one program across a phone, a browser, and a Mac, and replay real-device trajectories from five models. The release says the lab works on a phone too, with additional labs described in the release.
What It Means for Builders and What Comes Next
For developers building computer-use agents, OpenGCUA is less a single model than a reusable stack: a design pattern, a harness, evaluation traces, and real-device tooling. The OSWorld-v2 numbers suggest that harness changes can materially affect performance and step counts. The Cross-Device SDK preview points toward work that spans multiple screens, while the lab makes the tradeoffs visible without reading the full report first.
The project remains an ongoing release, not a finished product. The stated next steps are a model that unifies computer use and browser use, systematic improvements to execution efficiency, and real-device evaluation environments and benchmarks. The team invites the community to build it with them.
Learn more: the interactive lab is at tongyi-mai.github.io/OpenGCUA, the technical report is at arxiv.org/abs/2610.22200, and the code is at github.com/Tongyi-MAI/OpenGCUA.
Comments (0)