AI agents need millions of computers. Do they actually need to?

Oct 10, 2026 · Technical · Part 2 of 2

In part 1 I worked out what one training attempt costs for an AI that's learning to use a computer. The short version: a few dollars of model tokens, plus one entire computer reserved for the whole attempt, sitting idle more than 96% of the time while the model thinks. Tokens are getting cheaper on their own. The computer isn't.

If you want 10,000 attempts running at once, today's tooling wants 10,000 booted computers. Each with its own Ubuntu, its own Chrome, its own LibreOffice, all restored from the same snapshot, all started from the same screen. They are, to a first approximation, the same computer 10,000 times.

So: how much of the computer actually needs to exist once per attempt? This post walks down the stack one layer at a time and asks, at each layer, what is unique to this attempt and what is identical across a thousand of them. Along the way I'll cover what virtualization already shares (more than you'd think), what it still duplicates, and what the CUA-Sandbox paper found two weeks ago when it tried sharing the running software itself [1].

The top of the stack is already shared

Let's get the easy layers out of the way. The model is one set of weights on a cluster of GPUs. When a thousand rollouts each need a decision, they all send their screenshots to the same model, and it answers them in batches. Nobody runs a thousand copies of the model. The trainer, the thing that collects the scores and updates the weights, is also one program. In the CUA-Sandbox paper's own training run, one update used 128 rollouts in parallel against one model, on one host [1]. That's the normal shape.

The rollouts are, by definition, the things that must differ. Attempt 17 clicked on the wrong menu at step 3 and is now in a different place than attempt 18. The whole purpose of running many rollouts is to get many different trajectories to learn from. So the question isn't whether rollouts can be shared. It's how far below the rollout the "must be different" line actually reaches.

What's actually different between two attempts?

Picture two attempts at the same OSWorld task: "rename Sheet 1 to LARS Resources and make a backup copy." Both start from the same snapshot, the same spreadsheet open in LibreOffice Calc. Attempt A has right-clicked the sheet tab. Attempt B misclicked and opened the Format menu instead. What's different between the two computers right now?

The CUA-Sandbox authors put this precisely. A rollout needs its own mutable state: the files it has written, the database rows it has changed, the browser session it's logged into, what its windows look like. But the initialized runtime, meaning the application code and the already-started programs, is "common across rollouts" and "can remain common" [1]. Their one-line thesis: state is the natural unit of environment independence, not the whole computer.

That's the claim. Now let's see how much of it existing tools already deliver, layer by layer.

Layer: the disk

Start at the bottom. Each VM has a disk image: a big file, tens of gigabytes, holding Ubuntu and all the apps. Does each of 1,000 VMs need its own copy?

No, and this has been solved for a long time. The technique is called copy-on-write. All 1,000 VMs read from one shared base image. When a VM writes something, the write goes to a small private file that sits on top of the base. Reads check the private layer first, then fall through to the base. Nothing is copied until it's changed, and only the changed part is copied. The base is never touched.

How much does this matter in practice? CUA-Sandbox measured it on a 3.6 GB shopping-site database [1]. Making a fresh copy for a new attempt without copy-on-write took 5.5 seconds and 3.67 GB of disk. With copy-on-write it took 0.07 seconds and 0.11 MB. That's 77× faster and about 30,000× smaller, for the same logical result: a database this attempt can scribble on without affecting anyone else.

Verdict for the disk layer: solved. Modern setups share it, and sharing it costs almost nothing.

Layer: the starting state

Next up: getting the computer to the task's starting screen. A snapshot freezes a VM's disk and memory at a moment, so that any number of attempts can be reset to that moment. Copy-on-write makes snapshots cheap to keep and cheap to restore from, because restoring is just "point at the base, start with an empty private layer."

OSWorld's design, which I covered in part 1, leans on this: one shared snapshot of a clean desktop, then a small recipe per task to download a couple of files and open the right app [2]. The alternative, one snapshot per task, would have cost gigabytes each because a snapshot captures the whole memory of the machine. So OSWorld already shares the common starting point and only materializes the per-task difference.

Verdict: mostly solved. Restoring the start is cheap. What's not cheap is what comes next.

Layer: the running programs

Here's where it stops being solved. Disk is what's stored. Memory is what's running. Once you boot Ubuntu, start the desktop, launch Chrome and LibreOffice, those programs occupy memory: their code, their caches, their fonts, their window buffers, their internal data structures. And every VM boots them separately, so every VM holds its own copy of all of it.

This is the number that surprised me. CUA-Sandbox measured the memory per environment for the standard Docker-based setups of three benchmarks, running eight attempts in parallel [1]:

About 2 to 2.7 GB per attempt, of which the attempt's actual private state is a few megabytes. The rest is a copy of a running program that is byte-for-byte the same as the copy next door. On the 1.5 TB host the paper used, that's a ceiling of roughly 550 OSWorld attempts in flight on memory alone, before anything else gets a share. (That last calculation is mine, not the paper's.)

Why doesn't copy-on-write fix this too? Partly it does, a little. Operating systems can share identical memory pages between processes if they came from the same file on disk, so the code of Chrome might be shared across VMs on a good day. But a running program is mostly not its code. It's heaps, caches, decoded images, parsed configuration, open connections, all built up at startup, all living in memory that was allocated separately in each VM. There's no shared base for that. Each copy was constructed independently, so the operating system has no idea they're the same.

Verdict: not solved. This is where the "one computer per attempt" cost actually lives.

What CUA-Sandbox did about it

The CUA-Sandbox paper's move is to stop giving each attempt its own running programs, and instead run the programs once and give each attempt its own state capsule: a private bundle of the things that genuinely need to differ [1]. The web server runs once. The database engine runs once. But each attempt gets its own database branch (copy-on-write, as above), its own session, its own browser profile. Every action carries an environment ID, and a lookup table routes that action's reads and writes to the right capsule.

Three things about the design that I think are the important ones:

• They sort state into three kinds. Authoritative state, like databases and user files, is kept as a private copy-on-write delta over a shared base. Derived state, like caches and search indexes, gets a private namespace and is rebuilt if rebuilding is cheaper than copying. Ephemeral state, like display sockets and temporary IPC endpoints, is simply recreated when a capsule becomes active [1].
• Anything they can't cleanly route stays private. If some resource can't be given a per-attempt binding, it isn't forced into the shared runtime. It gets its own private backend, and sharing drops a bit. The isolation rule is never relaxed to gain efficiency [1].
• Switching state is a transaction. To reset or fork an attempt, they freeze its route, let in-flight requests drain, stage the new capsule, and only then publish it atomically. Anything still carrying the old generation number gets rejected [1]. More on why in a moment.

What they measured

They compared their setup against the official Docker-based environments for WebArena, VisualWebArena, and OSWorld, running eight attempts in parallel on the same host, with the same models, prompts, step limits and graders [1].

And the thing that actually makes the paper useful rather than just impressive: task success didn't drop. Across four models and three benchmarks, twelve pairs, the shared-runtime version scored the same or slightly higher every time [1]. They also trained a small model end-to-end on both setups, supervised fine-tuning then RL, and the gains from RL were within a tenth of a point of each other (3.88 versus 3.87 points on OSWorld). The cheaper environment learned the same lessons.

The ablation that explains everything

The most clarifying table in the paper is a three-way comparison on the web benchmark [1]. They ran eight attempts three ways: the official Docker setup; Docker plus copy-on-write for the database; and copy-on-write plus one shared runtime.

This is the cleanest statement I've seen of where the duplication lives. Copy-on-write is a storage trick, and it does its job: writable disk fell from 53 GB to 25 GB. But memory only dropped from 15.6 GB to 13.9 GB, because the eight web servers and eight database engines were still eight separately booted programs. Sharing the runtime is what collapsed memory, to 3.3 GB. And notice the red bar: the agent's own browser processes, the thing taking screenshots and clicking, stayed at about 7 GB no matter what. Nobody has shared those yet.

What breaks when you share more

This is the part the short version of this story usually skips, and it's the part I care about most, because it's where the real engineering is.

Problem 1: time travel

Suppose attempt 5 is being reset. Its old database branch is about to be swapped for a fresh one. Meanwhile, a request that attempt 5 sent a moment ago is still in flight inside the shared web server. If the swap happens mid-request, that request might read from the old branch and write to the new one, or the grader might inspect a half-swapped state. CUA-Sandbox calls this a mixed generation. They stress-tested it: with a naive swap, it happened in 100 out of 100 trials. With their freeze-drain-publish protocol, 0 out of 100 [1].

Problem 2: the stuff that isn't a database

Databases are the easy case: they already have branches and transactions. Browser profiles, cookies, caches, search indexes, the display, the inter-process message bus that desktop apps use to talk to each other, these all need their own adapter to route per attempt, and the adapters cost time. On one read-only web task, managing the non-database state added 0.84 seconds to setup and 1.09 seconds to every reset [1]. Not huge, but not free, and the list of things that need adapters grows with the richness of the environment.

Problem 3: the desktop is the hard case

Look back at the results chart. On the web benchmarks, memory per attempt dropped 4.5× to 9.2×, and on one site (GitLab) 51×. On OSWorld, the desktop benchmark, it dropped 3.2× overall and only 1.45× on the desktop-specific measurement in the paper's Table 3 [1]. Why the gap?

Because a website is a service with a clean seam between "the server" and "this user's data." A desktop isn't. The display server, the window manager, the open LibreOffice window with its document loaded, the message bus, the clipboard: these are one tangled session, and the session is the state. CUA-Sandbox's desktop adapter keeps "filesystem, application-profile, configuration, display, message-bus, and window state" private per attempt, and anything whose in-memory state "cannot be safely rebound" stays in a private backend [1]. That's honest, and it means on a desktop most of the running session is still duplicated. The paper's own closing line says that "broader mutation workloads, longer trajectories, and concurrency sweeps are needed to characterize the remaining sharing boundary."

Back to the question

Here's the accounting the paper gives, in plain words [1]. With N attempts in the conventional design, memory is N × (runtime + per-attempt execution overhead) + the sum of everyone's private state. With a shared runtime it's 1 × runtime + Q × execution overhead + the same private state, where Q is the number of attempts that are actually active at a given moment, which can be less than N because attempts that are waiting on the model don't need a live execution slot. The savings are (N − 1) runtimes plus (N − Q) idle slots.

The last point connects the two posts. Part 1 found that during a rollout, the computer is idle over 96% of the time, waiting for the model. If attempts don't each need a permanently reserved execution slot, if a slot can be handed to whichever attempt has an action to run right now, then the number of live slots you need is set by how many attempts are acting at this instant, not how many are in progress. CUA-Sandbox's design allows this (they call it reusing execution slots), though they don't push on it in the measurements.

Where I've landed

Modern virtualization already does a surprising amount of sharing. Copy-on-write disks, snapshots, deduplicated base images: these mean a training farm isn't as naive as "one complete computer per agent." Disk and starting state are effectively solved.

What isn't shared is the running software. Every attempt boots its own copy of the same programs into memory, and that memory, 2 to 2.7 GB per attempt in the measurements above, is what caps how many attempts a machine can hold. CUA-Sandbox showed that for websites you can collapse most of that by running the programs once and routing each attempt to its own state, with no loss in what the agent learns. For desktops, the win is real but smaller, because a desktop session doesn't have a clean seam between the shared part and the private part, and the paper's authors say so.

But computer-use RL is an unusual workload. It's an enormous number of independent environments, repeatedly starting from highly similar states, executing many of the same applications, and spending almost all their time waiting. That makes me wonder whether the right abstraction isn't a faster disposable VM, but a computer designed from the start around parallel rollouts: one that treats the running session itself the way copy-on-write treats a disk, so that a thousand attempts at "rename Sheet 1" share one booted LibreOffice until the moment they diverge.

I don't know yet how far that can be pushed on a real desktop, or where it breaks. That's what I'm working on with Drums, and the next post in this series will be measurements, not reading: one physical Mac, an increasing number of attempts, and a graph of where it falls over.

Vocabulary

• Rollout / attempt. One complete try at one task, from a known starting state to a score.
• Runtime. The running programs: the web server, the database engine, the desktop apps, once they're started and loaded into memory.
• State. The data an attempt has changed: files, database rows, sessions, what's on screen.
• Copy-on-write (CoW). Share one base; copy a piece only when someone writes to it, and only that piece.
• Snapshot. A frozen copy of a VM's whole state that it can be reset to.
• Container (Docker). A lighter-weight box than a VM: shares the host's kernel but has its own files and processes. Still boots its own copy of every program inside.
• State capsule. CUA-Sandbox's term for an attempt's private bundle of state, separate from the shared runtime.
• Generation. A version number on an attempt's state; bumps on every reset or fork, so stale requests can be recognized and rejected.

References

1. Yan, Jiao, et al., CUA-Sandbox: Efficient Environments for Computer-Use Agent Reinforcement Learning, 2026. arXiv:2609.32750
2. Xie et al., OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments, 2024. arXiv:2404.07972
3. Abhyankar, Qi, and Zhang, OSWorld-Human: Benchmarking the Efficiency of Computer-Use Agents, MLSys 2026. arXiv:2506.16042
4. Dai et al., OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software, 2026. arXiv:2609.39903

All figures are my redrawings of numbers reported in those papers, cited inline. The "550 attempts on a 1.5 TB host" figure is my arithmetic from the paper's per-attempt memory and host description, not a measurement.