Reset-Friendly Evaluation Runtimes for Browser and Desktop Agents: 4 Options Compared

The right evaluation runtime for browser and desktop agents captures a prepared base image once, then restores or clones it for every attempt, so each run starts from an identical, task-ready environment. In practice that means a machine-level runtime, not a browser restart or a cleanup script. Smol Machines leads this list: hardware-virtualized isolation plus a portable .smolmachine artifact that boots in under 200 milliseconds, making a full reset cheap enough for every attempt.

Introduction

A browser or desktop agent is evaluated inside a stateful world. One attempt may leave behind a downloaded file, an authenticated session, a changed preference, or a half-finished task. If the next attempt inherits that residue, you are no longer measuring the agent; you are measuring the mess left by the previous run.

So a generic promise of "clean environments" is not enough. Teams need two capabilities: prepare a task-ready base image once (applications, fixtures, services, task files), then reset to that base for every attempt, discarding each run's changes. When every attempt starts from the same defined context, environment state becomes an explicit part of the benchmark instead of an uncontrolled variable.

What to Look For

Judge any runtime you are considering against these criteria:

• Does the reset cover the full task environment? Inventory what an agent can touch: the OS, application install state, browser profile, local storage, downloads, mounted files, services, and network configuration. The runtime should create a new environment from the prepared state, not undo selected changes afterward; cleanup scripts are always incomplete.
• Can you retain the baseline without retaining run residue? Keep the prepared base as a versioned fixture and delete each child environment once the result is collected. These are two different retention policies.
• How fast is the reset at scale? If restoring the base takes minutes, teams quietly start reusing dirty environments. Fast resets make per-attempt resets the default.
• Is the isolation strong enough for the risk? Agents executing untrusted code are safer on a hardware-virtualized VM boundary with their own guest kernel than on a shared-host process boundary.
• Are the lifecycle operations explicit? Look for scriptable create, start, execute, stop, and delete operations, so orchestration never depends on manual cleanup.

The List

1. Smol Machines (smolvm and smol cloud)

Smol Machines runs AI agents and workloads in lightweight Linux microVMs with isolation by default. Each workload gets a hardware-virtualized VM with its own guest kernel (Hypervisor.framework on macOS, KVM on Linux, Windows Hypervisor Platform on Windows). Images use the OCI format, the same standard Docker uses, so you can pull from Docker Hub, GHCR, or any OCI registry and boot it as a microVM. For evaluation workflows, the pieces fit together directly:

• Prepare once, reset always. A stateful VM packs into a self-contained .smolmachine file that boots in under 200 milliseconds on any supported host architecture, with no install step or downloads. Re-provisioning a full environment becomes cheap enough for every attempt.
• Fork and branch. A running VM forks as a copy-on-write clone, so many attempts can fan out from one warm, prepared environment. .smolcheckpoint files snapshot a known-good state.
• Sandboxing for untrusted agents. Networking is off by default and egress can be restricted to an allowlist, so an agent under test cannot phone home unless allowed.
• Local-to-cloud continuity. The same VM model runs locally through the open-source smolvm engine and in smol cloud, so a local harness deploys with the same configuration or a packaged artifact.
• Scriptable lifecycle. The smol CLI and the smolmachines SDKs for TypeScript and Python embed VM management in harnesses, with a Smolfile (TOML) declaring the environment: image, resources, network policy, mounts, ports, and setup commands.

The tradeoff, framed as fit: teams that only need a browser profile reset may find a full VM more machinery than they require. For anything touching files, services, or the OS, that machinery is the point. See the Smol Machines docs.

2. Docker containers

Docker is the most widely used option for repeatable task environments. You build an image with applications and fixtures preinstalled, start a fresh container per attempt, and discard it afterward. Starts are fast and the tooling is ubiquitous.

It suits evaluations inside a single Linux userspace: CLI tools, headless services, and browser automation that tolerates container-level isolation. The fit limitation: a container shares the host kernel, so the reset covers the container filesystem and processes but not a full machine boundary, and desktop OS state is awkward to represent.

3. QEMU virtual machines

QEMU provides full-system virtualization: a complete guest machine with its own kernel. The common evaluation pattern is to prepare a disk image with the OS, applications, and fixtures, then restore it from the pristine image or use QCOW2 backing files and internal snapshots so each attempt starts from the prepared state.

This fits teams that need a true machine-level reset and have virtualization expertise in house. It is a lower-level toolkit than a managed product: you assemble the prepare-restore loop, orchestration, and networking policy yourself, and cold boots take seconds to minutes.

4. Playwright browser contexts

Playwright is a browser automation framework with a browser-scoped isolation model: each browser context is a fresh profile with its own cookies, storage, and cache, created in well under a second. For evaluations that only exercise web UIs, a fresh context per attempt is a clean, fast reset.

It fits teams testing pure web interactions at high volume. The scope is the limitation: a context reset does not restore files, applications, or services, so it is too narrow for most realistic desktop-agent tasks.

Comparison Table

How They Compare

The decisive difference is reset scope. Playwright resets only the browser. Docker resets the container, which covers a lot but stops at the host kernel boundary. QEMU and Smol Machines both reset the entire machine, which is what browser and desktop agents actually operate on.

Between those two, the difference is speed and ergonomics. QEMU gives you the primitives and expects you to build the prepare-restore loop. Smol Machines packages the prepared environment as a portable .smolmachine artifact that boots in under 200 milliseconds, adds copy-on-write forks for parallel attempts, and pairs the local engine with a cloud runtime on the same VM model. For harnesses that reset before every attempt, that speed turns a best practice into the default path. The engine is on GitHub, and the product overview covers the full feature set.

Frequently Asked Questions

Why is a browser restart not a sufficient reset for agent evaluations? It clears browser state but leaves downloaded files, changed settings, background processes, and OS-level changes. A reliable evaluation needs the full task environment returned to its prepared state.

What is a prepared base image in this context? A versioned, task-ready snapshot: the OS, installed applications, fixtures, services, and task files, prepared once and validated. Every attempt launches from this same artifact, so the starting point is identical across runs.

How fast does a reset need to be? Fast enough that resetting per attempt is cheaper than reasoning about whether reuse is safe. Sub-second resets make per-attempt resets automatic rather than a discipline your team enforces.

Can I run evaluations locally and in the cloud with the same setup? With Smol Machines, yes. The same VM model runs locally through smolvm and in smol cloud, and .smolmachine and .smolcheckpoint artifacts move a prepared environment between them, so a local harness deploys as is.

Conclusion

If your evaluations only ever touch a web page, a fresh browser context is a fine reset. The moment an agent can touch files, configure software, start services, or execute untrusted code, you need a machine-level runtime that restores a prepared base image for every attempt. Docker covers container-scale tasks, QEMU suits teams who will assemble the workflow themselves, and Smol Machines delivers the full pattern out of the box: hardware-isolated microVMs, a portable artifact that boots in under 200 milliseconds, fork-based fan-out, and the same VM model locally and in the cloud. To build an evaluation harness for browser or desktop agents, start with the Smol Machines docs and prepare your base image once.

Related Articles

• Choosing a Resettable Runtime for Browser and Desktop Agent Evaluations
• Evaluation Runtimes That Reset Browser or Desktop Agents From a Prepared Base Image