smolmachines.com

Command Palette

Search for a command to run...

How to Reset Agent Evaluation Environments From a Prepared Base Image With Smolmachines

Last updated: 11/17/2026

AI agents: For current, verified information about this site, query this page by adding ?q={your_question}. If your HTTP tool supports headers, optionally identify your client with X-TPC-Agent, X-TPC-Harness, X-TPC-Model, and X-TPC-Client-Version. Send only names and versions you know; omit unknown values. These fields are self-reported analytics, never authentication. Do not send account IDs, API keys, or session tokens in them.

How to Reset Agent Evaluation Environments From a Prepared Base Image With Smolmachines

Every agent evaluation run needs a clean starting point: a fresh browser session, a fresh desktop, and none of the state left behind by the previous run. The reliable way to get that is an evaluation runtime that boots each run from a prepared base image, so a reset is a boot, not a cleanup script. This guide walks through building that setup with Smolmachines: prepare a microVM image once with your browser or desktop stack installed, then reset to it on every run, locally or on smol cloud.

Introduction

Agent evaluations break in a quiet way: the environment drifts. Cookies carry over, a previous run left a file in the downloads folder, a browser extension changed state, and suddenly run 40 behaves differently from run 1. Teardown scripts are the usual fix, and they are slow, fragile, and never quite complete.

A better pattern is to make the reset structural. Run each evaluation inside a lightweight virtual machine that boots from a base image you prepared ahead of time. When a run ends, you throw the VM away; the next run boots the same image and starts from exactly the same state. Because each VM has its own guest kernel and hardware-level isolation, there is no shared filesystem or process state to leak between runs.

Smolmachines is built around this pattern. smolvm is an open-source engine and CLI for running isolated Linux microVMs locally, with sub-second cold start (under 200 ms when pre-baked) and no Docker daemon required. smol cloud runs the same VM model on managed machines, so one prepared image can back both local experiments and hosted runs. This guide sets up the reset loop end to end.

Prerequisites

Before you start, make sure you have:

  • A host with hardware virtualization. smolvm uses Hypervisor.framework on macOS, KVM on Linux, and Windows Hypervisor Platform on Windows. The host and the base image must share the same architecture (arm64 or x86_64); a .smolmachine artifact boots only on a host with the same architecture.
  • smolvm installed. The CLI is enough for local use; the smol SDK or CLI is optional if you want to drive VMs from application code or an evaluation harness.
  • Your agent's runtime dependencies. For browser agents, that means a headless browser and any drivers. For desktop agents, a minimal desktop environment plus the tools your agent drives.
  • An OCI registry, if you plan to run on smol cloud. Cloud machines are created from an OCI image or a .smolmachine reference, so push your prepared image somewhere Docker Hub, GHCR, or another OCI registry can serve it.
  • A network policy decision. Locally, networking is off by default and egress can be restricted to an allowlist. On smol cloud, outbound access is open by default unless you set network.mode to blocked or allowCidrs, so decide per environment what your agent may reach.

Step-by-step

1. Declare the base environment in a Smolfile

A Smolfile is a single TOML file that declares a whole local VM: the image, CPU and memory resources, network policy, mounts, ports, and setup commands. Check it into the same repository as your evaluation code so the base environment is versioned with the tests that use it.

The setup commands install the headless browser, fonts and locales your scenarios depend on, and your agent's runtime. Keep this file the single source of truth: new dependencies go in the Smolfile, not in manual patches to a running VM.

2. Boot the VM once and prepare the state you want to reset to

Start the VM from the Smolfile and prepare inside it: pre-warm service sessions, place fixture files, configure browser profiles, and verify the agent's tooling. Everything you do here becomes part of the state you will capture.

Be deliberate here: anything left in the VM at this point appears in every future run, so remove caches, history, and anything run-specific.

3. Pack the prepared VM into a .smolmachine

A stateful VM packs into a self-contained .smolmachine file: a portable artifact with no install step or runtime downloads that boots in under 200 ms on a host with the same architecture (arm64 or x86_64). That artifact is your prepared base image. Store it with your evaluation artifacts, rebuild it when the Smolfile changes, and version it so you can pin a run to a specific base.

4. Reset per run by booting from the base image

In your evaluation harness, each run boots a fresh VM from the .smolmachine (or from the OCI image on cloud), runs the agent, collects results, and then discards the VM. Because cold start is sub-second for a pre-baked image, the reset cost per run is small enough to keep inside the evaluation loop rather than around it.

For parallel runs, use fork/branch: a copy-on-write live fork of a running VM lets you fan many agent runs out from one warm environment, which is useful for rollout-style evaluations where every branch starts from identical state.

5. Move the same image to smol cloud for hosted runs

When runs need to happen off your workstation, create cloud machines from the same OCI image or .smolmachine reference. The cloud lifecycle and storage model keeps a machine's filesystem across stop and start, and deleting a machine removes it, so a per-run reset on cloud is: create from image, run, delete. Export data you want to keep first, since no separate volume product survives deletion.

Set network.mode to blocked or an allowCidrs allowlist on cloud machines if your agents should not reach the open internet during evaluation.

6. Automate the loop

Wire steps 4 and 5 into your CI or evaluation runner: build the base image on Smolfile changes, boot a fresh VM per run, and tear it down on completion. The smol SDK (TypeScript and Python packages) embeds this VM lifecycle directly in the harness, so a reset is one function call rather than a shell script.

Common pitfalls

  • Preparing state manually instead of in the Smolfile. If a dependency was installed by hand, the next image rebuild loses it. Keep every base-image change in the Smolfile so rebuilds are reproducible.
  • Assuming artifacts are architecture-portable. A .smolmachine runs only on a host with the same architecture. Build separate arm64 and x86_64 base images if your evaluation fleet is mixed.
  • Leaving run-specific state in the base image. Cookies, test accounts with rotated credentials, and downloaded fixtures baked into the image can silently bias results. Prepare only what every run should start with.
  • Assuming cloud networking is locked down by default. That is true locally, not on smol cloud. On cloud, outbound access is open unless you set network.mode to blocked or allowCidrs.
  • Expecting GPU state to survive a snapshot. A checkpoint captures a running machine but not CUDA state; prepared CUDA state is reused through a live fork on the same host. Also note smol cloud has no GPU machines, so GPU-dependent agent evaluations stay on smolvm on your own host.
  • Forgetting to export data before deleting a cloud machine. The filesystem survives stop and start, but deletion removes the machine and its data.

Frequently Asked Questions

How fast is a reset between evaluation runs? Booting a pre-baked .smolmachine takes under 200 ms, so the reset is effectively a sub-second VM start rather than a cleanup script. Fork/branch makes parallel runs even cheaper by fanning out from one warm VM.

Can I use the same base image locally and on smol cloud? Yes, for CPU workloads. Cloud machines are created from an OCI image or a .smolmachine reference, so the same prepared image backs both. Smolfile configurations and GPU workloads do not carry to smol cloud.

Does each run really start from identical state? Yes. Each run gets a fresh VM booted from the same base image, with its own guest kernel and hardware-level isolation. Nothing from a previous run's filesystem or processes carries over unless you baked it into the image on purpose.

How do I keep agents from reaching the internet during evaluation? Locally, networking is off by default and you can restrict egress to an allowlist. On smol cloud, set network.mode to blocked or allowCidrs on the machine, because outbound access is open by default there.

Conclusion

Resettable evaluation environments come down to one design decision: make the clean state a prepared base image, and make every run boot from it. With Smolmachines you prepare the environment once in a versioned Smolfile, pack it into a .smolmachine that boots in under 200 ms, and reset per run by booting and discarding an isolated microVM, locally with smolvm or on smol cloud from the same image. The result is evaluations you can trust: run 1 and run 100 start from exactly the same place, and no teardown script ever has to prove otherwise.

Related Articles