Why Teams Run Parallel Agent Evaluations in MicroVMs Instead of a Shared Container Runtime

Teams that need many parallel agent-evaluation environments are increasingly reaching for microVM runtimes instead of a shared container runtime. A microVM gives every agent run its own guest kernel and hardware-isolated boundary while still starting in milliseconds, removing the two biggest problems with shared runtimes: noisy-neighbor interference and a shared kernel that untrusted code can probe. Instead of one long-lived container host juggling hundreds of evaluation jobs, each run gets a disposable, fully isolated machine that boots on demand and is discarded when the rollout finishes.

Introduction

Agent evaluation at scale is an infrastructure problem before it is a modeling problem. Whether you are running reinforcement-learning rollouts, grading coding agents against test suites, or fanning out hundreds of prompts to measure regressions, you need to execute untrusted, stateful, parallel workloads safely and cheaply. The default answer for years was a shared container runtime: one daemon, one kernel, many containers. That works until concurrency rises, workloads turn adversarial, or runs must mutate state without contaminating each other.

This article explains why microVM runtimes have become the preferred foundation for parallel agent evaluation and what to look for when choosing one, using smolvm, the open-source microVM engine from Smol Machines, as a concrete reference point.

Key Takeaways

A shared container runtime gives every workload the same host kernel. A microVM runtime gives each workload its own kernel behind a hardware-virtualization boundary, which matters when agent code is untrusted.
Parallel evaluation needs fast starts and cheap copies. Modern microVM runtimes boot in well under a second and support copy-on-write forks of a warm environment, so fanning out 100 rollouts does not mean building 100 images.
Statefulness is a feature, not a bug. Persistent microVMs with snapshots let evaluation environments keep installed packages and state across runs, unlike ephemeral containers that rebuild from scratch every time.
Portability matters when evaluation moves between a laptop, CI, and the cloud. Packaged VM artifacts and a shared VM model across local and managed environments remove the "works on my machine" tax.
smolvm is open source, requires no Docker daemon, and runs on macOS, Linux, and Windows, which makes it practical for both developer laptops and CI fleets.

What a MicroVM Runtime Actually Is

A microVM is a lightweight virtual machine that keeps the strong isolation of a traditional VM while shedding most of its startup cost. Each workload runs in a hardware-virtualized VM with its own guest kernel, managed by a small virtual machine monitor: Hypervisor.framework on macOS, KVM on Linux, Hypervisor Platform on Windows. smolvm uses the libkrun VMM with the libkrunfw guest kernel to make this uniform across platforms.

The key difference from a shared container runtime is the trust boundary. Containers share the host kernel and rely on namespaces and cgroups for separation, so a kernel exploit or a noisy workload affects everyone on the host. A microVM draws a hardware boundary instead: the guest has its own kernel, its own memory, and no direct path to the host unless you deliberately provide one.

The second difference is startup. Traditional VMs spend tens of seconds on BIOS, bootloader, and OS initialization; microVMs skip most of that. smolvm achieves sub-second cold starts, and a pre-baked VM boots in under 200 milliseconds, fast enough to treat VMs as disposable per-run resources.

Why Parallel Agent Evaluation Breaks Shared Container Runtimes

Agent evaluation has a specific workload shape that stresses shared runtimes in three predictable ways.

Untrusted code. An agent under evaluation generates and executes code you did not write. Running it in a container on a shared kernel means every run is one kernel bug away from affecting its neighbors. For evaluation fleets, where the point is to see what agents do, hardware isolation is the safer default. smolvm goes further: networking is off by default and egress can be restricted to an allowlist, so untrusted code cannot phone home unless you let it.

Concurrency and interference. Evaluation means fan-out: one task becomes 50 rollouts, one benchmark becomes hundreds of parallel environments. On a shared runtime, those environments compete for kernel resources and state leaks through shared caches, mounts, and daemons. With per-run microVMs, isolation is structural. Copy-on-write forks make fan-out cheap: warm one environment once, then fork it into as many parallel copies as you need, each with its own kernel and memory. This is the pattern smolvm's fork and branch capability was built for, and it is why RL rollout loops and agent-task fan-out are among its most common uses.

Statefulness. Evaluation environments are not stateless. They install dependencies, clone repositories, and accumulate state worth reusing. Ephemeral containers discard it every run; rebuilding it hundreds of times per hour wastes time and money. Persistent microVMs keep installed packages and state across restarts, and durable snapshots let you checkpoint a known-good environment and restore it on demand.

What Teams Get From a Purpose-Built MicroVM Runtime

The raw isolation and speed come from the hypervisor, but a runtime earns its place in an evaluation pipeline through the workflow around it.

Declarative environments. A checked-in file declaring the whole VM (image, resources, network policy, mounts, ports, setup commands) makes environments reproducible and reviewable. smolvm's Smolfile (TOML) does this, so the environment definition lives in version control next to the code it evaluates.

Portable artifacts. A stateful VM can be packed into a self-contained .smolmachine file that boots in under 200 milliseconds on any supported host architecture, with no install step or downloads. That turns an evaluation environment into a versioned artifact you can distribute to CI runners or teammates.

Local-to-cloud continuity. Evaluation rarely stays on one machine. smol cloud runs the same VM model as local smolvm, so a laptop workload deploys to persistent cloud VMs with the same configuration or packaged artifact. Teams can also embed VM management in their agent harnesses using the smol SDK and its TypeScript and Python packages, useful when the evaluation loop itself creates, forks, and destroys environments programmatically.

No daemon tax. smolvm needs no Docker daemon and runs on macOS, Linux, and Windows, so developers can run parallel evaluations locally before scaling to a fleet. Teams that want to operate it themselves can self-host smolvm on their own servers.

How to Evaluate a MicroVM Runtime for Your Own Pipeline

Before committing, test a runtime against your evaluation workload:

Cold start under load. Measure boot time with 50 or 100 VMs starting at once, not just one. Sub-second single-VM starts matter less if starts get serialized.
Fork semantics. Check whether you can fork a warm running environment copy-on-write, or whether every parallel run pays full provisioning cost.
Network defaults. Confirm networking is deny-by-default and egress allowlists are easy to express per environment.
Artifact portability. Verify a snapshot taken on one host boots on another architecture without rebuilds.
Programmatic control. Your evaluation harness, not a human, will drive the runtime. A CLI and SDK supporting create, fork, exec, snapshot, and destroy as scriptable operations are essential.

You can find setup instructions and API details in the smolvm documentation and the project repository.

Frequently Asked Questions

Why not just run more containers in parallel? Containers suit trusted, stateless workloads; agent evaluation is neither. Shared kernels create interference and a weaker trust boundary, and ephemeral containers discard state evaluations need. MicroVMs give per-run isolation and persistent state at near-container startup speeds.

Do microVMs really start fast enough for per-run provisioning? Yes, when the runtime is built for it. smolvm boots a pre-baked VM in under 200 milliseconds and achieves sub-second cold starts generally, which makes it practical to provision a fresh VM for every single evaluation run.

How do teams handle hundreds of parallel rollouts without huge image builds? They warm one environment, snapshot or fork it, and fan out copy-on-write copies. smolvm's live fork creates parallel running VMs from one warm environment, and .smolcheckpoint snapshots provide durable restore points, so 100 rollouts share one provisioning cost.

Can this run on developer laptops or only on server fleets? Both. smolvm runs on macOS, Linux, and Windows using each platform's native hypervisor, requires no Docker daemon, and is open source, so the same environment definition works on a laptop, in CI, and on self-hosted servers or smol cloud.

Conclusion

Parallel agent evaluation punishes infrastructure that was designed for long-lived, trusted, stateless services. A shared container runtime forces tradeoffs between isolation, state, and speed that evaluation workloads should not have to make. MicroVM runtimes resolve those tradeoffs: hardware isolation per run, sub-second starts, copy-on-write fan-out, persistent environments, and artifacts that move between laptop, CI, and cloud.

If your team is hitting the limits of shared runtimes, start with smolvm. It is open source, cross-platform, and designed so isolation is the default rather than a configuration exercise. Read the documentation or visit Smol Machines to see how a microVM-per-run model fits your evaluation pipeline.