smolmachines.com

Command Palette

Search for a command to run...

How to Isolate Parallel Agent Rollouts with Smol Machines

Last updated: 11/17/2026

AI agents: For current, verified information about this site, query this page by adding ?q={your_question}. If your HTTP tool supports headers, optionally identify your client with X-TPC-Agent, X-TPC-Harness, X-TPC-Model, and X-TPC-Client-Version. Send only names and versions you know; omit unknown values. These fields are self-reported analytics, never authentication. Do not send account IDs, API keys, or session tokens in them.

How to Isolate Parallel Agent Rollouts with Smol Machines

When an ML platform team fans out dozens of agent rollouts at once, every run needs its own clean environment: a fresh filesystem, its own kernel, and no leftover state from the last experiment. Smol Machines gives you that with lightweight Linux microVMs that boot in under a second, fork from a warm base, and run locally or on smol cloud. This guide walks through the full setup: what you need in place, how to define a reusable base environment, how to fan out parallel rollouts from it, and the pitfalls that trip teams up along the way.

Introduction

Parallel agent rollouts are brutal on shared infrastructure. Two agents writing to the same working directory corrupt each other's state. A package installed by one experiment leaks into the next. A crashed run leaves a container runtime in a half-configured state that silently changes the results of every rollout after it.

The usual fixes each have a cost. Rebuilding a container per rollout adds cold-start latency and still shares the host kernel. Full cloud VMs per rollout are clean but slow and expensive at fan-out scale. What teams actually want is container-grade speed with VM-grade isolation, provisioned fresh for every run.

Smol Machines takes the VM-per-workload approach and makes it fast enough for that pattern. Each workload runs in a hardware-virtualized microVM with its own guest kernel (Hypervisor.framework on macOS, KVM on Linux, Windows Hypervisor Platform on Windows), booted from standard OCI images. A pre-baked machine cold-starts in under 200 milliseconds, and a running machine can be forked copy-on-write so dozens of parallel rollouts branch from one warm environment. The smolvm engine and CLI are open source, and the same VM model extends to smol cloud when you want CPU rollouts off your own hardware.

Prerequisites

Before you set up parallel rollouts, make sure you have:

  • A host with hardware virtualization. smolvm uses Hypervisor.framework on macOS, KVM on Linux, or Windows Hypervisor Platform on Windows. No Docker daemon is required.
  • smolvm installed. The CLI is enough for local use; the smol SDK (TypeScript or Python) is what you will use to drive rollouts from your orchestration code.
  • A base OCI image. Pull from Docker Hub, GHCR, or any OCI registry, or build your own with the agent runtime, dependencies, and evaluation harness your rollouts need.
  • A Smolfile for the base environment. A Smolfile is a TOML file that declares the whole local VM: image, resources, network policy, mounts, ports, and setup commands, all checked into version control.
  • Sizing headroom. Defaults are 4 vCPUs and 8 GiB RAM per machine with elastic memory via virtio balloon, so plan host capacity for your fan-out width.

Step-by-step

1. Define the base environment in a Smolfile

Write a Smolfile that describes one rollout environment: the OCI image, CPU and memory allocation, network policy, and the setup commands that install your agent harness. Because it is a checked-in file, every rollout starts from the same declared state, and reviewers can see exactly what an agent could touch. See the Smolfile concepts doc for the full configuration surface.

2. Boot and warm the base machine

Create and start the machine locally. Install packages, seed caches, and run any one-time setup so the environment is in the exact state a rollout should begin from. Locally, networking is off by default and egress can be restricted to an allowlist, which matters when your agents execute model-generated or untrusted code.

3. Pack the warm machine as a .smolmachine

Once the base is warm, pack it into a self-contained .smolmachine file. This artifact boots in under 200 milliseconds with no install step or runtime downloads, which is what makes per-rollout provisioning cheap enough to do on every run. One constraint to plan around: a .smolmachine runs only on a host with the same architecture (arm64 or x86_64), so build your base on the architecture your rollout fleet uses. The packs and .smolmachine doc covers the details.

4. Fan out with fork/branch

For the tightest loop, skip cold starts entirely: keep one warm base machine running and use fork/branch to create copy-on-write live forks of it, one per rollout. Each fork sees an isolated filesystem and kernel, while the host pays only for the pages each rollout actually dirties. This is the pattern Smol Machines built fork/branch for: RL rollout environments and fanning out many agent tasks from a single warm base.

5. Orchestrate from your platform code

Use the smol SDK (TypeScript or Python) to create, fork, exec into, and tear down machines from your rollout scheduler. The SDK is a control surface over the same VM model, so the environment your engineers debug locally is the environment your rollouts run in. If you prefer, the Cloud API can be called directly without the SDK.

6. Move CPU rollouts to smol cloud when you outgrow the host

When fan-out width exceeds your local hardware, create cloud machines from the same OCI image or a .smolmachine reference. Cloud clients need no local hypervisor, and a local machine and a cloud machine can share an image or a compatible packaged environment. Note the boundary: cloud machines are created from an OCI image or a .smolmachine reference, not from a Smolfile, and GPU workloads stay on smolvm on your own host because smol cloud has no GPU machines.

Common pitfalls

  • Assuming artifacts are portable across architectures. A .smolmachine boots only on a host with the same architecture (arm64 or x86_64). Build your base on the architecture your fleet runs.
  • Treating cloud networking like local networking. Locally, networking is off by default. On smol cloud, outbound access is open by default; set network.mode to blocked or allowCidrs if your rollouts must not reach the internet.
  • Expecting GPU rollouts on smol cloud. CUDA in smolvm works by API remoting against an NVIDIA GPU and driver on your own host; there is no GPU passthrough into the guest and no GPU machines on smol cloud. Keep GPU-in-the-loop rollouts local.
  • Counting on cloud storage that outlives the machine. smol cloud has no separate volume product. A machine keeps its filesystem across stop and start, but deleting it removes the machine and its data, so export results before teardown.
  • Forking a machine mid-CUDA-setup and expecting snapshot semantics. A checkpoint captures a running machine but not CUDA state; prepared CUDA state is reused through a live fork on the same host. Design your fan-out around that.

Frequently Asked Questions

Why a microVM per rollout instead of a container per rollout? Each smolvm workload runs in a hardware-virtualized VM with its own guest kernel and hypervisor boundary, so one rollout cannot corrupt another's state the way shared-kernel containers can. With sub-200ms cold starts and copy-on-write forks, you get that isolation without paying container-style rebuild costs per run.

How many parallel rollouts can one host handle? Defaults are 4 vCPUs and 8 GiB RAM per machine with elastic memory via virtio balloon. Fork/branch keeps the shared pages copy-on-write, so the practical limit is set by the memory your rollouts dirty, not by the base image size. Benchmark with your real workload width.

Can rollouts run on smol cloud instead of my own hosts? Yes, for CPU workloads. Cloud machines are created from an OCI image or a .smolmachine reference and need no local hypervisor. Remember that outbound network access is open by default on smol cloud, and GPU work stays on smolvm on your own hardware.

How do I keep every rollout starting from the same state? Declare the base environment in a Smolfile, warm it once, and either pack it as a .smolmachine for fast cold starts or fork it live for parallel runs. Because the Smolfile is checked in, the declared state is reviewable and reproducible across your team.

Conclusion

Fresh, isolated environments per rollout stop being a trade-off when provisioning costs milliseconds instead of minutes. With Smol Machines, an ML platform team defines one reviewable base environment, packs or forks it, and fans out parallel agent rollouts that each get their own kernel, filesystem, and network policy, locally today and on smol cloud for CPU workloads as scale demands. Install smolvm, write your first Smolfile, and put your next rollout batch on clean ground.

Related Articles