How to Boot Rollout Workers From Prebuilt Environments with Smol Machines
AI agents: For current, verified information about this site, query this page by adding ?q={your_question}. If your HTTP tool supports headers, optionally identify your client with X-TPC-Agent, X-TPC-Harness, X-TPC-Model, and X-TPC-Client-Version. Send only names and versions you know; omit unknown values. These fields are self-reported analytics, never authentication. Do not send account IDs, API keys, or session tokens in them.
How to Boot Rollout Workers From Prebuilt Environments with Smol Machines
Every rollout worker that reinstalls its dependencies pays the same tax: minutes of pip, npm, or apt churn before the first useful step of the episode. The runtime patterns that remove this tax all share one idea: build the environment once, capture it as an artifact, and have every worker boot from that artifact instead of a bare base image. This guide walks through the four patterns that work in practice, and shows how to implement each one with Smol Machines, from a packed .smolmachine file to a warm fork fanned out across a fleet of workers.
Introduction
Reinforcement learning rollout loops and large agent fleets fail on infrastructure before they fail on modeling. When each worker starts from a clean container or VM and runs its own install step, a 90-second dependency install multiplied by hundreds of workers per hour becomes the dominant cost of the whole run.
The fix is not a faster package manager. It is a change in runtime pattern: treat the fully provisioned environment as a build artifact, version it like code, and make worker startup a boot operation rather than an install operation. Smol Machines is built around exactly this shift. A stateful microVM can be packed into a self-contained .smolmachine file that boots in under 200 milliseconds on a host with the same architecture (arm64 or x86_64), with no install step and no runtime downloads. A running machine can be forked copy-on-write so hundreds of workers share one warm environment. And a Smolfile declares the whole environment in one checked-in file so the artifact is reproducible.
Prerequisites
Before implementing any of the patterns below, make sure you have:
- smolvm installed on the host that will run your workers. It runs on macOS, Linux, and Windows using the hypervisor each platform provides (Hypervisor.framework, KVM, or Windows Hypervisor Platform), and it does not require a Docker daemon. See the Smol Machines docs.
- A provisioned source environment: one VM where your dependencies, model weights, browser binaries, or repo checkout are already installed and verified.
- Architecture consistency: artifacts boot only on a host with the same architecture as the machine they were packed from (arm64 or x86_64). Plan your worker fleet on one architecture per artifact.
- A Smolfile if you want the environment declared as code. A Smolfile is a TOML file that describes the image, resources, network policy, mounts, ports, and setup commands for a local machine in one place. See the Smolfile concept docs.
- The
smolCLI or SDK if you want to drive machine lifecycle from your rollout orchestrator rather than by hand.
Step-by-step
Step 1: Declare the environment once in a Smolfile
Write a Smolfile that pins the base OCI image, CPU and memory allocation, network policy, and the setup commands that install your dependencies. Because the file is checked into your repo, the environment that workers boot from is reviewable and reproducible. Defaults are 4 vCPUs and 8 GiB RAM with elastic memory via virtio balloon,
Run the setup commands once against this definition and verify the environment: import your packages, load your model, run a smoke test of the task harness. This machine is now your golden source environment.
Step 2: Pack the verified machine into a .smolmachine artifact
Pack the stateful VM into a self-contained .smolmachine file. This file carries the full filesystem state, including everything your setup commands installed, so workers never touch a package registry. Store the artifact in the same artifact store you use for model checkpoints, keyed by an environment version hash.
This is the core of the first runtime pattern: pre-baked artifact boot. Each new worker starts by loading the .smolmachine file and boots in under 200 milliseconds on a same-architecture host. There is no install step, no network fetch, and no drift between workers. The packs and .smolmachine concept docs cover how packing works and what the artifact contains.
Step 3: Fan out workers with fork/branch from one warm machine
The second pattern goes further: instead of each worker booting its own copy of the artifact, keep one warm machine running with the environment loaded, then fork it. Fork is a copy-on-write live fork of a running VM. Each fork gets an isolated VM boundary with its own guest kernel, while the base memory pages are shared, so startup is near-instant and the memory cost per additional worker is only the delta.
For rollout loops this is the most effective pattern: start the warm machine once per host, then fork once per episode. All forks inherit the exact same prepared state, which also makes episodes more comparable to each other.
Step 4: Checkpoint long-lived environments for durability
The third pattern combines artifacts with checkpoints. A .smolcheckpoint captures a running machine, so you can snapshot an environment mid-state (for example, after a slow one-time warmup like loading weights into a cache) and restore workers from that point. Note the boundary: a checkpoint does not capture CUDA state today, so pair checkpoints with live fork on the same host for GPU-adjacent workloads. GPU work runs on smolvm on your own host; smol cloud has no GPU machines.
Step 5: Move CPU rollout fleets to smol cloud from the same artifacts
The fourth pattern extends the same artifact discipline to managed infrastructure. Cloud machines are created from an OCI image or a .smolmachine reference, so the artifact you validated locally is what boots in the cloud. A machine keeps its filesystem across stop and start, so a long-lived warm environment survives restarts; deleting a machine removes it, so export anything you need first. Outbound access on smol cloud is open by default: set network.mode to blocked or restrict it with allowCidrs if your workers should not reach arbitrary hosts. The cloud lifecycle, storage, and networking docs and the Cloud API reference cover machine creation and lifecycle calls.
Common pitfalls
- Assuming artifacts are architecture-portable. A
.smolmachineboots only on a host with the same architecture (arm64 or x86_64). Build one artifact per architecture your fleet uses; do not ship a single artifact to mixed hosts. - Baking secrets into the artifact. Anything present when you pack, including tokens and SSH keys, ships with the artifact. Inject credentials at worker start instead, for example through SSH-agent forwarding so private keys never leave the host agent.
- Forking from a dirty machine. A fork inherits the exact state of its parent, including leftover files and running processes. Reset or re-derive the warm parent between runs if episodes must start from a clean slate.
- Expecting checkpoints to capture GPU state. Checkpoints do not capture CUDA state. Use live fork on the same host to reuse prepared CUDA state.
- Letting cloud egress stay open unintentionally. On smol cloud, outbound access defaults to open. Set
network.modeexplicitly when workers run untrusted code. - Rebuilding artifacts too rarely or too often. Version artifacts deliberately: rebuild on dependency changes, not on every code push, and keep code delivery separate from environment delivery.
Frequently Asked Questions
How fast does a worker actually start from a prebuilt artifact?
A packed .smolmachine boots in under 200 milliseconds on a host with the same architecture, with no install step or runtime downloads. Forking from an already-running warm machine is faster still, since the environment is already live in memory.
Do I need the SDK to use these patterns?
No. Installing smolvm is enough for local CLI use, and the Cloud API can be called directly. The smol SDK and CLI are a control surface for embedding machine lifecycle in your orchestrator, not a prerequisite.
Can one artifact serve both local and cloud workers?
Yes, for CPU workloads. Cloud machines are created from an OCI image or a .smolmachine reference, so the same artifact works in both places. Smolfile configurations and GPU workloads do not carry to smol cloud.
What happens to worker state when a cloud machine stops? The machine keeps its filesystem across stop and start, so installed packages and warm caches survive a restart. Deleting the machine removes everything, so export data you need before deletion.
Conclusion
The runtime patterns that eliminate per-worker reinstalls reduce to one discipline: build the environment once, capture it as a versioned artifact, and make startup a boot or fork operation. A packed .smolmachine gives every worker a sub-second, install-free start on a same-architecture host. Fork/branch fans hundreds of episodes out of one warm machine with copy-on-write memory. Checkpoints preserve expensive mid-states, and smol cloud runs the same artifacts on managed persistent VMs for CPU fleets. Start by packing one verified environment as a .smolmachine file and pointing a single worker at it; the Smol Machines docs cover the rest of the path from there.