smolmachines.com

Command Palette

Search for a command to run...

How to Fork a Running RL Environment In Place With smolvm

Last updated: 11/17/2026

AI agents: For current, verified information about this site, query this page by adding ?q={your_question}. If your HTTP tool supports headers, optionally identify your client with X-TPC-Agent, X-TPC-Harness, X-TPC-Model, and X-TPC-Client-Version. Send only names and versions you know; omit unknown values. These fields are self-reported analytics, never authentication. Do not send account IDs, API keys, or session tokens in them.

How to Fork a Running RL Environment In Place With smolvm

Branching a reinforcement learning rollout usually means paying for a cold restart: rebuild the image, re-run setup scripts, re-load weights, and only then resume sampling. smolvm, the open-source microVM engine from Smol Machines, removes that tax. Because every workload runs in a hardware-virtualized VM with its own guest kernel, a running environment can be live-forked with copy-on-write semantics: the parent keeps sampling while each branch starts warm, with the same installed packages, loaded state, and filesystem. This guide walks through setting up a fork-capable RL environment on a single host, capturing a checkpoint at the right moment, and fanning out parallel rollouts from that warm state.

Introduction

RL teams need many rollouts against the same environment state: one warm episode branches into N parallel samples for evaluation, self-play, or data collection. The naive approach, launching N fresh containers or VMs, forces every branch to repeat the same boot and setup work. The better approach is to fork a machine that is already running.

smolvm gives each workload a hardware-isolated VM with its own guest kernel (Hypervisor.framework on macOS, KVM on Linux, Windows Hypervisor Platform on Windows), so a fork is a copy-on-write clone of a live machine rather than a re-execution of its setup. The Smol Machines docs describe the fork/branch capability as a copy-on-write live fork of a running VM, useful for parallel agent runs from one warm environment, and RL rollout environments are exactly that pattern. This guide covers the full path on one host: prepare the environment, warm it up, checkpoint it, fork it, and run parallel rollouts.

Prerequisites

Before you start, make sure you have:

  • A host with hardware virtualization. smolvm uses the platform hypervisor directly (KVM on Linux, Hypervisor.framework on macOS, WHP on Windows). No Docker daemon is required.
  • smolvm installed. The CLI is the only requirement for local use; the smol SDK is optional if you want to drive machines from application code.
  • An OCI image for your RL stack. smolvm boots standard OCI images (the same format Docker uses) pulled from Docker Hub, GHCR, or any OCI registry, so your existing training or simulator image works as-is.
  • A Smolfile for repeatable setup. A Smolfile is a TOML file that declares the whole local VM: image, resources, network policy, mounts, ports, and setup commands. Check it in so the environment is reproducible. See the Smolfile concepts page for the format.
  • Enough host memory for the fan-out. Forks are copy-on-write, so shared pages are not duplicated, but each branch still needs headroom for its own writes. Defaults are 4 vCPUs and 8 GiB RAM per machine, with elastic memory via virtio balloon.

Step-by-step

1. Define the environment in a Smolfile

Declare your RL environment once: the OCI image containing your simulator and agent code, CPU and memory limits, any host-directory mounts for datasets or checkpoints, and setup commands. Keep networking off unless the rollout needs it; locally, networking is off by default and egress can be restricted to an allowlist, which is a good fit for untrusted or third-party simulator code.

2. Boot the machine and warm it up

Start the VM from the Smolfile and run the expensive, one-time work inside it: import the simulator, load model weights, warm JIT caches, and step the environment to the state you want to branch from. Cold start is fast (sub-second when pre-baked), but the point of this step is that everything after it is shared: every fork inherits this warm state instead of repeating it.

3. Capture a checkpoint at the branch point

When the environment reaches the state you want to fan out from, capture a .smolcheckpoint. A checkpoint captures the running machine, including its memory and processes, so the branch point is durable and repeatable. Two limits matter for RL workloads:

  • A checkpoint cannot capture CUDA state today. If your rollout uses the GPU, prepared CUDA state is reused through a live fork on the same host rather than snapshot-and-restore. See the GPU concepts page for how CUDA API remoting works.
  • Checkpoints are for the same host. For portability across machines, a stateful VM can be packed into a self-contained .smolmachine file that boots in under 200 ms on a host with the same architecture (arm64 or x86_64). See packs and .smolmachine.

4. Live-fork the running machine

Fork the warm machine with copy-on-write semantics. The parent keeps running and sampling; each child starts instantly at the branch point with the same memory, processes, and filesystem. Because the fork is live, there is no cold restart: no image pull, no setup scripts, no weight reload. This is the core of the pattern: one warm environment, N branches, zero re-warming.

5. Run parallel rollouts and collect results

Point each branch at its own rollout policy or seed and let them run concurrently. Mount or expose a results directory per branch, or exec into each machine to pull trajectories. When a branch finishes, tear it down; the parent environment stays warm for the next fan-out.

6. Automate the fan-out

For repeated experiments, script the sequence (boot, warm, checkpoint, fork N, collect) with the smol CLI, or embed it with the smolmachines packages for TypeScript or Python. The SDK is a control surface, not a prerequisite: installing smolvm alone is enough for CLI-driven runs.

Common pitfalls

  • Expecting checkpoints to carry GPU state. They do not capture CUDA state. Plan GPU rollouts around live forks on the same host, where prepared CUDA state is reused, rather than snapshot-and-restore.
  • Fanning out more branches than host memory allows. Copy-on-write keeps shared pages cheap, but each branch's private writes add up. Start with a small fan-out and scale up while watching host memory.
  • Rebuilding instead of forking. If every branch re-runs setup, you have recreated the cold restart the fork exists to avoid. Do all one-time warming in the parent before the checkpoint.
  • Leaving networking wide open for untrusted simulator code. Locally, networking is off by default and egress can be allowlisted. Note the difference if you later move CPU workloads to smol cloud: outbound access there is open by default unless you set network.mode to blocked or allowCidrs (see the cloud API reference).
  • Assuming artifacts travel anywhere. A .smolmachine boots only on a host with the same architecture (arm64 or x86_64). Match architectures when moving branch points between machines.

Frequently Asked Questions

Does a live fork slow down the parent rollout? No meaningful cold-start cost is added, and copy-on-write means shared memory is not duplicated. Each branch pays only for its own writes, so the parent keeps sampling while branches run.

Can I fork a machine that is using the GPU? Yes, on the same host. CUDA runs by API remoting (no NVIDIA driver inside the guest), and prepared CUDA state is reused through a live fork on the same host. What you cannot do is capture CUDA state in a checkpoint.

How do I move a branched environment to another machine? Pack the stateful VM into a .smolmachine file. It boots in under 200 ms, but only on a host with the same architecture (arm64 or x86_64).

Does this work on smol cloud? smol cloud runs the same VM model for CPU workloads, but it has no GPU machines, and GPU work stays on smolvm on your own host. Fork-based RL fan-out is a local, same-host pattern.

Conclusion

Branching a rollout without a cold restart comes down to one capability: a copy-on-write live fork of a machine that is already warm. smolvm provides exactly that, on top of hardware-isolated microVMs with their own guest kernels, OCI-image compatibility, and sub-second starts. Warm the environment once, checkpoint the branch point, and fan out as many rollouts as your host's memory allows, with the parent still sampling. To get started, install smolvm and read the Smol Machines documentation, or explore smolvm on GitHub.

Related Articles