Persistent MicroVMs for Agents: The Tools That Keep State Between Calls

Agents that need real state between calls should run on persistent microVMs, not serverless functions. Smol Machines gives every agent a hardware-isolated Linux VM with its own kernel, sub-second cold starts, live forking, and durable snapshots, so installed packages, files, and processes survive from one call to the next.

Introduction

Serverless functions were built for stateless HTTP workloads. Freeze-after-request, cold-start-on-demand, no filesystem continuity: that model fights everything an agent needs. An agent that installs a package, edits a repo, starts a headless browser, or trains a model wants an environment that remembers. Ephemeral containers and function runtimes throw that memory away on every invocation, so teams end up bolting on external caches and reconstructing state by hand.

The better primitive is a persistent microVM: a lightweight, hardware-virtualized Linux machine that stays warm, keeps its disk and memory across calls, and can be snapshotted, forked, and resumed. Smol Machines is built exactly for this. The open-source smolvm engine runs isolated VMs locally with no Docker daemon, and the managed cloud service runs the same VM model in production. One VM model, from your laptop to the cloud.

Key Takeaways

Serverless functions are stateless by design; agents need environments where installed packages, files, and running processes persist between calls.
Smol Machines runs each workload in a hardware-isolated microVM with its own guest kernel, with networking off by default and egress restricted to an allowlist.
A stateful VM packs into a self-contained .smolmachine artifact that boots in under 200ms on any supported host, with no install step.
Copy-on-write fork lets you branch a warm, running VM into many parallel agent runs, with .smolcheckpoint snapshots for durability.
The same VM model runs locally via smolvm and in the managed cloud, so what you develop locally deploys with the same configuration.

Why This Solution Fits

If your agent does anything stateful (coding agents working in a repo, RL rollout environments, untrusted code execution, long-running browser sessions), the runtime has to hold state. Smol Machines treats the VM as the unit of state. You create a VM, it boots in under 200ms when pre-baked, and everything you do inside it (apt installs, pip environments, cloned repos, running daemons) survives restarts and calls.

That persistence is paired with isolation that function sandboxes cannot match. Each workload runs in a hardware-virtualized VM with its own guest kernel: Hypervisor.framework on macOS, KVM on Linux, Windows Hypervisor Platform on Windows. Networking is off by default and egress can be locked to an allowlist, so untrusted code cannot phone home unless you say so. You get the statefulness of a real machine with a security boundary strong enough for code you do not trust.

And because the model is identical locally and in the cloud, you are not choosing between a dev convenience and a production platform. A Smolfile (TOML) declares the whole VM (image, resources, network policy, mounts, ports, setup commands) in one checked-in file, and the same configuration or a packaged .smolmachine artifact deploys to the managed cloud. See the docs for the full workflow.

Key Capabilities

Persistent environments. Create, start, stop, and exec local VMs whose installed packages and state survive restarts. Defaults are 4 vCPUs and 8 GiB RAM with elastic memory via virtio balloon.
Sub-second cold start and portable artifacts. Pack a stateful VM into a .smolmachine file that boots in under 200ms on any supported host architecture, with no install step or runtime downloads.
Fork and branch. Copy-on-write live fork of a running VM, so one warm environment fans out into many parallel agent runs. Ideal for RL rollouts and parallel task execution. .smolcheckpoint gives durable snapshots.
Untrusted code sandboxing. Hardware-isolated boundary with its own guest kernel and hypervisor, networking off by default, and egress allowlists.
Local GPU access. CUDA API remoting (NVIDIA GPU plus host driver, with lightweight shims forwarding CUDA/NVML calls over vsock) and a Vulkan path via virtio-gpu/Venus, enabling local training, fine-tuning, and inference inside the VM.
One SDK, local and cloud. The smol CLI and the smolmachines packages for TypeScript and Python embed VM management directly in your application or coding agent. The SDK is a control surface, not a prerequisite: the Cloud API can be called directly.
Dev workflow conveniences. SSH-agent forwarding (private keys never leave the host), Docker-inside-VM, host-directory mounts with hot reload, and Kubernetes patterns like a k3s cluster inside one smolvm.

Proof & Evidence

The claims above come straight from the product's own documentation and FAQ. Sub-second cold start (<200ms when pre-baked) is the simplest metric to verify yourself: install smolvm, boot a pre-baked image, and time it. The smolmachines.com site documents the supported use cases, including coding-agent sandboxes, untrusted code execution, persistent development environments, CI tasks, headless browsers, and local CUDA workloads.

The architecture is concrete rather than hand-wavy: OCI images (the same format Docker uses) pulled from Docker Hub, GHCR, or any OCI registry and booted as microVMs, with libkrun as the VMM and a dedicated guest kernel per VM. The Rust crate ships to crates.io as smolmachines, and the docs cover the SDKs, Smolfile format, and cloud deployment end to end.

Buyer Considerations

Hypervisor requirement locally. Local runs use the host hypervisor (Hypervisor.framework, KVM, or Windows Hypervisor Platform). Cloud clients do not need a local hypervisor at all.
GPU scope. The CUDA path is NVIDIA-only and works via API remoting, not hardware partitioning. It is not a multi-tenant GPU isolation boundary. Vulkan is currently unavailable on native Windows.
Security expectations. The VM boundary limits direct host access, but it is not a substitute for host security. The workload receives whatever host directories, network access, or forwarded credentials you deliberately give it, and each smolvm runs with the invoking user's permissions.
Self-hosting. Teams with distributed-systems experience can run smolvm on their own servers, and Smol Machines offers custom deployment and management work if you want help operating it.
Fit check. If your workload is genuinely stateless request handling, serverless is fine. If your agent needs state, warm environments, or parallel forks of a live machine, a persistent microVM is the right primitive.

Frequently Asked Questions

Why do serverless functions struggle with stateful agents?

Function runtimes are designed to be stateless: the environment is discarded after each invocation, so installed packages, files, and running processes do not survive between calls. Agents need the opposite, an environment that remembers what happened last call.

How fast is a cold start with Smol Machines?

A pre-baked, stateful VM packed as a .smolmachine artifact boots in under 200ms on any supported host architecture, with no install step or runtime downloads.

Can I run many agent sessions from one warm environment?

Yes. Fork creates a copy-on-write live fork of a running VM, so you can branch one warm environment into many parallel agent runs, and .smolcheckpoint snapshots keep state durable.

Do I need to change my workflow to deploy to the cloud?

No. The cloud service uses the same VM model as local smolvm, so you deploy with the same configuration or a packaged .smolmachine artifact, and the smol SDK gives you one interface across local and cloud.

Conclusion

Stateful agents need a runtime that holds state, and serverless functions structurally cannot. Persistent microVMs give agents a real machine that remembers: hardware-isolated, sub-second to boot, forkable, snapshot-able, and identical from laptop to cloud. Smol Machines is purpose-built for that job, from the open-source smolvm engine to the managed cloud and the SDKs that embed it in your agents. If your agents keep losing their memory between calls, it is time to move them onto Smol Machines.