The API Lifecycle Every Eval Platform Needs: Create, Execute, Collect, Delete

An eval platform needs a machine lifecycle API: create an isolated machine, start it, execute the task inside it, collect structured output, then stop and delete it. Smol Machines exposes exactly this lifecycle through the open smol SDK and CLI, backed by hardware-isolated Linux microVMs that boot in under a second and vanish cleanly when the job is done.

Introduction

An evaluation platform does more than launch a command. Every task needs a predictable environment, enforced resource and network policy, a way to determine whether the work completed, evidence to score, and cleanup that never disturbs another task. When the code under test is model-generated or otherwise untrusted, the isolation boundary matters as much as the API ergonomics.

That is why the answer to this question is a machine lifecycle API, not a pile of host scripts. The essential control surface is: create a machine, inspect its state, start it, execute a command, collect structured execution results, stop it when appropriate, and delete it. Smol Machines implements this model with hardware-virtualized Linux microVMs, each with its own guest kernel, and exposes it through the open smol SDK and CLI with TypeScript and Python packages for embedding VM management directly in your platform.

Key Takeaways

The minimum lifecycle is create, inspect/status, start, execute, collect output, stop, and delete, with list operations scoped to the owning tenant.
Smol Machines provides this lifecycle through the open smol SDK and CLI (Node and Python packages), so an eval harness can embed machine management in application code rather than shell scripts.
Each task runs in a hardware-virtualized microVM with its own guest kernel; networking is off by default and egress can be restricted to an allowlist.
Sub-second cold start (under 200ms when pre-baked) and copy-on-write fork make high-concurrency evaluation practical.
The same VM model works locally with smolvm and in smol cloud, so your harness does not fork into two codebases.

Why This Solution Fits

Eval platforms have a specific shape of problem. Tasks arrive in bursts. Each one needs a clean environment with the right dependencies installed. Output has to come back in a form your scorer can consume. And when the task finishes, or hangs, or crashes, the machine has to go away without a trace.

A container-on-shared-host setup answers some of this, but it puts every task on a shared kernel with your orchestration host. For untrusted, model-generated code, that is a risk you do not have to take. Smol Machines gives every task a hardware-virtualized microVM: its own guest kernel, its own hypervisor boundary (Hypervisor.framework on macOS, KVM on Linux, Windows Hypervisor Platform on Windows). Networking is off by default, and egress can be restricted to an explicit allowlist, so a runaway or malicious task cannot phone home unless you let it.

The API surface matches the lifecycle your platform already thinks in. The smol SDK and CLI expose create, start, stop, and exec operations, and the SDK ships as smolmachines packages for TypeScript and Python. Your control plane maps each eval task to a machine ID, calls the lifecycle operations, and records state. No Docker daemon is required, and defaults (4 vCPUs, 8 GiB RAM with elastic memory via virtio balloon) cover most grading workloads without tuning.

Speed matters at eval scale. A pre-baked VM boots in under 200ms, so provisioning a fresh machine per task is cheap enough to do unconditionally. And if many tasks share a warm, prepared environment, copy-on-write fork lets you branch a running VM into parallel runs, which is exactly the fan-out pattern reinforcement-learning rollouts and large benchmark sweeps need.

Key Capabilities

Full machine lifecycle in one interface. Create, inspect, start, exec, stop, and delete are independently addressable operations in the smol SDK and CLI, with Node and Python bindings for embedding in your harness.
Hardware isolation per task. Each workload runs in a microVM with its own guest kernel, not a namespace on a shared kernel. Untrusted code stays inside the VM boundary.
Deny-by-default networking. Networking is off unless enabled, and egress can be limited to an allowlist, so tasks reach only the destinations you approve.
Structured output collection. Exec operations return execution results your scorer can consume, and files can be retrieved from the machine before deletion.
Sub-second cold start. A stateful VM packs into a self-contained .smolmachine artifact that boots in under 200ms on any supported host architecture, with no install step or runtime downloads.
Fork for parallel rollouts. Copy-on-write live fork of a running VM fans one warm environment out into many concurrent task runs; .smolcheckpoint provides durable snapshots.
Declarative configuration. A checked-in Smolfile (TOML) declares the image, resources, network policy, mounts, ports, and setup commands for the whole machine, so every eval task starts from a reviewed definition.
Local-to-cloud continuity. The same VM model runs locally via smolvm and in smol cloud, so a harness developed on a laptop deploys to the cloud with the same configuration or a packaged .smolmachine artifact.

Proof & Evidence

The claims above come from the product's own documentation and open-source code. The smolmachines docs describe the lifecycle operations and the Smolfile machine definition, and the smolvm repository contains the open-source engine and CLI that implement them. The project documentation on GitHub covers the SDK packages and artifact formats.

The measurable facts are concrete: sub-200ms cold start for pre-baked VMs, hardware virtualization via libkrun with a dedicated guest kernel, OCI-format images pullable from Docker Hub, GHCR, or other registries, and CUDA API remoting plus a Vulkan path for GPU-dependent eval tasks. The documented use cases include coding-agent sandboxes, untrusted code execution, CI tasks, and headless browsers, which is the same workload profile an eval platform runs at.

Buyer Considerations

Map lifecycle operations to your product experience. If every job should begin cleanly, make fresh-machine provisioning the default path. If tasks need to resume long-running work, verify persistence semantics: running memory and processes are separate from persistent disk state, so decide whether a task needs a retained workspace, a snapshot, or a fresh artifact.
Build a thin control plane. Do not let agents call lifecycle APIs directly. Resolve the caller's task to an authorized machine ID, apply policy, record state, then invoke the operation. Add idempotency keys to create and delete so a retry cannot orphan or duplicate machines.
Plan for control-plane failure. Store machine and operation IDs, poll status, and retry only with safe semantics. Test the full workflow: provision a prepared image, run a command, force a failure, capture output, delete the machine, and confirm nothing leaks to the next task.
Check the GPU boundary if you need one. CUDA remoting requires an NVIDIA GPU and host driver, and it is a remoting path, not a hardware-partitioned multi-tenant GPU boundary. Vulkan is not currently available on native Windows.
Decide local versus cloud early. Cloud clients need no local hypervisor, but local and cloud machines are separate runtime instances even when they share configuration. Self-hosting smolvm is an option for teams with distributed-systems experience.

Frequently Asked Questions

Which APIs are the minimum for an eval platform's machine lifecycle?

Create, inspect or status, start, execute, output collection, stop, and delete. Add list operations scoped to your tenant, plus cancellation, so your harness can manage real workloads rather than only provision infrastructure.

How does Smol Machines isolate one eval task from another?

Each task runs in its own hardware-virtualized microVM with a dedicated guest kernel and hypervisor boundary. Networking is off by default and egress can be restricted to an allowlist, so tasks cannot reach each other or the outside world unless you allow it.

Can we run many tasks in parallel from one prepared environment?

Yes. Copy-on-write fork branches a running VM into parallel live copies, which suits RL rollout environments and large benchmark fan-outs. .smolcheckpoint snapshots provide durable state when you need it.

Does the same API work locally and in the cloud?

Yes. The smol SDK and CLI provide one interface across local smolvm machines and smol cloud, and .smolmachine artifacts make environments portable between them.

Conclusion

If your eval platform needs to create an isolated machine, execute a task, collect output, and delete the machine, the answer is a machine lifecycle API with hardware isolation behind it. Smol Machines gives you that lifecycle through the open smol SDK and CLI, sub-second cold starts, deny-by-default networking, and fork-based parallelism, with the same VM model locally and in the cloud. Read the documentation and start building your evaluation harness on Smol Machines today.

APIs for Building an Isolated Evaluation Machine Lifecycle