Runloop vs Modal VM Sandboxes
Runloop gives each agent a devbox that can sleep when idle and keep running with no fixed end, built from Dockerfiles on x86_64 or arm64. Modal VM Sandboxes are real Linux VMs with their own kernel and faster CPUs, but each one ends within 24 hours.
The companies and products
Runloop
Runloop is a US platform of sandboxes, called devboxes, for building, running and testing AI coding agents.
You get Linux devboxes built from Dockerfile-style blueprints, disk snapshots, HTTPS tunnels, agent gateways that hide API keys, and Axons, event streams that drive agents such as Claude Code. Each devbox runs in a micro-VM on Runloop's own bare-metal hypervisor, with a container layer inside. It is built for teams shipping coding agents, plus evaluation and RL fine-tuning work on the Enterprise plan.
- Compliance
- SOC 2 Type 2
- Architectures
- x86_64 and arm64 base blueprints
- Own-cloud deploy
- Deploy to VPC on AWS, GCP and Azure (Enterprise)
- Regions
- Not documented or selectable
- Company
- Runloop AI, Inc.
Modal VM Sandboxes
Modal is a serverless cloud for AI work, and VM Sandboxes are the version of its sandboxes that run in a virtual machine with their own Linux kernel.
You create a VM Sandbox from Python, TypeScript or Go code with runtime="vm", run commands in it, and terminate it when the job is done. Modal built the runtime on the Rust-based Cloud Hypervisor, so Docker, FUSE, systemd and nested cgroups work, with the same API, images and prices as its gVisor sandboxes. It became generally available on 1 October 2026 and suits teams that want many short-lived agent machines next to Modal's inference and training.
- Funding
- Over $466M raised
- Generally available
- 1 October 2026; early customers launched over 20 million VMs before that
- Session length
- Default 5 minutes, maximum 24 hours
- Compliance
- SOC 2 on every plan; HIPAA on Enterprise
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “20th cheapest of 367” means 19 of the other 366 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | Runloop: price rank | Modal VM Sandboxes: price rank | bochka: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month Runloop fits fully and Modal is a compromise, because a Modal VM ends within 24 hours and loses files outside a Volume; bochka keeps its disk, adds /dev/kvm and a desktop to check the app. | 20th cheapest of 367 $8,152 a month | 84th of 367, with limits $759 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB Runloop can keep a devbox alive all month with an idle policy, while Modal VM Sandboxes stop at 24 hours; bochka has no session cap on paid plans. | 32nd cheapest of 366 $10,194 a month | Can't run it | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month Runloop fits many short code runs well, and bochka's per-plan start limits queue very large bursts. | 44th cheapest of 345 $525 a month | Can't run it | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month | 26th cheapest of 361 $3,301 a month | Can't run it | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month Runloop starts large waves of rollouts without published start limits, while Modal's 24-hour VMs and files kept only in Volumes make it a compromise here. | 4th cheapest of 360 $15,922 a month | 171st of 360, with limits $16,322 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 81st cheapest of 364 $2,562 a month | Can't run it | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month Both run Docker builds, and Modal's VM runtime did about 1.9 times Runloop's CPU work on the same test; bochka was faster than both. | 121st cheapest of 377 $1,554 a month | 188th of 377, with limits $1,064 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB | 113th cheapest of 368 $410 a month | Can't run it | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Runloop ships a desktop and a browser your agent can control, while on Modal you install Xvfb, VNC and the browser yourself; bochka streams its desktop at 60fps. | 39th cheapest of 369 $2,306 a month | Can't run it | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month | 40th cheapest of 367 $1,153 a month | Can't run it | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 68th cheapest of 346 $781 a month | Can't run it | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 266th of 338, with limits $15,611 a month | Can't run it | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month Both can limit where a sandbox connects: Runloop by hostname allowlist, Modal by IP range or domain allowlist (beta); bochka gives each one a full kernel with nested virtualization. | 10th cheapest of 360 $770 a month | 294th of 360, with limits $642 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month Neither fits: Runloop has no GPUs and Modal VM Sandboxes have none either (only Modal's gVisor sandboxes do), and bochka has none. | Can't run it | Can't run it | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month | Can't run it | Can't run it | Can't run it |
Key features
| Feature | Runloop | Modal VM Sandboxes | bochka |
|---|---|---|---|
| Isolation | Micro-VM on a custom bare-metal hypervisor, plus a container layer | VM with its own Linux kernel on Cloud Hypervisor | Full VM with its own kernel on dedicated AMD Ryzen 9 9950X hosts |
| CPU benchmark (4 vCPU / 8 GB) | 8.29 runs/s | 15.54 runs/s | 26.69 runs/s |
| Docker inside | Yes, on the Docker-in-Docker blueprints, as root | Yes, with systemd; start dockerd yourself as the entrypoint | Yes, with root and systemd in every sandbox |
| Nested virtualization | Not documented | Team and Enterprise plans only, on request | Yes, /dev/kvm on every plan |
| Longest session | 1 hour by default, 48 hours max; an idle policy replaces that limit | 24 hours (default 5 minutes) | No cap on paid plans; you set the time to live |
| Keeping files | Disk kept through suspend (Pro plan) or disk snapshots; shutdown ends the devbox | No paused state; only Volumes and snapshots keep files | Disk saved every minute and on stop, on every plan; stops are free |
| Idle sleep and wake | Suspend after idle, wake on an HTTP request to the tunnel (Pro) | Not offered | No idle timer; TTL based |
| Memory snapshots | Not offered | Yes, in alpha: snapshotting ends the sandbox, snapshots expire after 7 days | No; resume boots the VM again |
| Images and CPU type | Blueprints from Dockerfiles, on x86_64 or arm64 | Images from code, x86_64 | Templates from named snapshots, x86_64 |
| Inbound networking | One HTTPS tunnel per devbox, SSH through a TLS proxy, no raw TCP | Encrypted and raw TCP tunnels, connect tokens, inbound IP allowlist | HTTPS URLs for up to 50 ports, token-protected by default, plus raw TCP |
| Outbound rules | Hostname allowlist with wildcards, or block all | Block all, IP-range allowlist, domain allowlist (beta) | Open internet, no provider-side allowlist |
| Hidden API keys | Agent gateways for any API endpoint | Secrets are environment variables, or your own proxy in a sidecar | Credentials proxy for model providers and any HTTPS API; switch to another account when one runs out |
| Coding agents and desktop | Claude Code, Codex, OpenCode and others through Axons; desktop and browser | Examples only; set up Xvfb and VNC yourself | Claude Code, Codex and others driven from outside, 60fps desktop, Chromium and computer-use API |
| Where it runs | Regions not documented; your own AWS, GCP or Azure on Enterprise | US, EU and Asia-Pacific; pinning a region costs 1.15x to 1.75x | EU only (Germany, Finland, France), EU data residency |
Every feature Battleships tracks (43)
| Feature | Runloop | Modal VM Sandboxes | bochka |
|---|---|---|---|
| Full VM (own kernel) | Yes | Yes | Yes |
| Docker inside | Yes | Yes | Yes |
| Nested virtualization | No | Yes | Yes |
| Root | Yes | Yes | Yes |
| Systemd | Not published | Yes | Yes |
| Desktop GUI | Yes | No | Yes |
| Browser | Yes | No | Yes |
| Computer-use API | Yes | No | Yes |
| SSH | Yes | Yes | Yes |
| Persistent disk | Yes | Yes | Yes |
| Pause/resume | Yes | No | Yes |
| Automatic snapshots | Yes | No | Yes |
| Fork/clone | Yes | Yes | Yes |
| Live fork (no pause) | Yes | Yes | Yes |
| Memory fork | No | Yes | No |
| Custom image (Docker or snapshot) | Yes | Yes | Yes |
| ≥24 h sessions | Yes | Yes | Yes |
| Idle auto-stop | Yes | Yes | No |
| Wake on request | Yes | No | No |
| HTTPS preview URLs | Yes | Yes | Yes |
| Raw TCP inbound | No | Yes | Yes |
| Public IPv4 | No | No | No |
| Egress allowlist | Yes | Yes | No |
| Secret proxy | Yes | No | Yes |
| Secret proxy for any API | Yes | No | Yes |
| Preinstalled agents | Yes | No | Yes |
| Agent harness API | Yes | No | Yes |
| MCP server | Yes | No | No |
| GPU | No | No | No |
| Windows | No | No | No |
| Arm64 | Yes | No | No |
| Self-hosting | No | No | No |
| BYOC | Yes | No | No |
| Open source | No | No | No |
| Scoped API keys | Yes | Not published | Yes |
| Spend limits | Not published | Not published | Yes |
| Usage per sandbox (API) | No | Not published | Yes |
| Audit logs | Not published | Yes | No |
| SSO | Yes | Yes | No |
| SOC 2 | Yes | Yes | No |
| HIPAA | Not published | Yes | No |
| EU data residency | No | Yes | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
All three ran the hpc-sandbox-benchmarks Node.js build-and-test loop at 4 vCPU / 8 GB in the same run on 29 September 2026. bochka scored 26.69 runs a second, Modal VM Sandboxes 15.54 and Runloop 8.29, so Modal did about 1.9 times Runloop's CPU work and bochka about 1.7 times Modal's. The test measures CPU work only, not start time or network speed.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. Modal VM Sandboxes measured on: VM Sandbox, request = size.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | Runloop, a month | Modal VM Sandboxes, a month | bochka, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $113 · Basic | $15 · Starter · with limits | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $429 · Basic | $178 · Starter · with limits | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $5,268 · Basic | $2,229 · Starter · with limits | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $1,441 · Basic | $792 · Team | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $3,184 · Basic | $3,193 · Team · with limits | $2,000 · $2000/mo · with limits |
Runloop: Runloop bills per second for the CPU and memory you allocate while a devbox starts, runs, suspends or resumes, and bills its disk for the devbox's whole life. Basic has no plan fee; Pro adds $250 a month that is not usage credit and unlocks suspend and resume.
Modal VM Sandboxes: Modal bills by the second for the higher of the CPU and memory you request or what you use, until the sandbox ends. Starter is free with $30 of monthly credit, Team is $250 a month with $100 of credit, and egress above the free allowance costs $0.04 per GiB.
Modal VM Sandboxes: Plans start at $20 a month and the whole fee is usage credit, billed per second while a sandbox runs. A stopped sandbox costs nothing and keeps its disk.
Limits in those scenarios
Modal's 24-hour session cap and its lack of a paused state are why the workday, always-on and fleet scenarios do not fit there: files outside a Volume reset. Runloop fits those scenarios, but publishes no egress price and suspend on idle needs Pro. For 1,000 short jobs at once, Runloop fits, Modal's priced tier uses shared CPU above its request, and bochka's start limits queue the burst.
| Scenario | Runloop | Modal VM Sandboxes | bochka |
|---|---|---|---|
| One coding agent, workdays |
|
| None that bite |
| One agent that never stops |
|
| None that bite |
| 50 agents in parallel |
|
| None that bite |
| Sandboxes for your users |
| None that bite | None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use Runloop when
agents need devboxes that sleep when idle and keep running past a day, hostname allowlists, arm64 or Dockerfile builds, or a deploy inside your own cloud.
- Devboxes that keep running. Useful for agents that live on one machine: an idle policy lets a devbox run with no fixed end, where Modal stops at 24 hours.
- Sleep when idle, wake on a request. On the Pro plan, Runloop suspends an idle devbox and wakes it when an HTTP request reaches its tunnel.
- Dockerfiles, arm64 and your own cloud. Runloop builds images from Dockerfiles, offers arm64, and on Enterprise runs inside your own AWS, GCP or Azure account.
Use Modal VM Sandboxes when
you run big bursts of short jobs that finish within a day, need regions outside Europe, or already run your models on Modal.
- A real kernel and faster CPUs. Each Modal VM has its own Linux kernel with Docker, FUSE and systemd, and did about 1.9 times Runloop's CPU work.
- Huge bursts and memory snapshots. Useful for big waves of short jobs: Modal runs 5,000 containers at once on Team, and memory snapshots are in alpha.
- Regions and one AI platform. Modal runs in the US, EU and Asia-Pacific, next to its own inference, training and GPU functions on one account.
Where bochka fits
If your agents work for hours or days on one machine, installing packages, running Docker, builds and emulators, look at bochka. It is a full VM like Modal's, keeps its disk like a suspended Runloop devbox on every plan, and ran faster than both on the same CPU test.
A full machine with /dev/kvm
Runloop documents no nested virtualization, and Modal offers it only on Team and Enterprise after you ask support. Without /dev/kvm an Android emulator or a VM inside does not start. bochka exposes /dev/kvm on every plan.
The disk survives every stop
A Modal VM ends within 24 hours and its root disk goes with it, and a Runloop devbox is gone on shutdown unless you pay for Pro suspend or write snapshot code. bochka saves the disk every minute and stops are free.
More CPU work per hour
On the same test bochka ran 26.69 runs a second, Modal's VM 15.54 and Runloop 8.29. Agents spend most of their time waiting on installs, builds and tests, so each wait is longer on both, and the waiting is billed.
Use bochka when your agents work for hours or days in a machine they come back to, with Docker, /dev/kvm and a desktop, on faster CPUs and a disk kept on every plan. Compare it one to one: bochka vs Runloop and bochka vs Modal VM Sandboxes.
Questions people ask
Is Runloop cheaper than Modal VM Sandboxes?
Usually not. One always-on agent is $429 a month on Runloop and $178 on Modal, though Modal's 24-hour cap means that scenario does not fully fit there. For 1,000 short jobs at once they are close, $3,184 against $3,193, and bochka runs the always-on agent for $26 a month.
Which is faster, Runloop or Modal VM Sandboxes?
On the hpc-sandbox-benchmarks CPU test at 4 vCPU / 8 GB, Modal's VM runtime did 15.54 runs a second and Runloop 8.29, about 1.9 times apart. bochka did 26.69 on the same test.
Can Runloop and Modal VM Sandboxes run Docker?
Yes. Runloop runs Docker on its Docker-in-Docker blueprints, and Modal VM Sandboxes run Docker with systemd once you start dockerd. For Android emulators, Runloop documents no nested virtualization and Modal offers it on Team and Enterprise on request; bochka has /dev/kvm on every plan.
How long can a sandbox run on each?
A Modal VM Sandbox runs 5 minutes by default and 24 hours at most. Runloop's fixed lifetime defaults to 1 hour with a 48-hour maximum, but an idle policy replaces it, so a devbox can run all month. bochka has no session cap on paid plans.
Do they keep files when a sandbox stops?
A Modal VM has no paused state, so only Volumes and snapshots keep files after it ends. Runloop keeps the disk through suspend on Pro or in disk snapshots, and shutdown deletes it. bochka saves the disk every minute and on stop, on every plan.
Is bochka SOC 2 compliant?
SOC 2 is in progress. We're open about where we stand: live status at status.bochka.xyz and our data retention policy at docs.bochka.xyz/data-retention.
Try bochka
Install the CLI, then run bochka new.
Product preview