Runloop vs OpenAI Containers
Runloop gives each coding agent a devbox you create, keep running and reach through a tunnel, with Docker inside. OpenAI Containers are a tool inside OpenAI's model API: the model runs short pieces of code on a fixed image, and the box expires after 20 minutes without use.
The companies and products
Runloop
Runloop is a US platform of sandboxes, called devboxes, for building, running and testing AI coding agents.
You get Linux devboxes built from Dockerfile-style blueprints, disk snapshots, HTTPS tunnels, agent gateways that hide API keys, and Axons, event streams that drive agents such as Claude Code. Each devbox runs in a micro-VM on Runloop's own bare-metal hypervisor, with a container layer inside. It is built for teams shipping coding agents, plus evaluation and RL fine-tuning work on the Enterprise plan.
- Compliance
- SOC 2 Type 2
- Architectures
- x86_64 and arm64 base blueprints
- Own-cloud deploy
- Deploy to VPC on AWS, GCP and Azure (Enterprise)
- Regions
- Not documented or selectable
- Company
- Runloop AI, Inc.
OpenAI Containers
OpenAI Containers are the sandboxes OpenAI runs for its own model tools: Code Interpreter, Hosted Shell and the Agents API's hosted sandbox.
This is a different kind of product from bochka: a tool inside OpenAI's model API, not a machine you rent. You create a container through the API or let a Responses call make one, and the model runs Python or shell commands in it on a fixed Debian 12 image without sudo, until it expires after 20 minutes without use. It is built for apps that already call OpenAI models and want them to run code, read files and make charts without running any servers.
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “20th cheapest of 367” means 19 of the other 366 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | Runloop: price rank | OpenAI Containers: price rank | bochka: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month Runloop fits a software factory with Docker and devboxes that last; OpenAI containers have no Docker, no sudo and no disk that survives, and bochka ranks 3rd with nested virtualization on top. | 20th cheapest of 367 $8,152 a month | 356th of 367, with limits $17,371 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB | 32nd cheapest of 366 $10,194 a month | 341st of 366, with limits $13,140 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month Running model-written code is what OpenAI Containers are built for, but for large bursts Runloop ranks 44th and fully fits, while OpenAI does not publish how many containers can run at once. | 44th cheapest of 345 $525 a month | 194th of 345, with limits $6,000 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month Your users can reach a Runloop devbox through its HTTPS tunnel, but an OpenAI container has no inbound access at all; bochka ranks first here with URLs for up to 50 ports and raw TCP. | 26th cheapest of 361 $3,301 a month | 361st of 361, with limits $3,780 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month Runloop ranks 4th for RL rollouts: it starts large waves without published start limits, where OpenAI's container concurrency is unconfirmed and bochka's start limits queue very large waves. | 4th cheapest of 360 $15,922 a month | 191st of 360, with limits $30,000 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 81st cheapest of 364 $2,562 a month | 360th of 364, with limits $5,760 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month CI needs Docker and open internet: Runloop has both, OpenAI documents no Docker and Code Interpreter starts with no internet, and bochka's faster CPUs finish builds sooner. | 121st cheapest of 377 $1,554 a month | 345th of 377, with limits $1,728 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB Runloop can keep a devbox running all month with an idle policy; an OpenAI container has no SSH and is deleted after 20 idle minutes. | 113th cheapest of 368 $410 a month | 362nd of 368, with limits $1,051 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Runloop ships a desktop and a browser an agent can control, OpenAI's hosted browser runs inside the Agents API beta, and bochka streams a 60fps desktop you can embed. | 39th cheapest of 369 $2,306 a month | 351st of 369, with limits $5,280 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month | 40th cheapest of 367 $1,153 a month | 328th of 367, with limits $1,440 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 68th cheapest of 346 $781 a month | 328th of 346, with limits $1,755 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 266th of 338, with limits $15,611 a month | 311th of 338, with limits $35,100 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month Runloop can lock each devbox to a hostname allowlist and OpenAI containers start with no internet; bochka gives each run a full kernel with nested virtualization. | 10th cheapest of 360 $770 a month | 308th of 360, with limits $870 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month | Can't run it | Can't run it | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month | Can't run it | Can't run it | Can't run it |
Key features
| Feature | Runloop | OpenAI Containers | bochka |
|---|---|---|---|
| What it is | Devboxes you create, keep running and reach through the API and a tunnel | A sandbox tool the model uses inside OpenAI's API | A cloud VM you create, keep, stop and reach from outside |
| Isolation | Micro-VM on a custom bare-metal hypervisor, plus a container layer | A fully sandboxed virtual machine; hypervisor not named | Full VM with its own kernel on dedicated AMD Ryzen 9 9950X hosts |
| Root, Docker and nested virtualization | Docker on the dnd blueprints, as root; nested virtualization not documented | Commands do not run with sudo; no Docker or /dev/kvm documented | Root, systemd, Docker and /dev/kvm on every machine |
| Machine size | Picked by vCPU and memory, such as 2 vCPU / 4 GiB or 4 vCPU / 8 GiB | 1, 4, 16 or 64 GB of memory; vCPU not published | 4 vCPU / 8 GB by default, up to 16 vCPU / 32 GB |
| CPU architecture | x86_64 and arm64 | Not published | x86_64 only |
| Custom image | Blueprints built from Dockerfiles, plus disk snapshots | Fixed Debian 12 image; you upload files or skills into each container | Named snapshots of a set-up machine, reused as templates |
| Files after the session | Disk kept through suspend (Pro plan) or disk snapshots; shutdown ends the devbox | Data is discarded when the container expires | Disk saved every minute and on stop, on every plan; a stopped sandbox costs nothing and comes back |
| Lifetime | Fixed lifetime 1 hour by default, 48 hours max; an idle policy replaces it | Expires after 20 minutes without use | No session cap on paid plans; you set the time to live |
| Idle sleep and wake | Suspend after idle, wake on an HTTP request to the tunnel (Pro) | No sleep; an idle container expires and is deleted | No idle timer; TTL based, and a stopped sandbox resumes with its disk |
| Inbound access | One HTTPS tunnel per devbox, SSH through a TLS proxy; no public IP or raw TCP | None; you reach the container only through the API | SSH, HTTPS URLs for up to 50 ports, token-protected by default, raw TCP, dedicated IPv6 or IPv4 |
| Outbound internet | Open by default; hostname allowlist with wildcards, or block all | Code Interpreter and Hosted Shell: off until an admin sets an org domain allowlist; Agents API sandboxes open by default | Open internet on every plan; no provider-side allowlist |
| Keeping API keys out of the sandbox | Agent gateways for any API endpoint with bearer, header or basic auth | domain_secrets: a sidecar adds the real header for approved domains | Credentials proxy for model providers and any HTTPS API; switch a credential to another account when one runs out |
| Coding agents | Claude Code, Codex, OpenCode, Gemini CLI, DeepAgents, driven through Axons and Broker | Agents API (beta) drives a managed Codex harness, with steering, events and webhooks | Claude Code, Codex, pi, OpenCode, Prime Agent, Kimi Code, Mistral Vibe, driven from outside with a prompt, steer and events API |
| Where it runs | Regions not documented; Deploy to VPC on AWS, GCP or Azure (Enterprise) | US or EU processing for Code Interpreter; Agents API can also drive a sandbox you host | EU only (Germany, Finland, France), EU data residency |
Every feature Battleships tracks (43)
| Feature | Runloop | OpenAI Containers | bochka |
|---|---|---|---|
| Full VM (own kernel) | Yes | Yes | Yes |
| Docker inside | Yes | No | Yes |
| Nested virtualization | No | No | Yes |
| Root | Yes | No | Yes |
| Systemd | Not published | Not published | Yes |
| Desktop GUI | Yes | Yes | Yes |
| Browser | Yes | Yes | Yes |
| Computer-use API | Yes | Yes | Yes |
| SSH | Yes | No | Yes |
| Persistent disk | Yes | No | Yes |
| Pause/resume | Yes | No | Yes |
| Automatic snapshots | Yes | No | Yes |
| Fork/clone | Yes | No | Yes |
| Live fork (no pause) | Yes | No | Yes |
| Memory fork | No | No | No |
| Custom image (Docker or snapshot) | Yes | No | Yes |
| ≥24 h sessions | Yes | Yes | Yes |
| Idle auto-stop | Yes | Yes | No |
| Wake on request | Yes | No | No |
| HTTPS preview URLs | Yes | No | Yes |
| Raw TCP inbound | No | No | Yes |
| Public IPv4 | No | No | No |
| Egress allowlist | Yes | Yes | No |
| Secret proxy | Yes | Yes | Yes |
| Secret proxy for any API | Yes | Yes | Yes |
| Preinstalled agents | Yes | No | Yes |
| Agent harness API | Yes | Yes | Yes |
| MCP server | Yes | No | No |
| GPU | No | No | No |
| Windows | No | No | No |
| Arm64 | Yes | No | No |
| Self-hosting | No | Yes | No |
| BYOC | Yes | No | No |
| Open source | No | No | No |
| Scoped API keys | Yes | Not published | Yes |
| Spend limits | Not published | Not published | Yes |
| Usage per sandbox (API) | No | No | Yes |
| Audit logs | Not published | Not published | No |
| SSO | Yes | Yes | No |
| SOC 2 | Yes | Yes | No |
| HIPAA | Not published | Yes | No |
| EU data residency | No | Yes | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
On the hpc-sandbox-benchmarks Node.js build-and-test loop at 4 vCPU / 8 GB (run 36498291449, 2026-09-29), bochka scored 26.69 runs a second and Runloop 8.29, about 3.2 times apart. OpenAI Containers are not in that benchmark, and OpenAI does not publish how many vCPU each container size gets, so there is no comparable score for them. The test measures CPU work only, not start time or network speed.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. OpenAI Containers has no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | Runloop, a month | OpenAI Containers, a month | bochka, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $113 · Basic | $228 · API pay as you go · with limits | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $429 · Basic | $1,051 · API pay as you go · with limits | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $5,268 · Basic | $11,405 · API pay as you go · with limits | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $1,441 · Basic | $1,620 · API pay as you go · with limits | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $3,184 · Basic | $6,000 · API pay as you go · with limits | $2,000 · $2000/mo · with limits |
Runloop: Runloop bills per second for the CPU and memory you allocate while a devbox is starting, running, suspending or resuming, and bills its disk for the devbox's whole life. Basic has no plan fee; Pro adds a $250 fee each month that is not usage credit and unlocks suspend and resume.
OpenAI Containers: OpenAI bills each container session by the minute for the memory size you pick, with a 5-minute minimum, and model tokens are billed separately on top. The scenarios use the 16 GB size for 8 GB needs and the 4 GB size for 4 GB needs, since vCPU is not published.
OpenAI Containers: Plans start at $20 a month and the whole fee is usage credit, billed per second while a sandbox runs. A stopped sandbox costs nothing and keeps its disk.
Limits in those scenarios
Runloop fits all five scenarios; its fixed lifetime tops out at 48 hours unless an idle policy replaces it, and it publishes no egress price and no concurrency cap for paid plans. OpenAI Containers fit none of them as written: no persistent disk, no Docker, no internet in Code Interpreter and Hosted Shell until an admin allows domains, and no published vCPU. OpenAI also does not confirm how many containers can run at once, so the 300 user sandboxes and the 1,000-job burst are unconfirmed. On bochka the burst is limited by starts per minute, so even the $2,000 plan queues those starts.
| Scenario | Runloop | OpenAI Containers | bochka |
|---|---|---|---|
| One coding agent, workdays |
|
| None that bite |
| One agent that never stops |
|
| None that bite |
| 50 agents in parallel |
|
| None that bite |
| Sandboxes for your users |
|
| None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use Runloop when
you want devboxes for coding agents with Docker, a tunnel and a long life, need very large bursts of short jobs, hostname allowlists, arm64, or devboxes inside your own cloud.
- A devbox that lasts. Useful for agents that work for hours: a devbox can run all month with an idle policy, and Pro suspend keeps its disk.
- Docker, root and a way in. Useful for real repos: Runloop runs Docker as root and gives each devbox an HTTPS tunnel and SSH.
- Dockerfile images, arm64 and your own cloud. Runloop builds images from Dockerfiles, offers arm64, and on Enterprise runs inside your own AWS, GCP or Azure account.
Use OpenAI Containers when
your app already calls OpenAI models and the model only needs to run short pieces of code or shell commands on files you give it.
- Built into the model call. Useful if your app already calls OpenAI models: one tool line in a request gives the model a place to run code.
- Locked-down network by default. Useful if code must only reach chosen hosts: Code Interpreter and Hosted Shell have no outbound access until an admin allows domains.
- Data processing in the US or Europe. Code Interpreter can process data in the US or Europe, where Runloop does not document its regions.
Where bochka fits
If you are choosing between Runloop and OpenAI Containers to run coding agents, look at bochka when the agent needs a whole Linux machine it keeps for days. Each bochka sandbox is a full VM with its own kernel, and on the same CPU benchmark it ran about three times faster than Runloop. OpenAI does not publish a comparable score.
A full VM with /dev/kvm
Without /dev/kvm an Android emulator or a VM inside will not start: Runloop documents Docker-in-Docker but not nested virtualization, and OpenAI containers run without sudo or Docker, so apt installs and databases stop there too.
A disk that stays, on every plan
An OpenAI container deletes the agent's clone and installs after 20 idle minutes. On Runloop, shutdown ends the devbox, so keeping its disk means the $250 Pro plan fee for suspend, or snapshot code you maintain.
Reachable from outside
An agent that builds an app has to show it somewhere. OpenAI containers accept no inbound connections, and Runloop gives one HTTPS tunnel with no raw TCP or public IP. bochka gives HTTPS URLs for up to 50 ports, raw TCP and SSH.
Use bochka when your agents need a real machine they keep: root, Docker and /dev/kvm, a disk saved every minute, ports you can reach from outside, and CPUs about three times faster than Runloop's. Compare it one to one: bochka vs Runloop and bochka vs OpenAI Containers.
Questions people ask
Is Runloop cheaper than OpenAI Containers?
In our scenarios, yes. One coding agent on workdays costs $113 a month on Runloop and $228 on OpenAI, and 50 agents in parallel cost $5,268 versus $11,405, with OpenAI's model tokens billed on top. On bochka the same two come to $20 and $317.
Which is faster, Runloop or OpenAI Containers?
Runloop scored 8.29 runs a second on the hpc-sandbox-benchmarks Node.js loop at 4 vCPU / 8 GB, and bochka scored 26.69. OpenAI Containers are not in the benchmark and their vCPU per size is not published, so there is no fair comparison.
Can OpenAI Containers or Runloop run Docker?
Runloop runs Docker as root on its Docker-in-Docker blueprints. OpenAI's hosted shell runs commands without sudo, and no Docker support is documented. bochka gives root, systemd, Docker and /dev/kvm on every sandbox.
How long does a sandbox last on each?
An OpenAI container expires after 20 minutes without use and its data is discarded. A Runloop devbox has a fixed lifetime of 1 hour by default and 48 hours at most, unless an idle policy replaces it. A bochka sandbox has no session cap on paid plans and keeps its disk when it stops.
Can I open a web app running inside the sandbox?
On Runloop, yes, through one HTTPS tunnel per devbox, open or behind a bearer token. OpenAI containers have no ports, URLs or SSH. bochka gives HTTPS URLs for up to 50 ports, raw TCP and SSH.
Can I limit which websites a sandbox can reach?
Yes on both. Runloop allows only the hostnames you list, with wildcards, or blocks everything, and OpenAI's Code Interpreter and Hosted Shell have no outbound access until an admin adds domains. bochka has no provider-side egress allowlist, so you filter inside the VM.
Try bochka
Install the CLI, then run bochka new.
Product preview