Blacksmith vs WarpBuild
Blacksmith and WarpBuild both run your GitHub Actions jobs on fresh machines that are destroyed when the job ends. Blacksmith keeps chosen folders on sticky disks and costs less per minute in our scenarios, while WarpBuild can save a whole disk snapshot and can run its runners inside your own AWS, GCP or Azure account.
The companies and products
Blacksmith
Blacksmith is a CI service: it runs your GitHub Actions jobs on its own faster, cheaper runners instead of GitHub's.
You change the runner label in your workflow file, and each job runs in a fresh Firecracker microVM that boots in under 3 seconds and is destroyed when the job ends. It adds Docker layer caching and sticky disks that carry chosen folders from one run to the next, plus a Testbox CLI in early beta that lets a local coding agent run commands inside a CI job. It is built for teams that want quicker, cheaper GitHub Actions, not as a general sandbox you drive by API; a separate Sandboxes product is marked coming soon.
WarpBuild
WarpBuild sells hosted GitHub Actions runners: you change the runs-on label and your CI jobs run on its machines.
WarpBuild is a CI service, not an agent sandbox. Each GitHub Actions job runs in its own VM on Linux (x86-64 or arm64), Windows or macOS on Apple M4 Pro, and the VM and its disk are destroyed after the build. It also sells remote Docker builders, can run the same runners inside your own AWS, GCP or Azure account, and can host Claude Managed Agents sessions on its stock runners.
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “78th cheapest of 377” means 77 of the other 376 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | Blacksmith: price rank | WarpBuild: price rank | bochka: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month Agents that build and run whole apps need hosting and a kept disk; neither runner accepts inbound traffic, and bochka serves HTTPS on up to 50 ports. | 248th of 367, with limits $3,607 a month | 269th of 367, with limits $4,343 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB An assistant that lives on its machine all month does not fit a CI job on either, and WarpBuild jobs also stop at GitHub's 5 day cap. | 78th of 366, with limits $5,463 a month | 285th of 366, with limits $5,650 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month All three are a compromise for very short code runs: both runners bill per minute of job time, and bochka's start limits slow very large bursts. | 281st of 345, with limits $488 a month | 287th of 345, with limits $600 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month Giving each user a sandbox means starting machines from your own backend, which bochka does by API and both runners do only through GitHub workflows. | 268th of 361, with limits $1,563 a month | 278th of 361, with limits $1,890 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month | 276th of 360, with limits $7,488 a month | 281st of 360, with limits $9,000 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 354th of 364, with limits $1,188 a month | 294th of 364, with limits $1,440 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month Both fit by changing one runner label; Blacksmith costs less in our CI estimate ($708 against $864 a month), and bochka fits CI you drive by API or that needs a machine to stay up. | 78th cheapest of 377 $708 a month | 94th cheapest of 377 $864 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB A dev box needs a disk that stays; Blacksmith keeps only sticky disk folders and WarpBuild only disk snapshots that expire after 15 days, while bochka keeps the whole disk. | 180th of 368, with limits $207 a month | 182nd of 368, with limits $226 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Computer-use agents need a screen; Blacksmith and WarpBuild runners are headless, and bochka streams a 60fps Linux desktop. | 243rd of 369, with limits $1,088 a month | 264th of 369, with limits $1,320 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month | 131st of 367, with limits $588 a month | 280th of 367, with limits $720 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 130th of 346, with limits $719 a month | 306th of 346, with limits $878 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 262nd of 338, with limits $14,613 a month | 275th of 338, with limits $17,550 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month | 64th of 360, with limits $350 a month | 264th of 360, with limits $435 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month | Can't run it | Can't run it | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month Both have macOS runners, Blacksmith on Apple Silicon and WarpBuild on Apple M4 Pro, and bochka runs Linux only. | 28th of 36, with limits $1,508 a month | 29th of 36, with limits $1,520 a month | Can't run it |
Key features
| Feature | Blacksmith | WarpBuild | bochka |
|---|---|---|---|
| What it is | GitHub Actions runners; each machine lives for one workflow job | GitHub Actions runners and remote Docker builders; each machine lives for one job | A full VM per agent that you create, stop and start by API |
| Isolation | Firecracker microVM per job, destroyed with its state after the job | Its own VM per job, created on demand and destroyed after the build | Full VM with its own kernel on dedicated AMD Ryzen 9 9950X hosts |
| How a machine starts | A GitHub Actions job; Testbox warmup dispatches a workflow | A GitHub Actions job picked up by the runs-on label | One API call, SDK or CLI, from anywhere |
| How long it lives | Until the job ends; Testbox stops after 30 idle minutes by default | Until the job ends; GitHub stops a self-hosted runner job after 5 days | Until you stop it; no session cap on paid plans |
| What carries over between runs | Sticky disks: up to 5 folders per job, $0.50 per GB a month, evicted after 7 days unused | Cache, plus disk snapshots on Ubuntu cloud runners, $0.025 an hour each, deleted after 15 days | The whole disk, saved every minute and on stop; a stopped sandbox costs nothing |
| Fork | No; every machine starts fresh from a job | No fork; a later runner can boot from a saved disk snapshot in 45 to 60 seconds | Fork a running sandbox with its disk |
| Docker and nested virtualization | Docker with layer caching; /dev/kvm on x64 Linux runners, not on ARM | Docker on runners; /dev/kvm only on Linux x86-64, turned on with a label | Docker, root, systemd and /dev/kvm on every machine |
| Operating systems | Ubuntu x64 and ARM, Windows Server 2025 (public beta), macOS on Apple Silicon | Ubuntu x86-64 and arm64, Windows Server 2022 and 2025, macOS 14, 15 and 26 on Apple M4 Pro | Linux (x64) |
| Where it runs | Its own hardware in Ashburn, Phoenix, Amsterdam and Frankfurt | WarpBuild's cloud, or BYOC in your own AWS, GCP or Azure account | EU only (Germany, Finland, France) |
| Concurrency | Docs say there are no concurrency limits | Unlimited on Linux and Windows; high macOS concurrency through support | No session cap on paid plans; starts per minute and per day are limited by plan |
| Inbound access | Opt-in debug SSH, only for the user who triggered the job; no ports or URLs | All inbound traffic blocked; SSH into failed jobs for debugging | HTTPS URLs for up to 50 ports, token-protected by default; raw TCP; dedicated IP |
| Desktop and computer use | Headless runners | None; runners have no desktop stream | 60fps Linux desktop with Chromium and computer use |
| Coding agents | Its own [code]smith agent; Testbox CLI (early beta) for local agents; Sandboxes coming soon | Runs Claude Managed Agents sessions on stock Linux and macOS runners | Seven agents preinstalled, with a prompt, steer and events API |
| Keeping API keys out of the machine | GitHub Actions secrets and OIDC tokens available inside the job | GitHub secrets are passed into the runner environment | Credentials proxy for model providers and any HTTPS API; switch a credential to another account when credits run out |
Every feature Battleships tracks (43)
| Feature | Blacksmith | WarpBuild | bochka |
|---|---|---|---|
| Full VM (own kernel) | Yes | Yes | Yes |
| Docker inside | Yes | Yes | Yes |
| Nested virtualization | Yes | Yes | Yes |
| Root | Yes | Yes | Yes |
| Systemd | No | Not published | Yes |
| Desktop GUI | No | No | Yes |
| Browser | Yes | No | Yes |
| Computer-use API | No | No | Yes |
| SSH | Yes | Yes | Yes |
| Persistent disk | Yes | No | Yes |
| Pause/resume | No | No | Yes |
| Automatic snapshots | No | No | Yes |
| Fork/clone | No | No | Yes |
| Live fork (no pause) | No | No | Yes |
| Memory fork | No | No | No |
| Custom image (Docker or snapshot) | No | Yes | Yes |
| ≥24 h sessions | Yes | Yes | Yes |
| Idle auto-stop | Yes | Yes | No |
| Wake on request | No | Yes | No |
| HTTPS preview URLs | No | No | Yes |
| Raw TCP inbound | No | No | Yes |
| Public IPv4 | No | No | No |
| Egress allowlist | No | Yes | No |
| Secret proxy | No | No | Yes |
| Secret proxy for any API | No | No | Yes |
| Preinstalled agents | No | Yes | Yes |
| Agent harness API | No | No | Yes |
| MCP server | No | Yes | No |
| GPU | No | No | No |
| Windows | Yes | Yes | No |
| Arm64 | Yes | Yes | No |
| Self-hosting | No | No | No |
| BYOC | No | Yes | No |
| Open source | No | No | No |
| Scoped API keys | Not published | Yes | Yes |
| Spend limits | Not published | Not published | Yes |
| Usage per sandbox (API) | Yes | Yes | Yes |
| Audit logs | Not published | Not published | No |
| SSO | No | Yes | No |
| SOC 2 | Yes | Yes | No |
| HIPAA | No | No | No |
| EU data residency | Yes | Yes | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
There is no comparable measurement for Blacksmith or WarpBuild: neither was part of the same CPU benchmark run, so we cannot say which is faster. bochka did 26.69 runs per second on the StarSling hpc-sandbox-benchmarks Node.js loop at 4 vCPU / 8 GB, on dedicated AMD Ryzen 9 9950X hosts. Both runners compare their speed with GitHub's own runners, not with each other or with bochka.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. Blacksmith and WarpBuild have no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | Blacksmith, a month | WarpBuild, a month | bochka, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $36 · Pay-as-you-go · with limits | $57 · Pay as you go · with limits | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $207 · Pay-as-you-go · with limits | $237 · Pay as you go · with limits | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $2,364 · Pay-as-you-go · with limits | $2,851 · Pay as you go · with limits | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $663 · Pay-as-you-go · with limits | $810 · Pay as you go · with limits | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $1,488 · Pay-as-you-go · with limits | $1,800 · Pay as you go · with limits | $2,000 · $2000/mo · with limits |
Blacksmith: Blacksmith bills per minute of job time by vCPU and OS: Ubuntu x64 is $0.004 a minute for 2 vCPU, ARM $0.0025, Windows $0.008 and macOS $0.08 for 6 vCPU, with 3,000 free minutes a month. Sticky disks cost $0.50 per GB a month and a static IP $100 a month.
WarpBuild: WarpBuild bills per minute while a job runs, with no plan fee, from $0.003 a minute for 2 vCPU on arm64; Windows costs twice the Linux x86-64 rate and macOS $0.08 a minute. Cache, snapshot storage and restores cost extra, and BYOC runners add $0.002 a minute on top of your cloud bill.
Limits in those scenarios
None of the five scenarios fits either runner cleanly, because a machine exists only while a GitHub Actions job runs. On Blacksmith only sticky disk folders survive, so packages installed elsewhere reset; on WarpBuild the disk goes away entirely, and an agent meant to run all month hits GitHub's 5 day job cap. Both publish unlimited concurrency, but the 300 and 1,000 machine scenarios still mark it as unconfirmed, and neither publishes an egress price. On bochka, the 1,000-job burst is queued by start limits even on the $2,000 plan.
| Scenario | Blacksmith | WarpBuild | bochka |
|---|---|---|---|
| One coding agent, workdays |
|
| None that bite |
| One agent that never stops |
|
| None that bite |
| 50 agents in parallel |
|
| None that bite |
| Sandboxes for your users |
|
| None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use Blacksmith when
you want faster, cheaper GitHub Actions by changing one label, with sticky disk folders, US regions or Linux runners up to 32 vCPU.
- Lower per-minute prices. In every scenario we priced, Blacksmith comes in below WarpBuild, for example $708 against $864 a month for CI.
- Folders that stay without a restore. Teams that want node_modules or build caches ready on every job can keep up to 5 folders on sticky disks.
- US regions and big Linux machines. It runs in Ashburn and Phoenix as well as Amsterdam and Frankfurt, with Ubuntu x64 runners up to 32 vCPU and 128 GB.
Use WarpBuild when
your CI must run in your own AWS, GCP or Azure account, or you need newer macOS on Apple M4 Pro and Windows Server 2022.
- Runners in your own cloud. If CI must run in your own AWS, GCP or Azure account, WarpBuild BYOC runs there with your network rules.
- Wider Windows and macOS choice. Teams shipping desktop apps get Windows Server 2022 and 2025 and macOS 14, 15 and 26 on Apple M4 Pro.
- Whole disk snapshots. On Ubuntu cloud runners a job can boot from a saved disk snapshot, not just a few chosen folders.
Where bochka fits
Blacksmith and WarpBuild are CI services, and for GitHub Actions jobs one of them is the right pick. If what you are really running is agents, both make you start each machine from a workflow and lose it when the job ends. bochka gives each agent its own full VM that you start by API, keep as long as you like and reach from outside.
A machine that stays
An agent that installs tools and builds up state loses it when the job ends. Blacksmith keeps 5 sticky disk folders, evicted after 7 days; WarpBuild snapshots cost $0.025 an hour each, expire after 15 days and take 45 to 60 seconds to boot.
Reach in: hosting and a desktop
Neither runner takes inbound traffic, only debug SSH, so nobody can open the app an agent built or let a computer-use agent see a screen. bochka serves HTTPS on up to 50 ports, raw TCP and a 60fps desktop.
Agents you drive from outside, keys kept out
On both, machines start from GitHub workflows and secrets sit inside the job, where agent-written code can read them. bochka ships seven coding agents with a prompt, steer and events API, and a credentials proxy keeps real keys out.
Use bochka when your agents need their own machine outside CI: one you start by API, that keeps its disk, hosts what it builds, shows a desktop and runs coding agents you drive from outside. Compare it one to one: bochka vs Blacksmith and bochka vs WarpBuild.
Questions people ask
Is Blacksmith cheaper than WarpBuild?
In every scenario we priced, yes. CI runners come to $708 a month on Blacksmith and $864 on WarpBuild, and one machine running all month to $207 against $237. Both bill per minute of job time, and WarpBuild charges extra for cache and snapshots.
Which is faster, Blacksmith or WarpBuild?
We have no comparable measurement: neither was part of the same CPU benchmark run. Both compare their runners with GitHub's own, not with each other. bochka did 26.69 runs per second on the StarSling benchmark at 4 vCPU / 8 GB.
Can Blacksmith and WarpBuild run Docker?
Yes, both run Docker in their jobs. Blacksmith adds Docker layer caching and WarpBuild sells remote Docker builders. Both expose /dev/kvm only on x86-64 Linux runners.
Do Blacksmith and WarpBuild keep files between jobs?
Only partly. Blacksmith keeps up to 5 sticky disk folders per job, evicted after 7 days unused. WarpBuild destroys the disk but can boot a later Ubuntu runner from a disk snapshot, which is deleted after 15 days.
Can I run AI agents on Blacksmith or WarpBuild?
Partly. Blacksmith has a Testbox CLI in early beta for local agents and its own [code]smith agent, with a Sandboxes product marked coming soon. WarpBuild can host Claude Managed Agents sessions on its stock runners, each on a fresh VM torn down afterwards.
Do both have macOS and Windows runners?
Yes. Blacksmith runs macOS on Apple Silicon and Windows Server 2025 in public beta. WarpBuild runs macOS 14, 15 and 26 on Apple M4 Pro and Windows Server 2022 and 2025. bochka runs Linux only.
Try bochka
Install the CLI, then run bochka new.
Product preview