Buildkite vs GitLab.com hosted runners
Both are CI services that run each job on a fresh VM and delete it afterwards. Buildkite jobs can run 8 hours on Linux and keep chosen paths in cache volumes, while GitLab.com hosted runners stop every job at 3 hours and come built into every GitLab project.
The companies and products
Buildkite
Buildkite is a CI and software delivery platform, and hosted agents are the build machines it runs for you.
You write Buildkite pipelines, and each job runs on a fresh virtual machine that Buildkite starts for that job and destroys when it ends; only cache volumes carry files to the next job. Hosted agents come in Linux and macOS on Apple M4, with Windows in private preview, and you can also run Buildkite's agent on your own machines. It is a CI service, not an agent sandbox, so this page compares the part where they overlap: a machine where an agent or a job does its work.
- Where it runs
- Hosted agents run only in US East Coast data centers
- Compliance
- Maintains a SOC 2 Type II report
GitLab.com hosted runners
GitLab.com hosted runners are the machines that run GitLab CI/CD jobs for projects on GitLab.com, paid for in compute minutes.
This is a CI service, not a sandbox API. Each job gets a new VM on Google Cloud (Linux and Windows) or on AWS Mac hardware (macOS), with passwordless sudo, and the VM is deleted right after the job. It is built for building, testing and deploying the code in a GitLab project, and GitLab's Duo Agent Platform also runs Claude Code and Codex agents as CI/CD jobs on these runners.
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “43rd cheapest of 367” means 42 of the other 366 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | Buildkite: price rank | GitLab.com hosted runners: price rank | bochka: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month A coding agent building a whole app needs one machine for hours; Buildkite gives it up to 8 hours with cache volumes, GitLab cuts it off at 3 hours with a clean disk, and bochka keeps the same VM with no session cap. | 43rd of 367, with limits $11,648 a month | 347th of 367, with limits $14,454 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB An assistant that lives on its machine all month fits neither CI service, since jobs end after 8 hours on Buildkite and 3 on GitLab; bochka has no session cap on paid plans. | 351st of 366, with limits $17,564 a month | 353rd of 366, with limits $21,858 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month | 315th of 345, with limits $1,587 a month | 320th of 345, with limits $1,896 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month Neither CI service takes inbound traffic or hands machines to your own users; bochka gives each sandbox HTTPS on up to 50 ports and raw TCP. | 326th of 361, with limits $4,984 a month | 331st of 361, with limits $6,133 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month | 316th of 360, with limits $23,881 a month | 318th of 360, with limits $29,762 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 324th of 364, with limits $3,884 a month | 330th of 364, with limits $4,758 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month CI is what both are built for; GitLab's runners are the native choice for a GitLab project, Buildkite suits teams that want its pipelines and longer jobs, and bochka runs CI on full VMs with a disk that stays. | 203rd of 377, with limits $2,318 a month | 128th cheapest of 377 $2,089 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB A dev box has to stay up and keep its packages; both CI services reset the machine after each job, while bochka saves the whole disk every minute. | 354th of 368, with limits $715 a month | 358th of 368, with limits $805 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Buildkite's macOS desktop access is for a person debugging a job and GitLab's runners are headless; bochka streams a 60fps Linux desktop with computer use. | 70th of 369, with limits $3,500 a month | 342nd of 369, with limits $4,278 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month | 142nd of 367, with limits $1,772 a month | 144th of 367, with limits $2,118 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 142nd of 346, with limits $2,354 a month | 143rd of 346, with limits $2,854 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 317th of 338, with limits $46,814 a month | 319th of 338, with limits $58,429 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month | 85th of 360, with limits $1,184 a month | 87th of 360, with limits $1,383 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month GitLab offers a Linux runner with an NVIDIA T4 on Premium and Ultimate; Buildkite's hosted shapes and bochka's are CPU only. | Can't run it | Can't run it | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month Both run macOS: Buildkite on Apple M4 with current Xcode, GitLab on Apple silicon in beta for Premium and Ultimate; bochka runs Linux only. | 35th of 36, with limits $2,310 a month | 24th of 36, with limits $1,069 a month | Can't run it |
Key features
| Feature | Buildkite | GitLab.com hosted runners | bochka |
|---|---|---|---|
| Kind of product | CI platform; hosted agents run jobs from Buildkite pipelines | CI runners; jobs run from a GitLab project's .gitlab-ci.yml | Agent sandboxes you create and drive by API, CLI or SDK |
| Isolation | A VM per job, destroyed with its data afterwards; Apple Virtualization framework on macOS | A new VM per job on Google Compute Engine, deleted right after; Linux jobs run in a privileged Docker container | Full VM with its own kernel on dedicated AMD Ryzen 9 9950X hosts, with root and systemd |
| How long a job runs | Up to 8 hours on Linux and 4 on macOS, longer on request | Up to 3 hours, whatever the project setting | No session cap on paid plans; you set the time to live |
| Files between runs | Cache volumes keep chosen paths after successful jobs, best effort, up to 14 days (Pro and Enterprise) | Shared cache up to 5 GB per archive, removed after 14 days unused, or artifacts | The whole disk is saved every minute and on stop; a stopped sandbox costs nothing |
| Fork | Each job gets its own copy of the last committed cache volume | No; every job starts from a fresh VM and a container image | Fork a running sandbox with its disk |
| Docker and nested virtualization | Docker on Linux; macOS has the Docker CLI but no daemon; /dev/kvm not documented | Docker-in-Docker through privileged mode; /dev/kvm not documented | Docker, root, systemd and /dev/kvm on every machine |
| Reaching the machine | No exposed ports; a terminal into a running job through Buildkite's UI or bk job ssh | Inbound traffic is blocked, and hosted runners have no interactive web terminal | SSH, HTTPS URLs for up to 50 ports, raw TCP, dedicated IPv4 or IPv6 |
| Desktop and computer use | Browser-based desktop access to debug a running macOS job; no computer-use API | Headless; macOS runners do not support jobs that need UI interaction | 60fps Linux desktop with Chromium, computer use and browser control |
| Coding agents | You install Claude Code in your image or step and run it as a pipeline job | Duo Agent Platform runs Claude Code, Codex and other agents from issues and merge requests, on Premium and Ultimate | Claude Code, Codex and five more preinstalled, driven from outside with a prompt, steer and events API |
| Keeping API keys out | Anthropic proxy: jobs call it with the job token and Buildkite sets the key | GitLab-managed credentials for its Claude and Codex agents only; other secrets enter as CI/CD variables | Credentials proxy for model providers and any HTTPS API; switch a credential to another account when credits run out |
| Operating systems and GPUs | Linux (ARM64 on Enterprise), macOS on Apple M4 with Xcode, Windows in private preview; no GPUs | Linux x64 and Arm64, Windows (beta), macOS on Apple silicon (beta), and a Linux runner with an NVIDIA T4 | Linux x64 only, no GPUs |
| Biggest machine | Linux up to 8 vCPU / 32 GB on Pro and 64 vCPU / 256 GB on Enterprise; macOS up to 12 vCPU / 56 GB | Linux up to 32 vCPU / 128 GB on Premium and Ultimate | Up to 16 vCPU / 32 GB on the $100 plan and above |
| Self-hosted runners | Run Buildkite's agent on your own machines next to hosted ones | Open-source GitLab Runner (MIT) on your own machines, using no compute minutes | No; bochka hosts every sandbox |
| Regions | US East Coast only | US (Google Cloud us-east1 by default for Linux) | EU only (Germany, Finland, France), with EU data residency |
Every feature Battleships tracks (43)
| Feature | Buildkite | GitLab.com hosted runners | bochka |
|---|---|---|---|
| Full VM (own kernel) | Yes | Yes | Yes |
| Docker inside | Yes | Yes | Yes |
| Nested virtualization | No | No | Yes |
| Root | Not published | Yes | Yes |
| Systemd | No | No | Yes |
| Desktop GUI | Yes | No | Yes |
| Browser | Yes | Yes | Yes |
| Computer-use API | No | No | Yes |
| SSH | Yes | No | Yes |
| Persistent disk | Yes | No | Yes |
| Pause/resume | No | No | Yes |
| Automatic snapshots | Yes | No | Yes |
| Fork/clone | Yes | No | Yes |
| Live fork (no pause) | No | No | Yes |
| Memory fork | No | No | No |
| Custom image (Docker or snapshot) | Yes | Yes | Yes |
| ≥24 h sessions | No | No | Yes |
| Idle auto-stop | Yes | Yes | No |
| Wake on request | No | No | No |
| HTTPS preview URLs | No | No | Yes |
| Raw TCP inbound | No | No | Yes |
| Public IPv4 | No | No | No |
| Egress allowlist | No | No | No |
| Secret proxy | Yes | Yes | Yes |
| Secret proxy for any API | No | No | Yes |
| Preinstalled agents | No | Yes | Yes |
| Agent harness API | No | Yes | Yes |
| MCP server | Yes | No | No |
| GPU | No | Yes | No |
| Windows | Yes | Yes | No |
| Arm64 | Yes | Yes | No |
| Self-hosting | Yes | Yes | No |
| BYOC | No | No | No |
| Open source | No | Yes | No |
| Scoped API keys | Not published | Not published | Yes |
| Spend limits | Not published | Not published | Yes |
| Usage per sandbox (API) | No | No | Yes |
| Audit logs | Not published | Yes | No |
| SSO | Yes | Yes | No |
| SOC 2 | Yes | Yes | No |
| HIPAA | No | No | No |
| EU data residency | No | No | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
We have no comparable CPU measurement for Buildkite hosted agents or GitLab.com hosted runners: neither was part of the StarSling hpc-sandbox-benchmarks run. On that Node.js benchmark at 4 vCPU / 8 GB, bochka did 26.69 runs per second on dedicated AMD Ryzen 9 9950X hosts.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. Buildkite and GitLab.com hosted runners have no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | Buildkite, a month | GitLab.com hosted runners, a month | bochka, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $166 · Pro · with limits | $119 · Premium · with limits | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $715 · Pro · with limits | $805 · Premium · with limits | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $7,677 · Pro · with limits | $9,491 · Premium · with limits | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $2,204 · Pro · with limits | $2,658 · Premium · with limits | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $4,787 · Pro · with limits | $5,896 · Premium · with limits | $2,000 · $2000/mo · with limits |
Buildkite: Pro costs $30 per active user a month and includes 4,000 Linux vCPU minutes; after that Linux is $0.004 per vCPU minute and macOS $0.02, metered to the second. Enterprise pricing is custom.
GitLab.com hosted runners: Jobs use compute minutes equal to run time times a cost factor, from 1 for small Linux up to 12 for the biggest machines. Premium is $29 per user a month billed yearly with 10,000 minutes per group, and extra minutes cost $10 per 1,000, bought ahead.
GitLab.com hosted runners: Plans start at $20 a month, and the whole fee is usage credit spent per second on running sandboxes. A stopped sandbox costs nothing and keeps its disk.
Limits in those scenarios
Neither CI service fits the agent scenarios as a sandbox. Buildkite jobs end after 8 hours and keep only cache volume paths, and Pro shares one pool of 256 Linux vCPU, so 300 or 1,000 machines at once mostly wait in the queue. GitLab jobs end after 3 hours with nothing kept on disk, it publishes no limit on runners at once, and new jobs stop when a group's minutes run out. bochka queues starts above 90 a minute, so the 1,000-job burst needs more starts per day than its $2,000 plan allows.
| Scenario | Buildkite | GitLab.com hosted runners | bochka |
|---|---|---|---|
| One coding agent, workdays |
|
| None that bite |
| One agent that never stops |
|
| None that bite |
| 50 agents in parallel |
|
| None that bite |
| Sandboxes for your users |
|
| None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use Buildkite when
you want a CI platform with jobs up to 8 hours, cache volumes, macOS on Apple M4, or Linux machines up to 64 vCPU on Enterprise.
- Longer jobs with cache volumes. Useful for long builds: Linux jobs run up to 8 hours, and cache volumes carry chosen paths to the next job.
- macOS on Apple M4. Useful for iOS and Mac builds: Mac agents run current Xcode on Apple M4, up to 12 vCPU and 56 GB.
- Very large Linux machines. Useful for heavy builds on Enterprise: Linux shapes go up to 64 vCPU and 256 GB, with ARM64 too.
Use GitLab.com hosted runners when
your code lives on GitLab and you want CI that works with no setup, a T4 GPU runner, or free self-hosted runners.
- Built into GitLab. For a GitLab project, hosted runners work from the first pipeline, with caches, artifacts and no extra service to wire up.
- A GPU and more platforms. Teams that need a T4 GPU, Windows, Arm64 or macOS in CI get a hosted runner for each.
- Free self-hosted runners. The open-source GitLab Runner runs on your own machines, and those jobs use no compute minutes.
Where bochka fits
Buildkite and GitLab are CI services, and bochka is not one. If what you are really running is agents that work for hours or days, take inbound traffic or need a desktop, both CI services make you rebuild the machine every job. bochka gives each agent one full Linux VM that keeps its disk.
A machine that outlives the job
Buildkite stops a job after 8 hours and GitLab after 3, and both delete the VM. Cache volumes or a 5 GB shared cache bring back some files, but the agent still reinstalls tools each run. bochka keeps the whole disk with no session cap.
Reach it from outside
Neither CI service lets traffic in, so an agent's preview server, webhook or app has nowhere to be seen without a tunnel inside the job. bochka gives HTTPS URLs on up to 50 ports, raw TCP, SSH and a 60fps desktop.
Agents you drive from outside
On Buildkite an agent is a pipeline step you read through logs, and on GitLab it starts from an issue or merge request. bochka lets any program prompt and steer agents inside, with a credentials proxy for any HTTPS API, not only Anthropic or GitLab's agents.
Use bochka when what you run is agents on Linux for hours or days: one full VM that keeps its disk, Docker and /dev/kvm, a 60fps desktop, URLs you can reach, and coding agents you drive from outside. Compare it one to one: bochka vs Buildkite and bochka vs GitLab.com hosted runners.
Questions people ask
Is Buildkite cheaper than GitLab.com hosted runners?
It depends on the load. For one coding agent on workdays, GitLab comes to $119 a month against $166 on Buildkite, helped by included minutes. For one agent running all month Buildkite comes to $715 against $805, and for 50 agents in parallel $7,677 against $9,491; bochka is $20, $26 and $317 for the same three.
Which is faster, Buildkite or GitLab runners?
We have no comparable CPU measurement for either one. bochka did 26.69 runs per second on the StarSling Node.js benchmark at 4 vCPU / 8 GB.
How long can a job run on Buildkite and on GitLab?
Buildkite hosted jobs run up to 8 hours on Linux and 4 on macOS, and support can allow longer. GitLab.com hosted runner jobs stop at 3 hours, whatever the project sets. A bochka sandbox has no session cap on paid plans.
Can Buildkite and GitLab runners run Docker?
Yes on Linux. Buildkite Linux agents run Docker, while its macOS agents have the Docker CLI but no daemon. GitLab runs Docker-in-Docker through privileged mode. bochka has Docker, systemd and /dev/kvm on every machine.
Do files survive between jobs?
Only partly. Buildkite saves the cache volume paths you declare after a successful job, best effort for up to 14 days. GitLab carries files through a shared cache of up to 5 GB per archive or artifacts. bochka saves the whole disk every minute and on stop.
Can I run Claude Code on Buildkite or GitLab?
Yes, as a CI job on both. On Buildkite you install it in your step and its Anthropic proxy supplies the key; on GitLab Premium and Ultimate the Duo Agent Platform runs it from issues and merge requests. On bochka it is preinstalled and you prompt and steer it through an API.
Try bochka
Install the CLI, then run bochka new.
Product preview