AI Sandbox — Code Execution

• Area: Intelligence
• Path: services/ai/sandbox
• Kind: Isolated execution of AI-generated code (Firecracker microVMs + language packs)
• Status: v0.1.0 — foundation landed 2026-05-24. HTTP daemon (koder-sandbox) + operator CLI (ksandbox) + Go SDK (engines/sdk/go/sandbox) ship with the subprocess runtime (real, testable). Firecracker driver is the typed stub returning SANDBOX-RUNTIME-UNAVAILABLE-001; rootfs catalog + boot pipeline + vsock agent track in sandbox#014. Pre-flip status — subprocess driver suitable for devCIsingle-tenant trusted-workload deployments; not yet for hostile multi-tenant traffic (gates G3 + G4 still open).

Role in the stack

sandbox is the missing primitive that unblocks the entire "agent that writes code" surface of the stack. Agents generate code; without a sandbox they can't run it (or worse, they run it on the host). Today every product-side prototype that wants "run this snippet" reinvents some half-baked Docker shell-out — slow, leaky, no quotas, no audit. This sector consolidates the capability into one secure isolation primitive that all AI-side consumers (kode, agents, workflow, kortex code-tools) plug into.

It is the Koder analog of E2B, Daytona, and Modal sandboxes — built on Firecracker (sub-200ms boot, ~5MB overhead per microVM) with a gVisor adapter for compatibility tail.

Boundary vs neighbors

• services/ai/runtime is disjoint — runtime serves models; sandbox runs user code.
• services/ai/kode, services/ai/agents, services/ai/workflow, services/ai/playground are the primary consumers.
• products/dev/kortex consumes via "Run this" panels in the IDE.
• services/ai/trace receives spans for every session + exec.
• infra/data/kdb-blob holds artifact storage (file IO + snapshots).
• The long-standing tracking ticket services/ai/ai/backlog/pending/017-pluggable-execution-sandbox-backends.md is realized by this sector.

Features (v1 target)

• Firecracker microVM runtime, warm pool per language
• Sub-200ms session create from warm pool; < 1s cold boot
• Language packs: Python 3.11, Node 20, Bash + GNU coreutils
• Sync + async exec, SSE stdout/stderr streaming
• Default-deny network with allowlist (DNS proxy + iptables)
• FS quota (default 1GB), CPU quota (cgroups v2), memory cap with OOM kill
• Hard-kill on quota breach with partial-output preservation
• File IO API (upload, download, list, delete) with quota awareness
• Snapshot/restore for replay + persistent agent loops
• Per-tenant concurrent-session + daily-minute quotas
• Audit log of network connection attempts

Realized surfaces (live)

Beyond the v1 target above, these are implemented and validated:

• Incus runtime driver (internal/runtime/incus.go) — warm pool, per-session

  VM with a bridge IP, file IO via incus file push/pull, listening-port discovery from /proc/net/tcp. Optional capabilities are type-asserted so drivers opt in:

  • Upstream — resolves a session's dev server to http://<bridge-ip>:<port>,

    consumed by Koder Jet for AI-sites sandbox-origin preview proxying (jet#149): GET /v1/sandbox/sessions/{id}/upstream, invalidated on session end. The preview port auto-detects by heuristic or can be pinned per session via POST /v1/sandbox/sessions/{id}/preview {port} (SANDBOX-008 manual override; honored only once that port is listening, else 503).

  • Terminal / Resizer — interactive shell over incus exec -t driven

    through a local PTY (creack/pty); WS /v1/sandbox/sessions/{id}/terminal bridges it (binary = stdin/stdout, JSON text = resize control). Real-VM validated on s.khost1. The subprocess driver omits the capability → 501.

• Per-session resource enforcement (SANDBOX-003) — CPUmemorydisk/pids

  caps applied as a live incus config set right after acquire (warm-pool instances are launched before the session's limits are known), with limits.memory.enforce=hard for OOM-kill over silent throttle. Validated in-guest (memory.max/pids.max). Each exec reports peak_memory_kib (from memory.peak) and an OOM verdict (SANDBOX-EXEC-KILLED-OOM, via a per-exec memory.events oom_kill delta). Network mode is enforced: none masks the NIC (default-deny baseline, VM-validated — no routable IP), open keeps the bridge, and allowlist (SANDBOX-013, live-verified 2026-06-02) resolves+pins FQDNsCIDRs at create (pure `internalnetpolicy pkg, with an SSRF/rebinding guard that rejects private/reserved destinations unless in the Koder-internal set) and enforces them via a per-instance **Incus network ACL** (security.acls.default.egress.action=reject + allow rules, DNS to the pinned resolver) attached to eth0, torn down on destroy. Integration test proved 1.1.1.1:443 reachable / 8.8.8.8:443` blocked. The remaining #013 item is the plan-tier gate (free→402), blocked on Koder ID entitlements (no tier claim / 402 infra yet).

• Snapshot/restore (SANDBOX-016) — POST .../snapshot archives the

  session FS (tar.gz over incus exec, runtime.Archiver) into a tenant-scoped blobstore.Store; POST .../restore unpacks it back. Pluggable store (FS backend now, kdb-blob drop-in later); 30-day default retention; real-VM validated (archive→extract across instances). Was a 501.

• sandbox.errors aggregation (SANDBOX-009) — per-session, multi-tenant

  ring (last 50) for browserruntime errors: `POST ...errors (in-VM collector) + GET .../errors` (host "N errors" badge). The Chromium collector, UI badge, and "Fix it" child agent.run remain cross-component.

• Host BTRFS subvolume SDK (pkg/host/btrfs, SANDBOX-006) — importable

  SubvolumeManager (CreateMountSnapshotUnmountStatus + ScheduleDelete/ConfirmDelete behind a 7-day grace) for KPC's persistent per-instance rootfs; command-seam tested. Merged VM+subvolume provisioner + real-btrfs test remain.

• fs.watch WS (SANDBOX-020, done) — runtime.Watcher capability; the

  Incus driver polls the tree (~300ms, diffs ListFiles) into createdmodifieddeleted events relayed over WS .../fs/watch (live file tree, "files changed" badge, preview auto-refresh). Real-VM validated.

• Streaming roadmap (split from the SANDBOX-012 omnibus): gRPC Sandbox

  service + REST parity (#021) DONE; remaining — VNC-over-WS (#022), per-tier rate limits + WS leak fuzz (#023).

Future integrations (direction-capture, needs RFC)

Two architecturally-natural integrations are triaged but gated on owner ratification via sandbox-RFC-002 (proposed — runtime tiers + placement, two scaling axes):

• kbox WASM runtime (SANDBOX-024, RFC-002 Axis 1 / Q1a=ratified) — a

  runtime.Kbox driver (session = host workspace; kbox run a WASI image; host-FS file IO; caps = Exec + File IO). Core + e2e shipped — real-kbox roundtrip GREEN on dev-linux-box (Exec captures output via the mounted workspace: run -d→wait→read /sandbox/.stdout|.exit→rm, since this kbox build's run is detached + reaps fast containers). Remaining: WASM- runtime e2e + ~10 ms perf gate (blocked on box wasmtime/WASI-image provisioning), Upstream cap, live streaming. The faster ephemeral tier below the sandbox, alongside Incus.

• fleet-scheduled placement (SANDBOX-025) — the hyperscale path above

  the sandbox: Fleet (infra-RFC-004) schedules sandbox sessions across a fleet of hosts instead of the current single-host warm pool, lifting the SANDBOX-019 ceiling and removing the SPOF. Joint RFC with infra/net/fleet.

Primary couplings

RFC and bootstrap

• RFC: sandbox-RFC-001-foundations.kmd — accepted 2026-05-09
• Bootstrap ticket: services/ai/backlog/done/134-sandbox-bootstrap.md
• Implementation tickets: services/ai/sandbox/backlog/pending/{001..005}
• Tracking ticket (long-standing): services/ai/ai/backlog/pending/017-pluggable-execution-sandbox-backends.md

Self-hosted-first analysis (5 gates)

• koder_kit Dart binding + KoderSandboxOutput widget (#017) — engines/sdk/koder_kit/lib/src/sandbox/ ships the Dart binding mirroring engines/sdk/go/sandbox/: SandboxClient(baseUrl, token, httpClient?) with createSession / exec / streamExec (SSE with sync fallback) / writeFile / readFile / close; models Session / CreateSessionOptions / ExecOptions / ExecResult / ExecFrame; typed SandboxException(code, message, status) with two-step JSON decode. ansi.dart minimal SGR parser (~80 LOC, zero deps) handles 8-color + bright variants, bolddimitalic/underline, resets, unknown SGR silently dropped. KoderSandboxOutput({frames, textStyle, stderrColor, autoScroll}) Stateful widget subscribes to the frame stream, renders into SelectableText.rich with auto-scroll, shows exit-code + duration status bar on done frame. Re-exported from the koder_kit barrel. 16 unit tests (parser + model round-trips + mock HTTP + typed-error rejection) PASS on dev-linux-dek. Unblocks playground cells, kortex Run panels, kruze console.

gRPC surface (SANDBOX-021, 2026-06-02)

A typed gRPC face (koder.ai.sandbox.v1.Sandbox) now sits alongside the REST router, reusing the same session.Manager (no logic fork) — for in-Stack consumers like services/ai/tools that want a typed client. This slice ships the session lifecycle + unary Exec; auth is a unary interceptor calling the shared auth.Middleware.Resolve (identical token rules to HTTP), with sxerr→gRPC status mapping and cross-tenant → NotFound. Listener on [server].grpc_addr (empty disables). Verified by a REST↔gRPC parity test (identical results against one Manager). File-op RPCs + server-streaming (ExecStream/WatchFiles) shipped; gRPC Exec p95 measured 5.8ms (bufconn+subprocess, 150ms bar). SANDBOX-021 complete — gRPC at full REST parity. (Production Incus-path exec p95 is a separate runtime-perf measurement.)

Koder ID auth — JWKS validator (SANDBOX-027)

The v0 static-key/dev auth was upgraded to verify real Koder ID access tokens locally. auth.NewMiddleware(Config{...}) optionally takes a JWKS Validator (engines/sdk/go/auth.JWKSValidator — the same pattern canvascacheembed use, no bespoke JWT); when configured, Resolve(ctx, token) (the SANDBOX-021 single decision point shared by the HTTP handler and both gRPC interceptors) verifies the bearer against the cached JWKS and projects the verified claims onto Identity{TenantID, UserID=sub, Scopes=scope}. The audience is enforced — [auth].audience (default sandbox) must match the token's aud, so a token minted for another Koder service is rejected (proven end-to-end with an RSA-signed JWKS fixture). A present-but-invalid token is rejected, never downgraded to the dev tenant. In prd the dev/static-key shortcuts are refused at startup (fail-loud) — the JWKS validator is the only valid prod auth path. Config: [auth] issuer_url / jwks_url (derived from issuer when empty) / audience, and top-level env. This makes per-tenant isolation real for consumers like Koder Kli once they present an aud=sandbox token (minted via the Koder ID token-exchange grant, id/engine#246).

WS terminal command override (SANDBOX-026)

The interactive terminal (GET /v1/sandbox/sessions/{id}/terminal) accepts ?cmd=<program> + repeatable ?arg=<argv> to launch a specific program in the PTY instead of the default interactive bash (runtime.TerminalOpts.Args; blank cmd → bash, backward compatible). The override is exec'd as argv to incus exec inside the per-tenant isolated instance — never through a host shell — so it carries no injection/escape surface. The Go SDK exposes it as sandbox.TerminalOptions{Cmd, Args}; Koder Kli uses it to launch the agent adapter directly (dropping its earlier exec <Bin>-over-bash workaround).