Ephemeral compute lease protocol — TTL + idle-aware transient infra

Status

RATIFIED 2026-05-30. Opened the same day after a long multi-session run saturated s.khost1 to load average 36 (16 users, 7+ concurrent /k-go sessions). Each session spun up a dev build VM/container per heavy-work-isolation.kmd; nothing told the host which were idle or done, so incus exec started timing out and SSH reset under contention — work stalled not on code but on uncontrolled transient-infra lifecycle.

Ratification decisions (open questions, resolved 2026-05-30)

1. Heartbeat = the Koder session-lock. An active session's

   meta/context/notices/active/lock-* IS the lease renewal; archiving the lock IS the release. Reuse-first: one liveness system, not two. (A lease without a session lock — e.g. a CI run — uses an explicit renew call.)

2. Under pressure: hibernate by default, preserving state; priority labels

   let the scheduler evict-early the lowest-priority idle leases when snapshot cost outweighs the benefit.

3. Usage signal: hybrid — observed (kbox stats / koder-mon) with an

   explicit override (the session lock / renew). Catches forgetful requesters without mis-reaping a warm-but-paused lease.

4. Admission under pressure: queue-with-priority (not hard-deny) — a request

   at high load waits for capacity instead of thrashing the host. (Phase 1.)

5. Naming + home: daemon koder-leased (systemd, on Koder Kodix); CLI

   klease (+ kbox lease … convenience alias); component infra/net/lease/ (sibling of infra/net/kbox). Final forms registered per specs/naming/forms.kmd at Phase 0.

Summary

Add a lease layer over the Koder runtime substrate (kbox; incus today): a native protocol where a requester gets compute (a container or microVM) bound to a time-to-live and a usage signal, and a control plane on Koder Kodix auto-reaps it at expiry and hibernates it when idle — proactively under host pressure. This is the lifecycle/orchestration tier that infra-RFC-001 (the kbox runtime substrate) and infra-RFC-002 (a fixed CI runner pool) both lack: it makes transient infra self-cleaning and load-elastic instead of an ever-growing pile of orphan VMs.

The manual servers.md convention — humans annotating a VM "pode ser destruída" (14 instances today) — is the precursor this automates and enforces.

Motivation

Today the Stack creates transient compute with no lifecycle authority:

• Dev build VMs (dev-linux-{dek,kruze,kvs,kvg,kdb,koda,...}) are created

  ad-hoc per heavy-work-isolation.kmd, then linger forever. servers.md hand- labels most "pode ser destruída" — but nothing acts on it.

• CI runners (infra-RFC-002) are a fixed pool, sized for peak, idle the

  rest of the time.

• Nothing signals "in use" vs "idle", so the host can't reclaim resources;

  10 half-idle VMs at load 36 starve the one that's actually building.

• Nothing has a deadline, so cleanup is a human chore that doesn't happen.

The result is the exact failure observed: a shared host driven into thrash by sessions that each, locally reasonably, asked for a VM and never gave it back.

This is a solved pattern elsewhere — Fly.io Machines (auto-stop on idle + start on request = scale-to-zero), k8s (Lease + Pod TTL controller + descheduler eviction under pressure), Nomad — so per reuse-first.kmd we adopt the shape, and per self-hosted-first.kmd we build it native on kbox + Koder Kodix rather than bolting on an external orchestrator.

Design

The lease object

A lease is a contract for compute:

Lease {
  id            string          // kls-<short>
  requester     KoderID         // who owns it (and may renew/release)
  workload      WorkloadSpec    // kbox image/profile + resources (CPU/mem/storage/net)
  created_at    time
  expires_at    time            // HARD deadline: reaped at/after this unless renewed
  renew_grace   duration        // how long past last heartbeat before "idle"
  usage         UsageSource     // explicit-ping | observed | hybrid
  state         LeaseState
  labels        map             // priority, project, surface (e.g. "build/kruze")
}

WorkloadSpec is kbox-native: it names a kbox image + profile (the kbox profile from infra-RFC-001 / BOX-129) + resource caps; the lease control plane never re-implements provisioning, it drives kbox.

Lifecycle states

requested → admitted → active → idle → hibernated → expired → reaped
                ↑__________________________│ (resume on use / renew)

• requested → admitted: admission control (see §Mechanisms) accepts or

  queues/denies based on host pressure.

• active: running, recently signalled in-use.
• idle: no usage signal for renew_grace. Eligible for hibernation.
• hibernated: suspended to disk (CPU/RAM reclaimed); resumes on next use or

  renew. With a microVM backend (BOX-131) this is a millisecond snapshot/restore.

• expired → reaped: expires_at passed (and not renewed) → control plane

  deletes the workload + volumes (honoring a keep label / final snapshot if requested).

Three mechanisms

1. TTL + renewal (lifecycle). Every lease has expires_at. A renew/heartbeat

   pushes it forward. No renewal → expiry → reap. This is what kills orphan VMs: a build VM is a 2h renewable lease; the owning session dies → no renew → reaped. No silent eternal VMs.

2. Usage signal (idle detection). Two sources, hybrid by default:
   • explicit — the requester pings "in use" (natural fit: the existing *oder

     session-lock*can double as the heartbeat — see Open questions).

   • observed — the control plane reads CPU%/activity. Koder already exposes

     this: kbox stats ?stream=true (CPU%NetIOmem) and the observe stack (koder-mon/apm). Below an idle threshold for renew_grace → idle.

3. Saturation-aware scheduler/reaper (the load-36 fix). A daemon watches host

   load + the lease set and:

   • hibernates idle leases when load crosses a high-water mark (reclaim

     CPU/RAM, preserve state) — lowest-priority + longest-idle first;

   • reaps expired leases unconditionally;
   • admitsqueuesdenies new lease requests under pressure (so a request at

     load 36 is queued, not granted-into-thrash — exactly what should have happened to this session).

Integration — Koder Kodix ↔ kbox ↔ observe (native)

The owner's framing: this is a native framework wiring the OS to the runtime, not a bolt-on. Each tier is already Koder-native:

• Koder Kodix ships the control-plane daemon (proposed koder-leased, a

  systemd service): the lifecycle authority — admission, TTL reaper, idle hibernation, the lease registry (kdb or a local store). It is the component that "deletes the VM at expiry," as the owner described.

• kbox is the execution backend: koder-leased drives kbox run --system

  (BOX-129, just landed slices 1-2), kbox suspend/resume, kbox rm, profiles.

• kbox microVM (BOX-131) is the ideal hibernation substrate: firecracker

  snapshots/restores in ms → real scale-to-zero for idle leases.

• observe (koder-mon + kbox stats) feeds the usage signal for free.

Koder Kodix declares the lifecycle policy, kbox executes it, observe informs the decision. One framework binds three Koder products.

A thin client (kbox lease create/renew/release/list, or a klease CLI / API) is how /k-go, CI (infra-RFC-002 runners become leases), and humans request compute. heavy-work-isolation.kmd build VMs become the first consumer.

Phasing

Non-goals

• Not a cluster scheduler. Single-host (s.khost1) first, like infra-RFC-001

  Phases 0-2. Multi-host placement revisits when kbox cluster-state (kbox#115) ships.

• Not replacing infra-RFC-002. The runner pool stays; its runners become

  leases (ephemeral, scale-to-zero) instead of a fixed always-on pool.

• Not for permanent services. Production daemons (Koder ID, Flow, observe)

  are NOT leases — they are unbounded. Leases are for transient compute only.

Open questions

1. Heartbeat = session-lock? The Koder session-lock notices

   (meta/context/notices/active/lock-*) already mark "a session is working on X." Unifying them with the lease heartbeat would mean: an active session's lock is the renewal; archiving the lock is the release. Elegant, but couples two systems — decide.

2. Hibernate vs evict under pressure. Default hibernate (preserve state). But

   an idle+near-expiry lease — evict early to save the snapshot cost? Priority labels drive it; need the policy.

3. Usage signal granularity. explicit-only is precise but needs cooperation;

   observed-only catches forgetful requesters but mis-flags a VM that's "warm but paused." Hybrid (observed + explicit override) is the proposed default — ratify.

4. Admission policy. Hard deny vs queue-with-priority when load > threshold?

   Queue is friendlier (the request lands when capacity frees) but needs a waiting client.

5. Naming + home. koder-leased daemon + klease/kbox lease CLI; component

   at infra/net/lease/ (sibling of infra/net/kbox) or a kbox sub-surface? Per specs/naming/forms.kmd + policies/naming-aliases.kmd.

Acceptance criteria for this RFC

• [x] Owner ratifies the lease object + lifecycle states + the three mechanisms

  (2026-05-30).

• [x] Open questions resolved — see §Ratification decisions.
• [x] Phase 0 ticket opened: infra/net/lease/backlog/pending/001-phase0-lease-registry-ttl-reaper.md.

Cross-references

• infra-RFC-001 — kbox runtime substrate (leases run on it; BOX-129

  system-mode + BOX-131 microVM are direct dependencies of Phases 2-3).

• infra-RFC-002 — CI runner pool (its runners become leases under this

  protocol; the two compose: pool = a set of standing leases).

• heavy-work-isolation.kmd — the dev build VMs are the first consumer.
• self-hosted-first.kmd / reuse-first.kmd — native build on kbox/observe;

  shape borrowed from Fly Machines + k8s lifecycle.