KVS-RFC-003 slice d — Virtual always-fresh working copy
One line. Eliminate the frozen checkout entirely: a session's working copy is, logically,
origin@latest+ its own tracked deltas. A read of an untouched file resolves from current origin; "stale" becomes structurally impossible, with zero extra trees for N sessions.
1. Where slices c/b leave us (2026-06-25)
- slice c (freshness hook) — warns when a checkout drifts; auto-heals a
per-session worktree by calling
kvs wc sync. Mitigation, not elimination. - slice b (
kvs wc sync) — fast-forwards the checkout onto origin, keepingedits. The checkout is still a materialized point-in-time tree: between syncs it ages, and a sync is a discrete catch-up, not continuous freshness.
Both reduce the staleness window; neither removes the root cause (D12): the working copy is a frozen snapshot that drifts the instant a peer pushes. Slice d removes the frozen snapshot.
2. The model
The #188 no-index model already says: *the working copy is an auto-commit; the shared .git/index is never the mutable state*. Slice d takes it to its end:
- Base = a moving ref, not a checked-out tree. The logical base is
origin/master@now, re-resolved per read — never copied to disk as a frozen tree. - Reads resolve-on-read. Reading an untouched path returns the blob from the
current base (
git cat-file blob origin/master:<path>semantics, cached). No file ages because nothing is frozen. - Edits are deltas (an overlay). A write records a delta in a per-session overlay
(the #210 autosnapshot ref
refs/kvs/wc/<id>is the natural home). The "working tree" a tool sees = base-resolved ⊕ overlay. - Commit = block-merge (#206) + CAS (#208) onto current base. Exactly today's
advanceFinalize, except the parent is re-read at finalize (it already is — the CAS loop re-reads HEAD on miss). Non-overlapping peer edits never conflict; an overlapping path 3-way-merges via the AST/brace driver.
Net: a session can run for hours; an untouched file always reflects the latest origin; an edited file holds the local delta; commit reconciles. No drift, no add/add from a frozen base, no gutting from a stale tree.
3. Mechanism — the design fork
| Opt | What | Transparent to arbitrary tools? | Cost | Cross-platform |
|---|---|---|---|---|
| A — FUSE overlay | userspace FS: reads hydrate-on-read from the base via the object store, writes land in an upper delta dir | Yes (editorsgitgrep see a normal tree) | FUSE dep; per-read latency → needs an aggressive blob cache | Linux strong; macOS (macFUSE) weak; Windows separate |
| B — git index virtualization (Scalar/GVFS-style) | virtualize .git/index + on-demand object hydration; git itself drives freshness |
Yes (it is git) | heavy; close to a git fork; complex invalidation | git-native, but the virtual-FS hook is platform-specific |
| C — continuous-rebase (slice b on a tight loop) | keep a real checkout, kvs wc sync on a sub-second/edit-boundary cadence |
Yes | simplest (extends shipped slice b); still a materialized tree — freshness is bounded by the loop, not structural | trivial |
| D — KVS-mediated FS | all file access goes through kvs (a read/edit API or shim); no materialized checkout | only for tools that go through kvs | KVS-native, no FUSE; breaks any tool that reads the raw FS | trivial |
4. Recommendation — A (FUSE), staged on Linux first; C as the interim
Long-term (D12 root-cause + transparency): only A and B make staleness structurally impossible while staying transparent to the arbitrary tools an AI session and a human use (editors, git, grep, build tools). Between them, A (FUSE overlay) is the KVS-native fit: KVS already owns the object model (#188) and the overlay ref (#210); a FUSE layer that resolves-on-read from the KVS object store and writes deltas to the overlay is a clean extension, not a git fork (B is heavier and its virtual-FS hook is as platform-bound as FUSE without the simplicity).
Trade-off accepted: FUSE is a real dependency and macOS/Windows need separate backends. Mitigate by staging: the AI fleet runs on Linux (s.khost1, the build hosts, CI) — ship the Linux FUSE backend first, where 100% of the stalenessgutting incidents have occurred. Desktop (macOSWindows) keeps the shipped slice b/c (sync + warn) until a native backend lands — no regression, just not yet structural there.
Interim until the FUSE backend lands: tighten C — have the slice-c hook (and an optional kvs wc watch-driven loop) call kvs wc sync aggressively so the materialized-tree window shrinks toward zero. This is already 90% built (slice b + auto-wiring) and buys most of the benefit while A is designed/implemented.
5. Slices
- d0 — object-resolution cache. A fast
base:<path> → blobresolver over theKVS object store (content-addressed; invalidated when the base ref advances). The shared primitive both A and a tighter C need. Benchmark resolve-on-read latency.
- d1 — FUSE read path (Linux). Mount a per-session FS: reads resolve from the d0
cache at
base@now; the overlay ref (#210) supplies edited paths. Read-only proof: an untouched file always reflects the latest origin with nowc sync. - d2 — FUSE write path + commit. Writes land in the overlay;
kvs wc commitfinalizes the overlay via the existing
advanceFinalize(#206/#208) onto the re-read base. End-to-end: edit → commit with zero frozen tree. - d3 — fleet rollout. Default the FUSE working copy for fleet/CI sessions
(Linux); desktop stays on slice b/c. Telemetry: staleness incidents → 0.
6. Non-goals
- Replacing slice b/c on desktop before a native backend exists (they stay the safe
default there).
- A bespoke object store — d0 reuses the KVS/#188 object model, not a new substrate.
- Refreshing the shared mutable tree (RFC-003 §6 non-goal still holds — virtual
copies are per-session by construction, which is exactly why this scales to N with no per-session disk: hyperscale D4).
7. Open questions
- d0 cache: in-process vs a shared local daemon (one cache for all sessions on a host)?
A host-shared daemon amortizes hydration across sessions (hyperscale) — likely right.
- Overlay GC: when is a session's
refs/kvs/wc/<id>overlay reclaimed? (Tie tokvs session end+ a TTL sweep.) - Interaction with virtual branches (#076): an overlay is a lightweight vbranch —
unify the two models or keep distinct?
Slice d0 SHIPPED (2026-06-27, /k-go) + §7 open-question resolved
§7 in-process-vs-daemon → resolved: in-process library first (D9 incremental/ reversible). A host-shared resolution daemon (amortizing hydration across sessions on one host) is a real hyperscale win but a later optimization — ship the library now, promote to a daemon when multi-session-same-host amortization is measured-needed.
internal/core/baseresolve.Resolver (in-process): SetBase(commit) (advancing drops the cache — entries are base-specific; same-base is a no-op so a hot path may SetBase(origin@now) freely), OID(path)/Blob(path) (cached path→oid via rev-parse <base>:<path>; oid→bytes is content-addressed/cheap; negatives cached). Concurrency-safe; doesn't poison the cache if the base advances mid-read. Direction- agnostic: the shared primitive both mechanism A (FUSE) and the interim C (tight continuous-rebase) build on. Tests (resolve+cache, drop-on-base-advance, absent→negative) + bench green on dev-linux-kvs: cached hit ~80 ns/op (vs a ~ms git shell-out — the cache earns its keep). Next: d1 (FUSE read path, Linux) over this resolver.
Slice d1a SHIPPED (2026-06-27, /k-go): the read core (split of §5 d1)
§5 d1 (FUSE read path) splits cleanly into d1a — the read core (done here) and d1b — the FUSE wiring (next). d1a is the logic the FUSE nodes call, and is also exactly what mechanism C (tightened continuous-rebase) reads through — so it ships once, dep-free and mount-free, fully unit-tested:
internal/core/baseresolve/tree.go:Tree(dir)(mode-awarels-tree -z) +Stat(path)— the directory/stat companion to d0's path→blob, resolve-on-read atbase@now.internal/core/vwc/provider.go: the composedoverlay ⊕ baseread model(
Stat/Blob/Tree, overlay-first, Tree-union with overlay-override). nil overlay = the d1a read-only proof; theOverlayinterface is the seam d2 (write path) fills.
Tests assert the core guarantee — advancing the base reflects the latest content on read with no re-checkout — so staleness is structurally impossible at the read layer (D12 root-cause). Green on dev-linux-kvs.
d1b (next): hanwen/go-fuse/v2 read-only nodes over vwc.Provider + kvs wc
mount + real-mount integration test. Feasibility confirmed: dev-linux-kvs carries /dev/fuse + fusermount3 + CAPSYSADMIN.
Slice d1b SHIPPED (2026-06-27, /k-go): the FUSE mount
§5 d1 is complete (d1a read core + d1b FUSE wiring). internal/core/wcfuse (//go:build linux) is a read-only hanwen/go-fuse/v2 filesystem over vwc.Provider — Lookup/Getattr/Readdir/Open/Read/Readlink resolve at base@now, git modes mapped to FUSE, zero attr/entry timeout, a refresher that re-fetches + advances the base. CLI: kvs wc mount/unmount. A real-mount integration test on dev-linux-kvs (root + devfuse) proves the §2 guarantee end-to-end: an untouched file read through the mount reflects the latest origin after a base advance, with no re-checkout — PASS. go-fuse is an add-only dep; desktop keeps slice b/c (§4).
Next: d2 (FUSE write path — overlay + advanceFinalize onto the re-read base; the vwc.Overlay seam is ready) → d3 (Linux fleet/CI default).
Slice d2a SHIPPED (2026-06-27, /k-go): the FUSE write path
The write half of §5 d2 landed (d2b = commit-finalize remains). New pkg internal/core/wcoverlay is the writable upper-dir overlay (impl of vwc.Overlay: CopyUpWhiteoutMkdir + StatBlobChildren/Removed). vwc.Provider gained Removed (whiteouts) + BaseStat. internal/core/wcfuse gained read-WRITE nodes (CreateMkdirUnlinkRmdirSetattr/Rename + real-fd write handle, copy-up on first write) and MountRW; kvs wc mount is read-write by default (--ro for read-only). A real r/w FUSE mount test proves edits land in the overlay, the base bytes are never touched, deletes white-out, and readdir composes — PASS on dev-linux-kvs.
Next: d2b — kvs wc commit finalizes the upper delta (added/edited + whiteouts) via advanceFinalize (#206/#208) onto the re-read base, then clears the overlay (durable form = #210 autosnapshot ref). Then d3 (Linux fleet/CI default).
Slice d2b SHIPPED (2026-06-27, /k-go): kvs wc commit finalizes the overlay
§5 d2 is complete. internal/core/wcfinalize.Finalize applies the overlay delta (added/edited blobs + whiteouts, dir-whiteouts expanded) onto the re-read base (temp index → read-tree base → update-index → write-tree → commit-tree -p base): peers' non-overlapping work is carried, only the overlay's paths change (no silent revert), symlinks/exec preserved, ErrNothingToCommit on a no-op tree. wcoverlay gained WalkEntryModeClear. CLI kvs wc commit --overlay <dir> [-m][--onto][--clear] produces the commit and leaves publish to the established CAS push / kvs sync (never advances a ref or touches the shared tree — safe on the concurrent monorepo). Test PASS off-laptop; with the d2a mount test (shared upper-dir contract) the full edit→mount→commit loop is proven.
Next: d2b-tail (auto-publish CAS + auto-clear, so it's one command) → d3 (Linux fleet/CI default + telemetry staleness→0; bundle the kvs release).