KVS-RFC-003 slice d — Virtual always-fresh working copy

proposed

One line. Eliminate the frozen checkout entirely: a session's working copy is, logically, origin@latest + its own tracked deltas. A read of an untouched file resolves from current origin; "stale" becomes structurally impossible, with zero extra trees for N sessions.

1. Where slices c/b leave us (2026-06-25)

  • slice c (freshness hook) — warns when a checkout drifts; auto-heals a

    per-session worktree by calling kvs wc sync. Mitigation, not elimination.

  • slice b (kvs wc sync) — fast-forwards the checkout onto origin, keeping

    edits. The checkout is still a materialized point-in-time tree: between syncs it ages, and a sync is a discrete catch-up, not continuous freshness.

Both reduce the staleness window; neither removes the root cause (D12): the working copy is a frozen snapshot that drifts the instant a peer pushes. Slice d removes the frozen snapshot.

2. The model

The #188 no-index model already says: *the working copy is an auto-commit; the shared .git/index is never the mutable state*. Slice d takes it to its end:

  • Base = a moving ref, not a checked-out tree. The logical base is

    origin/master@now, re-resolved per read — never copied to disk as a frozen tree.

  • Reads resolve-on-read. Reading an untouched path returns the blob from the

    current base (git cat-file blob origin/master:<path> semantics, cached). No file ages because nothing is frozen.

  • Edits are deltas (an overlay). A write records a delta in a per-session overlay

    (the #210 autosnapshot ref refs/kvs/wc/<id> is the natural home). The "working tree" a tool sees = base-resolved ⊕ overlay.

  • Commit = block-merge (#206) + CAS (#208) onto current base. Exactly today's

    advanceFinalize, except the parent is re-read at finalize (it already is — the CAS loop re-reads HEAD on miss). Non-overlapping peer edits never conflict; an overlapping path 3-way-merges via the AST/brace driver.

Net: a session can run for hours; an untouched file always reflects the latest origin; an edited file holds the local delta; commit reconciles. No drift, no add/add from a frozen base, no gutting from a stale tree.

3. Mechanism — the design fork

Opt What Transparent to arbitrary tools? Cost Cross-platform
A — FUSE overlay userspace FS: reads hydrate-on-read from the base via the object store, writes land in an upper delta dir Yes (editorsgitgrep see a normal tree) FUSE dep; per-read latency → needs an aggressive blob cache Linux strong; macOS (macFUSE) weak; Windows separate
B — git index virtualization (Scalar/GVFS-style) virtualize .git/index + on-demand object hydration; git itself drives freshness Yes (it is git) heavy; close to a git fork; complex invalidation git-native, but the virtual-FS hook is platform-specific
C — continuous-rebase (slice b on a tight loop) keep a real checkout, kvs wc sync on a sub-second/edit-boundary cadence Yes simplest (extends shipped slice b); still a materialized tree — freshness is bounded by the loop, not structural trivial
D — KVS-mediated FS all file access goes through kvs (a read/edit API or shim); no materialized checkout only for tools that go through kvs KVS-native, no FUSE; breaks any tool that reads the raw FS trivial

4. Recommendation — A (FUSE), staged on Linux first; C as the interim

Long-term (D12 root-cause + transparency): only A and B make staleness structurally impossible while staying transparent to the arbitrary tools an AI session and a human use (editors, git, grep, build tools). Between them, A (FUSE overlay) is the KVS-native fit: KVS already owns the object model (#188) and the overlay ref (#210); a FUSE layer that resolves-on-read from the KVS object store and writes deltas to the overlay is a clean extension, not a git fork (B is heavier and its virtual-FS hook is as platform-bound as FUSE without the simplicity).

Trade-off accepted: FUSE is a real dependency and macOS/Windows need separate backends. Mitigate by staging: the AI fleet runs on Linux (s.khost1, the build hosts, CI) — ship the Linux FUSE backend first, where 100% of the stalenessgutting incidents have occurred. Desktop (macOSWindows) keeps the shipped slice b/c (sync + warn) until a native backend lands — no regression, just not yet structural there.

Interim until the FUSE backend lands: tighten C — have the slice-c hook (and an optional kvs wc watch-driven loop) call kvs wc sync aggressively so the materialized-tree window shrinks toward zero. This is already 90% built (slice b + auto-wiring) and buys most of the benefit while A is designed/implemented.

5. Slices

  1. d0 — object-resolution cache. A fast base:<path> → blob resolver over the

    KVS object store (content-addressed; invalidated when the base ref advances). The shared primitive both A and a tighter C need. Benchmark resolve-on-read latency.

  2. d1 — FUSE read path (Linux). Mount a per-session FS: reads resolve from the d0

    cache at base@now; the overlay ref (#210) supplies edited paths. Read-only proof: an untouched file always reflects the latest origin with no wc sync.

  3. d2 — FUSE write path + commit. Writes land in the overlay; kvs wc commit

    finalizes the overlay via the existing advanceFinalize (#206/#208) onto the re-read base. End-to-end: edit → commit with zero frozen tree.

  4. d3 — fleet rollout. Default the FUSE working copy for fleet/CI sessions

    (Linux); desktop stays on slice b/c. Telemetry: staleness incidents → 0.

6. Non-goals

  • Replacing slice b/c on desktop before a native backend exists (they stay the safe

    default there).

  • A bespoke object store — d0 reuses the KVS/#188 object model, not a new substrate.
  • Refreshing the shared mutable tree (RFC-003 §6 non-goal still holds — virtual

    copies are per-session by construction, which is exactly why this scales to N with no per-session disk: hyperscale D4).

7. Open questions

  • d0 cache: in-process vs a shared local daemon (one cache for all sessions on a host)?

    A host-shared daemon amortizes hydration across sessions (hyperscale) — likely right.

  • Overlay GC: when is a session's refs/kvs/wc/<id> overlay reclaimed? (Tie to

    kvs session end + a TTL sweep.)

  • Interaction with virtual branches (#076): an overlay is a lightweight vbranch —

    unify the two models or keep distinct?


Slice d0 SHIPPED (2026-06-27, /k-go) + §7 open-question resolved

§7 in-process-vs-daemon → resolved: in-process library first (D9 incremental/ reversible). A host-shared resolution daemon (amortizing hydration across sessions on one host) is a real hyperscale win but a later optimization — ship the library now, promote to a daemon when multi-session-same-host amortization is measured-needed.

internal/core/baseresolve.Resolver (in-process): SetBase(commit) (advancing drops the cache — entries are base-specific; same-base is a no-op so a hot path may SetBase(origin@now) freely), OID(path)/Blob(path) (cached path→oid via rev-parse <base>:<path>; oid→bytes is content-addressed/cheap; negatives cached). Concurrency-safe; doesn't poison the cache if the base advances mid-read. Direction- agnostic: the shared primitive both mechanism A (FUSE) and the interim C (tight continuous-rebase) build on. Tests (resolve+cache, drop-on-base-advance, absent→negative) + bench green on dev-linux-kvs: cached hit ~80 ns/op (vs a ~ms git shell-out — the cache earns its keep). Next: d1 (FUSE read path, Linux) over this resolver.

Slice d1a SHIPPED (2026-06-27, /k-go): the read core (split of §5 d1)

§5 d1 (FUSE read path) splits cleanly into d1a — the read core (done here) and d1b — the FUSE wiring (next). d1a is the logic the FUSE nodes call, and is also exactly what mechanism C (tightened continuous-rebase) reads through — so it ships once, dep-free and mount-free, fully unit-tested:

  • internal/core/baseresolve/tree.go: Tree(dir) (mode-aware ls-tree -z) +

    Stat(path) — the directory/stat companion to d0's path→blob, resolve-on-read at base@now.

  • internal/core/vwc/provider.go: the composed overlay ⊕ base read model

    (Stat/Blob/Tree, overlay-first, Tree-union with overlay-override). nil overlay = the d1a read-only proof; the Overlay interface is the seam d2 (write path) fills.

Tests assert the core guarantee — advancing the base reflects the latest content on read with no re-checkout — so staleness is structurally impossible at the read layer (D12 root-cause). Green on dev-linux-kvs.

d1b (next): hanwen/go-fuse/v2 read-only nodes over vwc.Provider + kvs wc mount + real-mount integration test. Feasibility confirmed: dev-linux-kvs carries /dev/fuse + fusermount3 + CAPSYSADMIN.

Slice d1b SHIPPED (2026-06-27, /k-go): the FUSE mount

§5 d1 is complete (d1a read core + d1b FUSE wiring). internal/core/wcfuse (//go:build linux) is a read-only hanwen/go-fuse/v2 filesystem over vwc.ProviderLookup/Getattr/Readdir/Open/Read/Readlink resolve at base@now, git modes mapped to FUSE, zero attr/entry timeout, a refresher that re-fetches + advances the base. CLI: kvs wc mount/unmount. A real-mount integration test on dev-linux-kvs (root + devfuse) proves the §2 guarantee end-to-end: an untouched file read through the mount reflects the latest origin after a base advance, with no re-checkout — PASS. go-fuse is an add-only dep; desktop keeps slice b/c (§4).

Next: d2 (FUSE write path — overlay + advanceFinalize onto the re-read base; the vwc.Overlay seam is ready) → d3 (Linux fleet/CI default).

Slice d2a SHIPPED (2026-06-27, /k-go): the FUSE write path

The write half of §5 d2 landed (d2b = commit-finalize remains). New pkg internal/core/wcoverlay is the writable upper-dir overlay (impl of vwc.Overlay: CopyUpWhiteoutMkdir + StatBlobChildren/Removed). vwc.Provider gained Removed (whiteouts) + BaseStat. internal/core/wcfuse gained read-WRITE nodes (CreateMkdirUnlinkRmdirSetattr/Rename + real-fd write handle, copy-up on first write) and MountRW; kvs wc mount is read-write by default (--ro for read-only). A real r/w FUSE mount test proves edits land in the overlay, the base bytes are never touched, deletes white-out, and readdir composes — PASS on dev-linux-kvs.

Next: d2b — kvs wc commit finalizes the upper delta (added/edited + whiteouts) via advanceFinalize (#206/#208) onto the re-read base, then clears the overlay (durable form = #210 autosnapshot ref). Then d3 (Linux fleet/CI default).

Slice d2b SHIPPED (2026-06-27, /k-go): kvs wc commit finalizes the overlay

§5 d2 is complete. internal/core/wcfinalize.Finalize applies the overlay delta (added/edited blobs + whiteouts, dir-whiteouts expanded) onto the re-read base (temp index → read-tree base → update-index → write-tree → commit-tree -p base): peers' non-overlapping work is carried, only the overlay's paths change (no silent revert), symlinks/exec preserved, ErrNothingToCommit on a no-op tree. wcoverlay gained WalkEntryModeClear. CLI kvs wc commit --overlay <dir> [-m][--onto][--clear] produces the commit and leaves publish to the established CAS push / kvs sync (never advances a ref or touches the shared tree — safe on the concurrent monorepo). Test PASS off-laptop; with the d2a mount test (shared upper-dir contract) the full edit→mount→commit loop is proven.

Next: d2b-tail (auto-publish CAS + auto-clear, so it's one command) → d3 (Linux fleet/CI default + telemetry staleness→0; bundle the kvs release).