Performance baseline registry
Anchor numbers for performance-sensitive components in the Koder Stack. One row per component bench run. Add a new entry whenever a bench moves materially or a new backend lands; do not let ad-hoc CI noise rewrite existing rows.
Each entry must reference the exact bench binary + flags so the result is reproducible. The hardware string identifies the laptop or runner the sample was taken on (uname -a head + CPU model + RAM is sufficient). Numbers from different hardware are not directly comparable; use the same row only if you can re-run on the same machine.
Schema
| Column | Meaning |
|---|---|
date |
YYYY-MM-DD of the run (UTC). |
component |
<area>/<sector> of the binary under test. |
bench |
Bench binary path + relevant flags (sufficient to reproduce). |
dataset |
Workload size (corpus + query count, or whatever applies). |
metric |
p50, p95, p99, throughput, etc. |
value |
Measured number with unit. |
target |
RFC-defined target (or — if untracked). |
pass |
✓ if value meets target, ✗ if it regresses, — if untracked. |
hardware |
One-line hardware tag. |
notes |
Anything noteworthy (recent change, blockers, follow-ups). |
Entries
servicesaimemory — inline embedder + in-process MemoryStore
| date | component | bench | dataset | metric | value | target | pass | hardware | notes |
|---|---|---|---|---|---|---|---|---|---|
| 2026-05-10 | servicesaimemory | make bench-full (go run ./bench --episodes=10000 --queries=1000 --top-k=10 --threshold=0.5) |
10 K episodes / 1 K queries | populate (ep/s) | 70 497 | — | — | koder-laptop (Linux 6.12) | First baseline; #010 partial. Bench reuses recall.Service, exercising the full Search + Put path. |
| 2026-05-10 | servicesaimemory | (same) | (same) | p50 | 5.28 ms | < 50 ms | ✓ | koder-laptop | ~10× headroom at 10 K. Brute-force cosine scales O(N) per query — won't hold at 100 K (hence #007). |
| 2026-05-10 | servicesaimemory | (same) | (same) | p95 | 7.87 ms | — | — | koder-laptop | |
| 2026-05-10 | servicesaimemory | (same) | (same) | p99 | 10.45 ms | < 200 ms | ✓ | koder-laptop | ~20× headroom at 10 K. |
| 2026-05-10 | servicesaimemory | (same) | (same) | throughput | 176.5 q/s | — | — | koder-laptop | Single-goroutine. Multiply by core count for replica capacity ceiling. |
servicesaimemory — kdb-vector + kdb-pgwire (production store, #011)
| date | component | bench | dataset | metric | value | target | pass | hardware | notes |
|---|---|---|---|---|---|---|---|---|---|
| 2026-06-23 | servicesaimemory | bench --store=kdb (inline embedder, RELEASE gateway) |
2 K episodes / 500 queries | populate (ep/s) | 16 | — | — | dev-linux-id → dev-linux-kdb (local bridge, RTT 0.058 ms) | First real kdb-vector baseline (#011). Per-insert graph construction reads/writes sled nodes. |
| 2026-06-23 | servicesaimemory | (same) | (same) | p50 | 109 ms | < 100 ms | ✗ | (same) | Latency-bound by per-request sled node reads — PersistentHnswIndex::search rebuilds VectorCache per call (no cross-request cache); filter over-fetch forces ef=128. Root cause + fix: kdb#808 (+ #807). Network ruled out (RTT 0.058 ms). |
| 2026-06-23 | servicesaimemory | (same) | (same) | p95 | 128 ms | — | — | (same) | |
| 2026-06-23 | servicesaimemory | (same) | (same) | p99 | 137 ms | < 500 ms | ✓ | (same) | Tight distribution (~fixed per-call sled-read cost), not a tail. |
| 2026-06-23 | servicesaimemory | (same) | (same) | throughput | 9.7 q/s | — | — | (same) | Single-goroutine. 100K acceptance run gated on kdb#808 (cache) — a 100K populate at 16 ep/s ≈ 1h and would only confirm the same per-node-read cost. |
| 2026-06-24 | servicesaimemory | bench --store=kdb (inline embedder, RELEASE gateway + kdb#808 warm-handle registry) |
2 K episodes / 500 queries | p50 | 105.7 ms | < 100 ms | ✗ | dev-linux-id → dev-linux-kdb (SQL→real PG17 loopback; vector→gateway) | kdb#808 Slice 1 (warm adjacency cache across requests): only ≈3 % (109→105.7 ms). Decisive: the adjacency cache was NOT the dominant cost — at filtered ef=128, ~256 per-node read_vector+is_tombstoned sled reads/query survive (no cross-request vector/meta cache). Real levers: #808 Slice 2 (vector+meta cache) + #807 (filtered-HNSW drops ef). SQL backend is PG17, not kdb-pgwire: the ANY($1::text[]) hydrate can't encode on kdb pgwire (kdb#809). |
| 2026-06-24 | servicesaimemory | (same) | (same) | p95 / p99 / mean | 122.6 / 130.1 / 102.5 ms | — | — | (same) | vs #011 baseline 128 / 137 / 103.4 ms — uniformly ~3 % better (warm adjacency), confirming the bottleneck is elsewhere (per-node data/meta reads + ef over-fetch). |
| 2026-06-24 | servicesaimemory | bench --store=kdb (inline embedder, RELEASE gateway + kdb#808 Slice 2: vector + tombstone-meta cross-request cache) |
2 K episodes / 500 queries | p50 | 6.7 ms | < 100 ms | ✓ | dev-linux-id → dev-linux-kdb (SQL→real PG17 loopback; vector→gateway) | kdb#808 Slice 2 — target CRUSHED, ~16× vs baseline (109→6.7 ms). Caching the per-node read_vector + is_tombstoned reads across requests (enabled by Slice 1's warm handle) removed the ~256 surviving sled reads/query. Confirms Slice 1's diagnosis: vector + meta reads, not adjacency, were the cost. Generalised the #059 slab-LRU into ByteLru<K,V> (3 caches, one machine). |
| 2026-06-24 | servicesaimemory | (same) | (same) | p95 / p99 / mean | 9.9 / 20.6 / 7.5 ms | < 500 ms (p99) | ✓ | (same) | vs Slice 1 122.6 / 130.1 / 102.5 ms. |
| 2026-06-24 | servicesaimemory | (same) | (same) | populate / throughput | 73 eps / 133.8 qs | — | — | (same) | populate 15→73 eps (~5×; insert also reads neighbour vectorsmeta, now cached). At 100K scale insert slows to 28 ep/s (graph construction visits more nodes — #807 lever). |
| 2026-06-24 | servicesaimemory | bench --store=kdb (inline embedder, RELEASE gateway, kdb#808 Slice 2) |
100 K episodes / 2000 queries | p50 | 19.6 ms | < 100 ms | ✓ | dev-linux-id → dev-linux-kdb (SQL→real PG17; vector→gateway) | kdb#808 ACCEPTANCE @100K MET (5× under). Latency 6.7→19.6 ms from 2K→100K (deeper graph visits more nodes; some misses at the 256 MB cap) but well under target. Cache cap held the working set. Closes kdb#808 + memory#011. |
| 2026-06-24 | servicesaimemory | (same) | (same @100K) | p95 / p99 | 36.3 / 45.0 ms | < 500 ms (p99) | ✓ | (same) | p99 11× under target. throughput 47.6 qs; populate 28 eps (58 min). |
engineslangkoda — kasm assembler (ET_REL mode)
| date | component | bench | dataset | metric | value | target | pass | hardware | notes |
|---|---|---|---|---|---|---|---|---|---|
| 2026-05-16 | engineslangkoda | ./kasm -c /tmp/koder_stage1_src.asm /tmp/kasm_stage1.o |
koder stage1 bundle, 809 883 lines (~30 MB) | wall (line 1–170 K) | 5.8 s | ≤ 7× nasm -O0 |
— | koder-laptop (Linux 6.12) | Pre-cascade baseline. Steady chunk_ms 85–137 ms up to line 140 K then super-linear cliff (243→960→3176→>100 000 ms). Tracked by engines/lang/koda#764. |
| 2026-05-16 | engineslangkoda | (same) | (same) | wall (full bundle) | DNF (≥1800 s) | ≤ 90 s | ✗ | koder-laptop | Pre-cascade baseline. Killed by SIGKILL after >100 s wall on the chunk past line 170 000. Was blocking #748 criterion 3. |
| 2026-05-16 | engineslangkoda | nasm -O0 -felf64 /tmp/koder_stage1_src.asm -o /tmp/nasm.o (reference) |
(same) | wall (full bundle) | ~13 s | — | — | koder-laptop | External reference assembler. kasm's target envelope (≤7× nasm) anchors to this number. |
| 2026-05-20 | engineslangkoda | (same) | koder stage1 bundle, 817 K lines (post-cascade build) | wall (full bundle) | 47 s | ≤ 90 s | ✓ | s.khost1.dev-linux-koda (Linux 6.12) | Post-cascade PASS — RFC-019 cascade (#770#773#774#771) + label intern (#765). 90 s target met by 43 s margin. #748 criterion 3 PASS. Cliff eliminated; rerun via VM per `policiestest-host-isolation.kmd`. |
| 2026-05-24 | engineslangkoda | make build-safe × 3 generations |
full koder + smoke roundtrip | wall × 3 (gen-123) | 11m51s / 12m16s / 11m51s | bootstrap stable | ✓ | s.khost1.dev-linux-koda | #772 validation gate PASS — gen-2 =gen-3 md5 87d99225… byte-identical (fixed point) and matches historical 2026-05-20 phase-4a canonical. RFC-019 cascade FULLY CLOSED (#750/#770/#771/#772/#773/#774). |
| 2026-05-24 | engineslangkoda | make test (full regression suite) |
1747-line full output | wall + PASS/FAIL count | 35m25s · 260 P / 50 F-files (127 F-lines) | suite green | ✗ (baseline) | s.khost1.dev-linux-koda | Pre-Phase-1 baseline — predates apt install python3 + chmod fixes. Captured pre-#784. |
| 2026-05-24 | engineslangkoda | make test (post Phase-1 env setup) |
1808-line full output | wall + PASS/FAIL count | 36m50s · 277 P / 33 F-files (108 F-lines) | suite green | ✗ (Phase 1 done) | s.khost1.dev-linux-koda | #784 Phase 1 PASS — 17 FAIL files cleared by python3 install + chmod. Remaining 33 FAILs scheduled for Phase 2 (Cat. B binaries) + Phase 3 (Cat. C substantive triage). |
| 2026-05-24 | engineslangkoda | xvfb-run -a make test (post Wave 2a+2b) |
823-line full output | wall + PASS/FAIL count | ~58min · 278 P / 33 F-files (107 F-lines) | suite green | ✗ (Phase 2 partial) | s.khost1.dev-linux-koda | #784 Wave 2a+2b SHIPPED — bundling shim in scripts/test-native.sh (resolves require for native tests) + koder_ui_server PATH symlink + apt install xvfb + libgtk-4-1. Native test_ui_hello now PASS. NUI/X11 tests start under xvfb but don't emit PASS (interactive GUI by design — Cat. C redesign work). |
| 2026-05-28 | engineslangkoda | ./kasm -c /tmp/koda_stage1_src.asm /tmp/stage1.o (post-#783#792#787/#776 closeout) |
koder stage1 bundle, 832 K lines (v7.28.1 source) | wall (full bundle) × 3 | 7.2 s ±5 % (.o md5 byte-identical) | ≤ 90 s | ✓ | s.khost1.dev-linux-koda | 6.5× faster than 2026-05-20 (47 s). Cascade: v7.27.1 .keys.sort (#787) + v7.27.2-4 perf (#797#796#709) + v7.28.0 #711 + v7.28.1 #799 + inhibit-counter dc5ac430fb (#760 kasm-side). 240K-300K range now linear (1.87→1.99→2.13→2.62 s). 280K spike (was 81 s) is GONE — closes #776 acceptance #2/#3. |
| 2026-05-28 | engineslangkoda | (same) | kasm bundle, 199 K lines | wall × 3 | 1.10 s (.o md5 f641deec… byte-identical) |
— | — | s.khost1.dev-linux-koda | Closes #783 (kasm-self SEGV ≥197K). Anchored by tests/regression/215-kasm-self-scale-783-792.test.sh. |
| 2026-05-28 | engineslangkoda | scripts/bench-hash.sh 5000000 (MODE=inline) |
5 M decimal-string keys, in-loop construction | insert ops/s @ 5 M | 1.72 M (v7.28.1) → 3.73 M (v7.28.5) = 2.2× | rust ≥ 10 M | ✗ (closing) | s.khost1.dev-linux-koda (Xeon 8176) | Naive consumer baseline. Inline key construction dominates: ~55-64 % of lookup wall is "k"+i.to_s allocation per iteration. |
| 2026-05-28 | engineslangkoda | (same, MODE=prealloc) | 5 M pre-built keys | insert ops/s @ 5 M | 1.65 M (v7.28.1) → 6.37 M (v7.28.5) = 3.86× | rust ≥ 10 M | ✗ (closing) | s.khost1.dev-linux-koda | Isolates Hash internals. 75 % → 50 % load factor (v7.28.5) is the dominant lever — fewer probes per insert outweighs the one extra doubling. |
| 2026-05-28 | engineslangkoda | (same, MODE=hint — Hash.new(N)) |
5 M pre-built keys, pre-sized table | insert ops/s @ 5 M | 8.56 M (v7.28.4) ≈ 8.71 M (v7.28.5) | rust ≥ 10 M | ~ (1.2-3.5× from rust) | s.khost1.dev-linux-koda | Hash.new(N) collapses the doubling cascade (#729 task 2 step 1). Hint mode is the realistic kodec workload (kodec knows frame counts ahead). Step 2 (load factor) is structurally inert for pre-sized tables (1.0× ratio confirmed). |
| 2026-05-28 | engineslangkoda | (same, MODE=hint) | 5 M lookups | lookup ops/s @ 5 M | 17.9 M (v7.28.5) | rust ≥ 50 M | ✗ (closing) | s.khost1.dev-linux-koda | Best-case lookup throughput. Remaining gap to std::HashMap reference is the tuning queue's next 4 items (barrier short-circuit / SwissTable / SipHash / Robin Hood). djb2 string hash + linear probe still the bottleneck per ticket #729 audit. |
| 2026-05-28 | engineslangkoda | scripts/bench-gc-pause.sh 10000000 (v7.28.9, default heap) |
10 M small 4-int objects, ~10 % retained | minor/major collections + max pause | 0 minor / 0 major / 0 ns max pause (937 ms wall) | RSS ∝ surviving 10 % (#730 task 3) | ✗ (defect found) | s.khost1.dev-linux-koda (Xeon 8176) | #730 task 2 baseline reveals a defect, not a tuning knob: _rt_array_alloc wraps its 2 allocs in _gc_inhibit, which routes every array to OLD gen and skips the minor-GC trigger → transient arrays are NEVER collected; RSS grows with total allocation, not survivors. Reproduces even at 2 MB young / 256 MB old. Fix needs a defer-trigger / pin-during-construction RFC (removing the inhibit naively reintroduces the half-built-array GC walk). gc_max_pause_ns is vacuous for this workload until the GC actually fires. |
| 2026-05-31 | engineslangkoda | re-arm on the TRANSIENT bench — re-arm gen2 (58fb8e4e, no card table) vs #800 (478d7556) |
N transient 4-int arrays, ~10% retained | wall + minor/major + Max RSS | 5M: 1779ms (1.57× slower) minor 1→15, major 3→0; 20M: 33913ms (2.35× SLOWER) minor 1→183, major 15→11; RSS@10M 948 MB (vs ~2 GB), minor 183 | re-arm faster + RSS∝survivors on transient | ✗ (re-arm reverted) | s.khost1.dev-linux-koda (Xeon 8176) | NEGATIVE — falsifies "re-arm alone suffices for transient". The re-arm fires (minor grows with N) but is 2.35× SLOWER at 20M: each of the 183 minor GCs pays .gc_minor_clear_old's O(old) walk REGARDLESS of old→young edge count (the cost is the full old-gen mark-clear, not the edges). So the card table (young-only mark → delete clear_old) is required even for the transient workload, not just self-compile. Also RSS only ~halved (948 MB, not ∝survivors ~80 MB): the old gen never shrinks (no compaction/munmap) → stays at high-water → RSS∝survivors additionally needs old-gen management (RFC-020 territory), beyond the card table + re-arm. #730 is a multi-piece GC effort: card table + re-arm + old-gen shrink. |
| 2026-05-31 | engineslangkoda | #800 GC attribution — bench-gc-pause.sh on the #800 binary (478d7556), minor vs major |
N transient 4-int arrays, ~10% retained | minor/major count + majortotalns vs wall | 1M: minor 1major 0; 5M: minor 1major 3 (615ms54% wall); *0M: minor 1/major 15 (12.4s86% of 14.4s wall)* | identify the dominant GC cost for #730/#807 | n/a (diagnostic) | s.khost1.dev-linux-koda (Xeon 8176) | Decisive: the #800 cost is MAJOR-GC-dominated; minor fires exactly ONCE (latches) at every N. Mechanism: minor latches after the first collection → transients spill to old gen → old gen fills → major GC thrashes (86% of wall at 20M). Reframes #807: the minor-GC card table delivers ~zero on the current workload (minor=1), BUT is the prerequisite for the #730 re-arm, which re-collects young so transients DON'T spill → old stays small → major thrash collapses (the 2 GB→149 MB RSS drop already shows the reduced spill). So #807 success criterion is the transient bench (major_count↓ + RSS∝survivors + wall↓ with re-arm + card table), NOT self-compile time (which has legitimate old→young edges). |
| 2026-05-31 | engineslangkoda | #800 re-land self-compile cost — A/B d4c0582c (master, no card table) vs 916e752877 (#807 increment-1) on dev-linux-koda |
gen2 self-compiling the 817 K-line stage1 | wall s/gen, interleaved 2× | BASE (no card) 267s / 340s; CARD (#807 inc-1) 260s / 410s — equal within heavy host noise | self-compile ≤1.2× pre-#800 (~11s) | ✗ (#807 inc-2 target) | s.khost1.dev-linux-koda (Xeon 8176, ~8 parallel sessions) | Two results. (1) #807 increment-1 (card-table BSS + barrier dirties card, behavior-neutral) adds NO measurable self-compile cost (CARD≈BASE; CARD1<BASE1) — the barrier is 9 constant instructions/store. (2) Current master (post-#800-re-land) self-compiles in ~260–340s vs ~11s pre-#800 = ~25×: the cost of enabling young-gen collection (the O(old) .gc_minor_clear_old + GC firing during self-compile). This is the documented Option-3 interim cost (RFC-021); #807 increment-2 (dirty-card scan replacing the O(old) walk) is what removes it. CI gate #794 self-compiles 2× so its wall ~doubled to ~9 min — tolerable interim. |
| 2026-05-31 | engineslangkoda | minor-GC re-arm (#730 task-2, held patch) — self-compile bootstrap on master f469768fa1 + overlay |
817 K-line stage1 self-compile (gen2→gen3); 10 M-array transient storm for RSS | self-compile wall + Max RSS | ~47× SLOWER (gen1→gen2 ~10 s; gen2-with-re-arm→gen3 >8 min @ 100 % CPU, killed); RSS 149 MB (vs ~2 GB baseline) | self-compile ≤1.2× baseline + RSS ∝ survivors | ✗ (re-arm reverted) | s.khost1.dev-linux-koda (Xeon 8176) | NEGATIVE result → #807. The re-arm achieves RSS∝survivors but each of its thousands of minor GCs pays RFC-021 Option-3's full O(old-gen) old→young root scan → O(minor×old) blowup. Confirms the card table (Option 2, #807) is the throughput prerequisite for #730's re-arm AND #801's de-inhibits, not just a pause optimization. Re-arm patch held for re-application on top of #807. |
| 2026-06-02 | engineslangkoda | re-arm WITH #807 2a (young-only mark, clear_old deleted) + list-compaction — bench-gc-pause.sh re-arm gen1 (684e82a9) vs master gen4 (478d7556) |
N transient 4-int arrays, ~10% retained | wall + minor/major + max_pause + max RSS (time -v) |
5M: re-arm 1799 ms / minor 15 / major 0 / pause 0 / RSS 140 MB vs master 1329 ms (1.35×) / minor 1 / major 3 / pause 311 ms / RSS 926 MB; 100K/1M identical | re-arm affordable + RSS∝survivors | ◐ (partial — card O(dirty) still needed) | s.khost1.dev-linux-koda (Xeon 8176) | Diagnostic, NOT a commit. Re-arm is correct on the bench (rc0, exact retained) and at 5M reduces RSS 6.6× + eliminates major-GC pause (0 vs 311 ms) at 1.35× wall. BUT: (1) the *.6× is N5M-specific — row 60 (pre-2a) measured RSS 948 MB @10M (old gen never shrinks → high-water; full ∝survivors needs old-gen shrink, RFC-020); only benched to 5M with 2a. (2) The self-compile gen3 TIMED OUT >900 s (vs 396 s for 2a alone): on the large-remembered-list self-compile the list-compaction's O(persistent-live-edge list) per-minor scan is the new cliff (distinct from row 63's deleted clear_old O(old) cost). Two independent confirmations that the per-minor old→young discovery must be O(dirty cards)*→ the #807 card-table scan is the required next increment; gentle-trigger REFUTED (cost is per-minor scan, not fire frequency). Candidate /root/cand807-rearm md5 f2de9e61. |
| 2026-05-29 | engineslangkoda | FFI drop-result fast path (#732 step 3, v7.28.13) — scripts/bench-ffi.sh 100000000, 3 trials/binary |
100M ffi_call0(nop) at statement position (result discarded; kodec #227 nop() class) |
wall ms (DROP loop) | 266 ms (266266267) vs v7.28.12 ~274 ms (267272282) = ~3 % / 0.08 ns/call | wall ≤ pre-#732 | ✓ (no slower; marginal) | s.khost1.dev-linux-koda (Xeon 8176) | Measured — corrects the initial −19 ms/10M op-count estimate. Elides −4 opscall (shlorpushpop) but FFI is indirect-call-latency-bound, so the win is only ~3 % + a more consistent loop (no slow tail), not the op-count arithmetic. USED path (return propagated) is byte-identical across versions; its ±6 % swing is code-layout noise > the drop-path win. Structurally confirmed (drop site has no shlorpush) + self-host fixed point 233cc326… + regression 225. Takeaway: trampoline-op cuts (steps b/c) are wall-marginal for nop class; the real kodec FFI lever is call frequency. |
| 2026-05-29 | engineslangkoda | (same harness, USED regime) | 100M acc += ffi_call1(nopret, 1) (return propagated into a KL int) |
wall ms (USED loop) | 430 ms ≈ 4.4 ns/call (9 cy @ 2.1 GHz) | rust extern "C" ≈ 0 trampoline |
✗ (gap real) | s.khost1.dev-linux-koda | Koda-side half of kodec #227 acceptance #2. The USED path keeps the full untagsave-rspaligncallrestoretag trampoline — this is the cost kodec #227 actually measures (it propagates the return). Closing #227 needs the Rust S10 reference (benchmark_tdd.rs scenario 10 is still a `/ TODO` stub) + the kodec comparator wiring — not blocked on lang#732 anymore. |
enginescompresskzip — pluggable compressor backends at 1 GiB scale
Production-readiness milestone (v0.2.0, Passo 4 of the roadmap). Validates whether the LZMA2 / Brotli backends scale predictably past the 100 MiB workloads captured at engines/compress/kzip/bench/baseline.md. Full per-tool numbers (incl. FOSS reference comparators at 100 MiB) live in the module's bench file; this section anchors only the kzip-specific 1 GiB rows.
| date | component | bench | dataset | metric | value | target | pass | hardware | notes |
|---|---|---|---|---|---|---|---|---|---|
| 2026-05-19 | enginescompresskzip | bash bench/bench.sh --workload random-1g --tools kzip-baseline,kzip-lzma2,kzip-brotli |
1024 MiB /dev/urandom |
compress (s, zstd:3) | 8.548 | — | — | koder-laptop (Linux 6.12) | kzip-baseline. Linear scale-up from 100 MiB (1.88s → 8.55s ≈ 4.5×). RSS 552 MiB. |
| 2026-05-19 | enginescompresskzip | (same) | (same) | compress (s, lzma2:6) | 39.650 | — | — | koder-laptop | ~4.6× slower than zstd, RSS 2.2 GiB. Predictable LZMA cost; no ratio gain on /dev/urandom. |
| 2026-05-19 | enginescompresskzip | (same) | (same) | compress (s, brotli:6) | 9.983 | — | — | koder-laptop | ~1.2× zstd time, RSS 733 MiB. No ratio gain on incompressible data; speed/RAM matches zstd. |
| 2026-05-19 | enginescompresskzip | bash bench/bench.sh --workload text-1g --tools kzip-baseline,kzip-lzma2,kzip-brotli |
1024 MiB repeating text | output size (MiB) | 0.05 (zstd) | — | — | koder-laptop | kzip-baseline output 0.05 MiB on uniform repeating-seed text. zstd wins at this scale on synthetic-uniform data. |
| 2026-05-19 | enginescompresskzip | (same) | (same) | output size (MiB) | 0.07 (lzma2) | — | ✗ | koder-laptop | Counter-RFC-001: lzma2 produces MORE bytes than zstd on this corpus. RFC's "20-30% better than zstd" claim NOT validated. |
| 2026-05-19 | enginescompresskzip | (same) | (same) | output size (MiB) | 0.01 (brotli) | — | — | koder-laptop | Brotli wins at 1 GiB on repeating text: 5× smaller than zstd, 7× smaller than lzma2. Tail-serve advantage compounds. |
Open follow-up (kzip): rerun on a real-world mixed-vocabulary corpus (wikitext, linux-source dump) before claiming LZMA2 ratio in operator-facing docs. The synthetic text-1g workload is too uniform — every backend converges to >99.99% reduction; the differences are noise-level (~0.06 MiB on a 1 GiB input).
Resolved follow-up (2026-05-19): bench rerun on the kzip source tree (28 MiB real-world mixed Go+docs+tests corpus). Result: lzma2:6 = 23.29 MiB out (0.7% smaller than zstd:3 = 23.46 MiB) at 2.4× compress time + 3.8× RSS. Brotli:6 matches lzma2's ratio at 1.08× zstd's time. RFC-001 "20-30% better" LZMA2 claim falsified on real corpora at this scale. Default zstd dominates the speedratio Pareto for typical kzip workloads (<1 GiB mixed corpora). Operator guidance update: reserve --compressor=lzma2 for multi-GB cold-archival workloads with workload-specific AB test confirmation; docs/usage/compression.kmd to be updated accordingly (sub-follow-up).
| date | component | bench | dataset | metric | value | target | pass | hardware | notes |
|---|---|---|---|---|---|---|---|---|---|
| 2026-05-19 | enginescompresskzip | bash bench/bench.sh --workload source --tools kzip-baseline,kzip-lzma2,kzip-brotli |
kzip source tree (28.08 MiB mixed) | ratio (lzma2 vs zstd) | −0.7% | −20 to −30% per RFC-001 | ✗ | koder-laptop (Linux 6.12) | LZMA2 falsified on real corpus. Output 23.29 vs 23.46 MiB. Compress time 2.4× zstd, RSS 3.8× zstd. |
| 2026-05-19 | enginescompresskzip | (same) | (same) | ratio (brotli vs zstd) | −0.9% | — | — | koder-laptop | Brotli matches lzma2's ratio at 1.08× zstd's time. Better Pareto point than lzma2 on this corpus. |
Pending baselines
services/ai/memory— kdb-vector @ 100 K meeting the RFC target (p50 <100 ms, p99 < 500 ms). The bench harness ships (
#011,make bench-100k,--store=kdb);#007(store swap) +#008(real embedder) are done. The 100 K target run is now blocked onkdb#808— kdb-vector recall is latency-bound by per-request sled node reads (no cross-request cache: 109 ms p50 already at 2 K). Re-runmake bench-100konce #808 lands.services/ai/gateway— round-trip overhead for cached vs uncachedproxy (RFC-001 follow-up; bench harness not yet written).
services/ai/cache— exact-match lookup p99 vs raw upstream (blockedon
#002skeleton).engines/lang/koda— kasm residual mid-range timing varianceCLOSED 2026-05-28 (
#776→ done). Cascade since 2026-05-20 brought full 832 K from 47 s → 7.2 s and the 280 K spike from 81 s → 2.13 s. See the 2026-05-28 rows above.
infranetbox — kbox container runtime cold-start
Tracked per meta/docs/stack/registries/self-hosted-pairs.md row 13: performance:cold_start_benchmark is the open gate before kbox flips from stable → official versus DockerPodman. The bench below is a placeholder row — the actual run + numbers land alongside the cold-start benchmark harness (kbox `testsperfcold-start), opened as a follow-up ticket when an isolated benchmark VM (s.khost1) is ready per policies/test-host-isolation.kmd`.
| date | component | bench | dataset | metric | value | target | pass | hardware | notes |
|---|---|---|---|---|---|---|---|---|---|
| 2026-05-20 | infranetbox | kbox run --rm alpine:3.18 /bin/true × N (cold rootfs each) |
alpine:3.18, 1 layer, ~3.4 MiB | p50 wall | — | ≤ podman run p50 × 1.1 |
— | s.khost1.dev-linux-koda | Placeholder. Bench harness opening as follow-up. Cliff: until this row populates, cold_start_benchmark gate of pair #13 stays open. |
| 2026-05-20 | infranetbox | (same) | (same) | p99 wall | — | ≤ podman run p99 × 1.2 |
— | (same) | Same row as above; populated together. |
productsdevflow — public API latency (single-client home LAN baseline)
First Flow baseline. Captures TTFB + total-response timings against flow.koder.dev from the developer laptop. Numbers anchor public client-perceived latency for the Forgejo-fork (HTTP handlers + PAT auth + repo index). Server CPU time is not isolated here — the sample includes ISP RTT + TLS + EVEO edge. Re-run via products/dev/flow/engine/perf-baseline/bench.sh; full snapshot at products/dev/flow/engine/perf-baseline/2026-05-24.md.
| date | component | bench | dataset | metric | value | target | pass | hardware | notes |
|---|---|---|---|---|---|---|---|---|---|
| 2026-05-24 | productsdevflow | engine/perf-baseline/bench.sh (ITER20, WARMUP3) |
5 scenarios × 20 iter | home p50 TTFB | 361.856 ms | — | — | koder-laptop (Linux 6.12 + home Wi-Fi + BR ISP → flow.koder.dev) | FLOW-116 first baseline; LANDING_PAGE = login 303 redirect. Includes ISP RTT + TLS. |
| 2026-05-24 | productsdevflow | (same) | (same) | user_login p50 TTFB | 382.839 ms | — | — | (same) | Custom signin.tmpl (RFC-006 §1; FLOW-115 locks). Auto-redirect-to-Koder-ID hint paints inside p50. |
| 2026-05-24 | productsdevflow | (same) | (same) | api_version p50 TTFB | 1613.977 ms | — | — | (same) | Surprisingly slow for a fixed JSON literal. PAT validation + TLS dominate; warrants server-side OTel breakdown (FLOW-103/106). |
| 2026-05-24 | productsdevflow | (same) | (same) | apirepossearch p50 total | 1743.073 ms | — | — | (same) | /api/v1/repos/search?limit=5 against the current corpus. p95 → p99 widens (2755 ms p99) — index cache misses + auth round-trip. |
| 2026-05-24 | productsdevflow | (same) | (same) | apirepokoder p50 total | 248.528 ms | — | — | (same) | /api/v1/repos/Koder/koder hot-cache hit. p99 1011 ms reflects occasional misses. Fastest p50 in the suite. |
| 2026-05-25 | productsdevflow | go test -run TestOAuthT2_KoderIDStub_CallbackWire (in-process, pkg/oauthstub, no network) |
1 test, single-isolation | oauthcallbackfull_dance wall | ~200 ms (FLOW-118 obs.) | < 500 ms (flag) | — | test-host VM (s.khost1; per heavy-work-isolation) | FLOW-144. Full /authorize → callback → /token dance against the canonical OAuth stub — the operation that gates real sign-in perf, NOT measured by the user_login TTFB row (page only). Value from FLOW-118 wire-suite observation; informational, not gating until re-captured on an idle test-host. Drift past 500 ms ⇒ goth/oauth2 client OR Flow callback-handler regression. |
| 2026-05-25 | productsdevflow | go test -run TestOAuthT8_KoderIDStub_RefreshRotation (in-process, pkg/oauthstub, no network) |
1 test, single-isolation | oauthrefreshtokenroundtrip wall | ~270 ms (FLOW-118 obs.) | < 600 ms (flag) | — | test-host VM (s.khost1; per heavy-work-isolation) | FLOW-144. Refresh-token rotation round-trip (T8). Value from FLOW-118 wire-suite observation; informational, not gating until re-captured on an idle test-host. Promote to a gate after OTel server-side split (FLOW-103106142). |
| 2026-05-24 | tools/design-gen | og.BuildPerPage (exec.Command per page, #106 NumCPU pool) |
1755 pages × NumCPU=16 | per-page OG wall | ~55 s | — | — | s.khost1 fonts-smoke Incus runner | Cold-start dominated. Each page spawns a Chrome process (~500ms init). Chromedp rewrite (#118) attempted but reverted — serializes through single DevTools connection; filed as pending for focused investigation. |
| 2026-05-24 | tools/design-gen | og.BuildPerPage (#118 chromedp, per-worker browser + fresh tab) |
30 pages × 1 browser, serial | per-page render (warm) | ~90 ms/page | — | — | koder-laptop (sandbox; NOT representative for parallel) | #118 deadlock FIXED (per-worker independent Chrome + fresh tab + CDP transparency override; dropped the --default-background-color flag that hangs DOM.enable on Chrome 148+). Serial warm per-page validated ~90ms (vs ~500ms cold-start). PARALLEL throughput NOT cleanly measurable on this laptop AT THE TIME: 8 browsers → 489 mspage wall (worse than serial). Root cause is measurement NOISE, not hardwareChrome: the box (i7-13700HX 24T, 62 GB RAM, 32 GB devshm @ 1%, no cgroup caps) is capable, but was running concurrent heavy jobs from other sessions (nasm@99%, koderkruze@54%, multiple claudejavagit; loadavg 7-11, swap full). Adding 8 Chrome processes on a saturated machine → contention. Full-corpus 15s gate (`KDSOGPERFTEST=1`) must run on an IDLE dedicated runner; pending — see next row. Auto worker default capped at 4 (was NumCPU → 24 browsers thrashed). |
| 2026-05-24 | tools/design-gen | og.BuildPerPage (#118 chromedp) OG_CACHE_BUST=1 real renders |
400 pages (200×2 locales) | full-corpus wall | 47 s @ 4 workers · 39 s @ 12 workers | < 15 s (#118 gate) | ✗ | s.khost1 dev container (32 cores, loadavg 10→28 during run) |
DECISIVE finding: more workers do NOT improve throughput — 4 workers = 117 mspage, 12 workers = 98 mspage, both ~serial. The 15s gate (premised on ~16× parallelism @ 50 ms/page) is NOT meetable on a SHARED host: parallel headless Chrome is process-heavy, so N browsers saturate an already-loaded box (load hit 28 with 12) without throughput gain. Per-page floor is ~90 ms (fresh-tab chromedp), above the gate's optimistic 50 ms premise. Gate is blocked on a genuinely IDLE dedicated CI runner, NOT on the code. Deadlock fix itself is correct + validated. cap=4 confirmed right (extra workers only saturate). |
| 2026-05-27 | tools/design-gen | og.BuildPerPage (#118 chromedp, browser reuse fixed) |
80 pages (40×2 locales), OG_CACHE_BUST=1, 2 workers |
Chrome launches · per-page wall | 3 launches (2 workers + probe) · 59.6 ms/page · 4.77 s | ≤ workers+probe launches (structural gate) | ✓ | koder-laptop (host-independent gate — no idle runner needed) | ROOT-CAUSE CORRECTION of rows 122/123. The #118 "per-worker browser reuse" was NOT actually in effect: each worker created browserCtx but never Run it, so chromedp lazily allocated a fresh Chrome process per page on the first Run of each per-page tab — process-per-page cold start (500 mspage) was still happening (measured 81 launches / 574 mspage for 80 pages). The "90 ms warm" of row 122 was never the real make gen path. Fix: boot each worker's browser ONCE via a trivial Navigate (a no-action force-start hangs the CDP handshake on Chrome 148+) directly on browserCtx (not a WithTimeout child — ExecAllocator ties process lifetime to the Run ctx). Result: 81→3 launches, 574→60 ms/page, 45.9→4.77 s for 80 pages (10× faster). The 15 s wall-clock gate is retired — unmeetable on a shared host AND unable to separate the cold-start regression (500 mspage) from contention noise (98→489 mspage observed). Replaced by a structural spawn-count gate (tests/regression/og_perf_test.go::TestRegressionOGPerPageBrowserReuse, make test-og): asserts Chrome is launched once per worker (+probe), not per page — host-independent (a counting shim execs the real binary by absolute path so Chrome's child re-execs bypass it), catches the regression on ANY Chrome-capable host including the laptop. |
| 2026-05-25 | productshorizontalkruze | kbench (testsbenchmarkcomparative, repeats=5 median) |
canvas2d-fps fixture (3000 shapes, 8s) | render (fps, higher=better) | Kruze 28.4 / Chrome 42.3 | — | — | s.khost1.dev-linux-kruze VM (4 vCPU; 112-core host @ load ~10; software-GL headless/Xvfb) | KRUZE-181 first comparative. Kruze's web engine IS Chromium (CEF) in OSR mode, so "Kruze vs Chrome" measures the Flutter+CEF-OSR shell overhead, not the engine. Paint-bound canvas: Kruze −33% FPS — the OSR composite path cost. PRELIMINARY (software-GL + shared host); not a perf gate. Built on Flutter 3.44.0 (KRUZE-182 unblocked). |
| 2026-05-25 | productshorizontalkruze | kbench (repeats=5 median) |
svg-animation fixture (2000 nodes, 8s) | render (fps, higher=better) | Kruze 35.7 / Chrome 41.9 | — | — | s.khost1.dev-linux-kruze VM (software-GL headless) | Paint-bound SVG: Kruze −15% FPS. Less than canvas — SVG mixes retained-mode DOM (engine) with paint, so the OSR penalty is diluted. PRELIMINARY. |
| 2026-05-25 | productshorizontalkruze | kbench (repeats=5 median) |
dom-churn fixture (300 rows, 8s) | throughput (ops/s, higher=better) | Kruze 75.8 / Chrome 69.7 | — | — | s.khost1.dev-linux-kruze VM (software-GL headless) | Thesis nuance confirmed: DOM churn is main-thread engine/layout work (NOT paint), so the OSR overhead doesn't bite — Kruze ≈ Chrome (even nudged ahead, within noise). The shell cost is on compositing, not the shared Chromium engine. PRELIMINARY. |
| 2026-05-25 | productshorizontalkruze | kbench (repeats=5 median) |
process-tree PSS across 4 fixtures | memory (PSS MB, lower=better) | Kruze ~30-35% higher (static 542408 · canvas 619476 · svg 613478 · dom 647504) | — | — | s.khost1.dev-linux-kruze VM | Most robust delta: Kruze's Flutter engine + CEF-OSR texturecompositor adds a consistent ~+30-35% PSS over native Chrome on every page, idle or busy. PRELIMINARY. Canonical run pending: chmod GLES (hardware GL), repeats≥5 on an idle-quiesced host, + SpeedometerMotionMark/JetStream (mirror.sh refs + selectors need fixing). |
| 2026-05-28 | productshorizontalkruze | kbench (repeats=3 median) |
Speedometer 3 (standard harness, mirrored) | responsiveness (runs/min, higher=better) | Kruze 28.80 / Chrome 42.30 | — | — | s.khost1.dev-linux-kruze VM (4 vCPU; software-GL headless) | First standard harness. Speedometer 3 measures real-world SPA responsiveness (TodoMVC, ReactVueAngular). Kruze −32% vs Chrome — consistent with the fixture-based canvasSVG deltas, confirming the OSR shell overhead hits all workloads, not just paint. Memory: Kruze 2338 MB / Chrome 1446 MB (+62%). CPU: Kruze 42.1s / Chrome 29.5s (+43%). FCP ≈par (93100ms). JetStream 3 + MotionMark timeout on this VM (software rendering too slow); deferred to hardware-GL run. |
| 2026-06-02 | productshorizontalkruze | kbench (repeats5 median, `KBENCH_ANGLEgl-egl`) |
canvas2d-fps + dom-churn + static fixtures | Chrome hardware-GL reference (T4) | Chrome canvas2d 60.13 fps · dom-churn 100.14 ops/s · static pss 564 MB · canvas pss 603 MB · dom pss 663 MB · cpu ~11.7s | — | — | s.khost1 bench-kruze-gpu LXC (Tesla T4, ANGLE gl-egl → ANGLE (NVIDIA … Tesla T4 … OpenGL ES 3.2), no-DISPLAY surfaceless, idle 112-core host) |
KRUZE-181 canonical-GPU attempt. Chrome on the real T4: canvas2d 42.3→60.13 fps (hits the rAF ceiling → paint no longer the bottleneck under hardware GL) and dom-churn 69.7→100.14 ops/s vs the software-VM rows above. These are the Chrome GPU reference numbers. No Kruze row: a canonical Kruze GPU number is physically unobtainable on this card — see the finding row below. |
| 2026-06-02 | productshorizontalkruze | finding (not a measurement) | ANGLE gl-egl renderer probe, Chrome 148 + patched Kruze CEF 148 | why the canonical GPU comparative can't run on a headless Tesla | Chrome no-DISPLAY → NVIDIA Tesla T4; Chrome with $DISPLAY set → Mesa llvmpipe; Kruze (KRUZE_ENABLE_GPU=1) → Mesa llvmpipe |
— | — | s.khost1 bench-kruze-gpu LXC |
Definitive. ANGLE gl-egl reaches the NVIDIA EGL device only surfacelessly (no $DISPLAY). chrome --headless=new sidesteps X → gets the T4. Kruze is a GTK app → requires $DISPLAY → its CEF GPU process binds the X11 EGL, which under Xvfb is Mesa llvmpipe (proven: even Chrome falls to llvmpipe the moment DISPLAY=:99 is exported, --ozone-platform=headless included). The Tesla T4 has no display engine and the container has no nvidia_drv.so, so Xorg-on-Tesla is not available without fragile full-driver provisioning. ⇒ Chrome (no-X) and Kruze (needs-X) are mutually exclusive at GPU on this card; run.sh's xvfb-run forces both to software. A canonical Kruze hardware-GL number needs a display-capable GPU (the owner's laptop / a real Xorg+NVIDIA seat), not a headless compute card. Enablement now in place for that seat: env-gated KRUZE_ENABLE_GPU=1 (webviewapp.cc, default-off) + `KBENCHANGLE / KBENCHKRUZEGPU harness selectors. Reusable infra: bench-kruze-gpu LXC recipe = incus init images:debian13 … -c nvidia.runtimetrue -c nvidia.driver.capabilitiescompute,utility,graphics,video + gpu device (add **before** first start) + copy host 10_nvidia.json into usrshareglvndegl_vendor.d`. RESOLVED same day — the display-GPU seat was realized on the owner laptop; see the canonical comparative row below. |
| 2026-06-02 | productshorizontalkruze | kbench (repeats5 median, `KBENCH_ANGLEgl-egl KBENCHKRUZEGPU=1, real :1` display) |
static + canvas2d + dom-churn fixtures | ★ CANONICAL hardware-GL comparative (Kruze vs Chrome, both on a real GPU) | canvas2d Kruze 60.11 / Chrome 60.16 fps (−0.1%, parity) · dom-churn Kruze 143.75 / Chrome 173.49 opss (−17.1%) · PSS KruzeChrome static 860485 (+78%) canvas 880509 (+73%) dom 970/587 (+65%) · CPU canvas Kruze 2.47 < Chrome 3.05s, dom 6.05 vs 5.25s | — | — | owner laptop note (RTX 4070 Laptop, X11 :1, NVIDIA direct rendering, idle load 0.5/24c) — gpu-consumer-laptop carve-out (heavy-work-isolation.kmd §R9) |
The canonical run the whole #181 saga was waiting for. Both browsers on the real RTX 4070 ([Kruze/GPU] hardware acceleration active … RTX 4070; patched CEF via KRUZE_ENABLE_GPU=1). Thesis-refining result: the software-VM paint deficit (rows 135140: canvas −33%, Speedometer −32%) was largely a software-GL artifact — on real GPU, paint-bound canvas2d is parity (both saturate the 60 fps rAF ceiling; a heavier uncapped render harness — MotionMarkJetStream — is needed to find the true ceiling, now runnable on this seat). dom-churn (main-thread enginelayout) is −17% (the OSRembedding tax on churn-heavy work; was ≈par on the noisy software VM). The robust, persistent gap is MEMORY: Kruze +65–78% PSS on every page — confirming the KRUZE-202 thesis (resource efficiency = memory, not per-frame render). ⚠️ CORRECTED by the Speedometer row below: paint parity does NOT mean real-world parity — the Speedometer-3 responsiveness deficit (−32.7%) persists on GPU, so the compositor rewrites (199200201) address paint (already at parity) and do not address the real deficit, which lives in the main-threadOSRembedding path. |
| 2026-06-02 | productshorizontalkruze | kbench (repeats3 median, `KBENCH_ANGLEgl-egl KBENCHKRUZEGPU=1, real :1`) |
Speedometer 3 (standard harness, mirrored) | ★ real-world responsiveness on a real GPU | Kruze 103.0 / Chrome 153.0 runs/min (−32.7%) · PSS Kruze 1814 / Chrome 1137 MB (+60%) · CPU 48.4 / 37.8s (+28%) | — | — | owner laptop note (RTX 4070, X11 :1, idle 0.5/24c) — gpu-consumer-laptop carve-out |
The decisive heavy-harness result. On the software VM (row 140) Speedometer was Kruze 28.80 / Chrome 42.30 = −32%; on a real GPU it is −32.7% — essentially UNCHANGED. ⇒ Unlike the 60-fps-capped canvas2d (which reached parity on GPU), the real-world SPA responsiveness deficit is GENUINE and GPU-independent — it lives in the main-thread / OSR round-trip / Flutter-shell-contention path (consistent with dom-churn −17%), NOT in GPU paint compositing. This corrects the canonical row's over-reach: kruze matches Chrome on GPU-bound paint, but carries a real ~−17 to −33% deficit on main-thread-bound real-world work plus the +60–78% memory premium. The actual levers are (a) the OSRembedding main-thread path and (b) memory (KRUZE-202) — not the paint-focused compositor rewrites (199200201), which target a cost that's already at parity. JetStreamMotionMark did not complete headless (harness auto-start/score-selector needs version-specific config — focused follow-up). |
| 2026-06-24 | enginessdkkroma | kroma-bench (release, headless) |
80×24, escape-heavy flood (coloured cells + CUP + clears) | VT parse+build throughput · per-frame CPU build (snapshot+Grid+Vello scene, no GPU) | VT 50 MB/s · per-frame build 83 µs median (~12,100 fps CPU ceiling), layout+paint 35 µs | CPU-floor baseline | ✅ (gates: VT>20MB/s, frame<20ms) | s.khost1.dev-linux-kdb (Xeon, llvmpipe) | CPU-representative feel baseline for the Kroma terminal (RFC-019 §4.2 measured-feel leg, CPU part; kroma#024). Directly answers the kterm flood pain (kterm#002: the flood floor was the Dart VT parser+paint) — the Rust vte parser does 50 MB/s and the full-frame CPU build is 83 µs, so the live frame rate is GPU-present-bound, not CPU-bound. GPU present is NOT measured here — this host is llvmpipe (software Vulkan: first render = seconds of shader compile, not representative). Real-GPU frame-time + request→painted vs Tilix = the stack-RFC-012 corpus follow-up under the §R9 gpu-consumer-laptop carve-out. |
| 2026-06-24 | enginessdkkroma | kroma-gpu-bench (release, offscreen render-to-texture, reuses one device, HighPerformance adapter) |
80×24 full coloured terminal frame (866×502 px) | real-GPU cold first-frame (incl. shader compile) · warm steady-state GPU render | cold 41.1 ms · warm 831 µs median (~1,200 fps render ceiling) | feel-gate: frame << 16.6 ms (60fps) | ✅ | owner laptop note (RTX 4070 Laptop GPU, NVIDIA 610.43.02, Vulkan) — gpu-consumer-laptop carve-out, heavy-work-isolation.kmd §R9 |
The real-GPU half of the RFC-019 §4.2 measured-feel gate (kroma#026), the number llvmpipe couldn't give (row above). Built off-laptop on dev-linux-kdb, binary run on the laptop's discrete GPU OFFSCREEN (render-to-texture, no swapchainwindow → no PSRcompositor → no freeze risk). A full terminal frame = ~83 µs CPU build (row above) + ~831 µs GPU render ≈ 0.9 ms → ~1,100 fps ceiling, vs the 60fps (16.6 ms) / 144fps (6.9 ms) budgets — Kroma is ~18-20× under budget per frame; cold first-frame 41 ms ≪ Flutter/GTK ~1-2 s. Remaining feel piece: request→painted vs Tilix (full stack-RFC-012 corpus, XTEST harness — kroma#024 Next). |
No regression targets defined yet — promote to gates after the OTel pipeline (FLOW-103/106) can publish server-side timings split from transport, and kbox-equivalent staging exists for an apples-to-apples client.
How to add an entry
- Run the bench you intend to record. Capture the full command line.
- Append rows to the appropriate sub-section above. Keep one entry per
metric so trends per metric are easy to grep.
- If the entry replaces an older row (same component + same bench + same
hardware), leave the old row in place and add the new row beneath it so historical drift is visible. Optionally annotate the older row's
notescolumn with a→ superseded YYYY-MM-DDpointer. - Reference the entry from the bench's own README.md so future readers
land here automatically.
⟲ Recovered baselines (KVS-285 content-revert recovery, 2026-06-25)
These benchmark rows/sections were silently reverted during the 2026-06-24 shared-tree gutting / 94-commit-fork incident (KVS#285): committed perf baselines that a stale-base commit (c8b03b0f, 2026-06-23) overwrote — git log stayed clean but the file content lost ~13 KB of real measurements. Restored verbatim from the pre-incident high-water-mark blob 934253303d (2026-06-22). The kbox cold-start rows below are the real measured numbers (kbox#174) and supersede the "infranetbox" placeholder (— values, 2026-05-20) still above. Re-file into the canonical sections when convenient; kept here as an attributed block to avoid disturbing the rest of the (concurrently-edited) registry.
| 2026-06-12 | engineslangkoda | ring microbench (per-actor distinct globals; koda src.kd src.asm→nasm→ld→run) |
2-actor ping-pong, 2 M msgs | throughput | ~22.9 M msg/s (87.5 ms); N10→26.4 M, N100→23.5 M, N=1000→11.0 M (todos 1 M msgs) | M:N > Rust (#728/kodec#224) | ✗ (baseline) | s.khost1.dev-linux-koda (Xeon 8176) | Baseline REAL do scheduler cooperativo 1:N (atores de fato executam — destravado por #826 fix + #827 HALT-drain). Throughput cai ~2.4× de N10→N1000: overhead de task-switch ∝ nº de tasks na run-queue. Substitui a row INVÁLIDA (atores nunca rodavam). Régua p/ S2-S7 do #728. LIÇÃO: koda não tem locais de função → usar globais DISTINTOS por ator (senão contadores compartilhados corrompem). |
| 2026-06-12 | engineslangkoda | spawn N tasks triviais (136 KB/task) → run |
muro de escala (default GC vs --no-gc arena) |
maior N que roda | GC on: N≈1500–2000 (256 MB → abort rc=134); --no-gc: N≥30000 (4 GB, ok) |
100 k tasks (#728 S2/S3) | ✗ (gargalo = GC) | s.khost1.dev-linux-koda (Xeon 8176, 1 TB RAM) | O muro é a GC, não memória nem stack-size (VM tem 1 TB livre; brk do heap = 8 GB). Com GC default, 256 MB de task-stacks vivos atinge o _gc_threshold (268435456) → a GC aborta (rc=134/SIGABRT) ao varrer o conjunto vivo. Com --no-gc (arena) escala pra N=30000 (4 GB) sem problema → scheduler+memória aguentam 30k+ tasks; o gargalo é GC-em-escala. Prioriza #728 S3 (GC escalável) > S2 (stack-size). GC-abort rastreado em #828. S2 (136 KB→≤16 KB) multiplica o teto arena ~8.5×. |
infranetkbox — kbox container runtime cold-start
Tracked per meta/docs/stack/registries/self-hosted-pairs.md row 13: performance:cold_start_benchmark is the open gate before kbox flips from stable → official versus DockerPodman. Populated 2026-06-09 by the cold-start harness (`infranetkboxtestsperfcold-start/run.sh, N=30 timed runs each + 1 warmup, image pulled up front) on s.khost1.dev-linux-box` (incus container, ext4, kernel 6.12, kbox overlay fast-path per kbox#174). Baseline = podman 5.4.2 on the same host, same loop.
| 2026-06-09 | infranetkbox | kbox run --rm alpine:3.18 /bin/true × 30 (cold rootfs each) |
alpine:3.18, 1 layer, ~3.4 MiB | p50 wall | 129 ms | ≤ podman run p50 × 1.1 (724 ms) |
✅ | s.khost1.dev-linux-box | kbox 1.1.0 (syscall overlay, kbox#174). podman 5.4.2 same host: p50 658 ms. kbox = 0.20× podman → gate cold_start_benchmark of pair #13 PASSES. Harness: tests/perf/cold-start/run.sh. |
| 2026-06-09 | infranetkbox | (same) | (same) | p99 wall | 225 ms | ≤ podman run p99 × 1.2 (2646 ms) |
✅ | (same) | podman p99 2205 ms (same run). 0 fails on both runtimes. |
infradatakdb — auto-commit-WAL group-commit queue (kdb#792, ε-unblocking)
The #792 off-lock fsync (the GroupCommit leader dup()s the WAL fd + fsyncs without holding the WalWriter lock, so concurrent committers append during the fsync and coalesce onto it). 3-way A/B vs the no-WAL default + the
791 inline path, ext4/md0 (fsync ≈ 2.3 ms), 2000 auto-commit INSERTs × 32
conns. This is what turns the coordinator into real fsync-count coalescing (1984 → 385 fsyncs, ~5 commits/fsync) and collapses the auto-commit-WAL cost from 5.33× (just #791) to 1.22× vs no-WAL — the evidence that justifies 789-ε / #543-2c. Re-run: same bench, TMPDIR=/root/waltmp.
| date | component | bench | dataset | metric | value | target | pass | hardware | notes |
|---|---|---|---|---|---|---|---|---|---|
| 2026-06-08 | infradatakdb | acw_group_commit_bench (--ignored), TMPDIR=/root/waltmp (ext4, fsync ≈ 2.3 ms) |
2000 auto-commit INSERTs × 32 conns | throughput (decoupled-WAL #792) | 920 ins/s | → no-WAL | ✓ | s.khost1.dev-linux-kdb (ext4 devmd0) | Off-lock fsync queue. fsyncs 385 (0.193/ins). 1.22× cost vs no-WAL (was 5.33×). |
| 2026-06-08 | infradatakdb | (same) | (same) | throughput (no-WAL default) | 1127 ins/s | — | — | (same) | Baseline — gate off, 0 fsyncs. The absolute cost ceiling the ε flip is measured against. |
| 2026-06-08 | infradatakdb | (same) | (same) | throughput (inline-WAL) | 166 ins/s | — | — | (same) | group_commit off → 1984 fsyncs under the lock. 6.78× cost; the #792 queue beats it 5.5×. |
| 2026-06-08 | infradatakdb | (same) | (same) | fsync coalescing (#792) | 1984 → 385 | ≪ stmt_count | ✓ | (same) | Real group commit: ~5 commits per fsync under 32-way concurrency. |
infradatakdb — auto-commit-WAL fsync decouple (kdb#791)
A/B of the #791 group-commit decouple for auto-commit INSERTs: decoupled (KDB_GROUP_COMMIT=1, fsync off the adapter lock) vs inline (=0, fsync under the lock). Storage is decisive: on tmpfs (fsync ≈ 2.2 µs) there is no win to observe (fsync ≈ free); the meaningful run is on ext4/md0 (fsync ≈ 2.3 ms). The win is the fsync leaving the adapter write lock (committers pipeline append/apply), not fsync-count coalescing (count stays ≈ 1/insert — that needs the #792 commit queue). Re-run: cargo test -p kdb-pgwire --test pgwire_v2 -- --ignored --nocapture
acw_group_commit_bench with TMPDIR=/root/waltmp (ext4).
| date | component | bench | dataset | metric | value | target | pass | hardware | notes |
|---|---|---|---|---|---|---|---|---|---|
| 2026-06-08 | infradatakdb | acw_group_commit_bench (--ignored), TMPDIR=/root/waltmp (ext4, fsync ≈ 2.3 ms) |
2000 auto-commit INSERTs × 32 conns | throughput (decoupled) | 191–207 ins/s | > inline | ✓ | s.khost1.dev-linux-kdb (ext4 devmd0) | Decoupled consistently faster across 3 runs. Wall ~10 s, stable. |
| 2026-06-08 | infradatakdb | (same) | (same) | throughput (inline) | 124–176 ins/s | — | — | (same) | Inline holds the adapter lock during the 2.3 ms fsync → wall 11–16 s, spike-sensitive. |
| 2026-06-08 | infradatakdb | (same) | (same) | speedup (decoupled/inline) | 1.09–1.61× (~1.3× mean) | ≥ 1× (amortize) | ✓ | (same) | THE #791 acceptance evidence. fsync COUNT unchanged (≈1/insert) — coalescing → #792. Variance = shared host. |
| 2026-06-08 | infradatakdb | (same), TMPDIR=/tmp (tmpfs, fsync ≈ 2.2 µs) |
(same) | speedup (decoupled/inline) | 1.00× | — | — | (same) | NEGATIVE-control: fsync ≈ free → nothing to amortize. Never bench WAL fsync on tmpfs. |
| 2026-06-10 | infradatakdb | kdb-bench --mode pgwire-bulk-select --pg-tls (#769 slice 2c), gateway --pgwire-tls-ktls auto|off |
400 ops × 1 MB bulk SELECT × 4 clients, persistent TLS conns | throughput (kTLS auto vs off) | auto ~398 / off ~434 MB/s (−9%, 4 rounds) | ≥ +15% (offload gate) | ✗ | throwaway incus VM --vm Debian13 kernel 6.12.90, 2 vCPU, loopback (kTLS-capable, tcp_available_ulp=tls) |
#769 kTLS offload: NO software-kTLS throughput win on loopback — slight regression. Offload VERIFIED engaged (strace setsockopt(TCP_ULP,"tls")=0 + metric eligible). Root cause: userspace rustls already uses AES-NI (kTLS-SW saves no crypto CPU); loopback has no NIC DMA copy to elide; CorkStream+ULP setup adds overhead. The kTLS win needs NIC HW offload (CONFIG_TLS_DEVICE) on a real high-bandwidth link — asset-gated. Feature ships default-OFF (slice 2a); correctness proven (2b). Numbers from a single loopback VM — not comparable across hardware. |
| 2026-06-10 | infradatakdb | ~/temp/gcp-soak.sh (#796), musl gateway, primary+writers taskset -c0 / replica taskset -c1, netem 73ms on lo (~146ms RTT) |
2 writers heavy load (50-row txns) + 100 probes, single host single clock | replication lag p99 (Phase 8.2, during load) — DEDICATED CPU | p50 723 / p90 1087 / p99 1129 ms (true apply ≈934 ms less ~196 floor) | < 1000 ms | ~ | GCP koder-fleet-geo-us n2d-standard-2 (2 vCPU AMD EPYC DEDICATED, 8 GB) |
#796 dedicated-CPU p99 — BORDERLINE. Got a real dedicated host (Compute SA + resize e2-micro→n2d). Dedicated CPU improved raw p99 1263→1129 but did NOT clear the literal <1s gate (130ms over) — the residual is the *146ms simulated WAN RTT, not a kdb apply defect (apply-only p99 ≈934ms PASS; median 723ms well under; converges 100%). Gate-semantics owner-decision:*apply-path-only → PASSclose; end-to-end-incl-WAN → re-run on a real 2-machine LAN (no netem floor). Node restored to e2-micro after. Harness bug: final summary psql hangs under netem (data salvaged from probe.outpoller.out). |
| 2026-06-10 | infradatakdb | (same, SUPERSEDED) | (same) | replication lag p99 — burstable | p50 797 / p90 1111 / p99 1263 ms | < 1000 ms | ✗ | GCP e2-micro (BURSTABLE) | Earlier run on the burstable e2-micro — superseded by the dedicated-CPU row above (CPU-steal inflated the tail). Kept for the burstable-vs-dedicated delta (1263→1129). |
| 2026-06-10 | productshorizontalkruze | kbench (repeats1, `KBENCHKRUZEGPU1 ANGLE-Vulkan NVIDIA ICD, real :0`) |
MotionMark 1.x (standard harness, mirrored) — uncapped render | MotionMark complexity score, Kruze vs Chrome on the real RTX 4070 | Kruze 700.91 / Chrome 1017.68 (ratio 0.689 = −31.1%) · FCP Kruze 76 / Chrome 308 ms · load Kruze 56 / Chrome 38 ms | — | — | owner laptop note (RTX 4070, X11 :0) — gpu-consumer-laptop carve-out · ⚠️ advisory: busy host loadavg 14.19/24 (concurrent Claude sessions; idle_warning:true in the JSON) — the same-run ratio is the headline (symmetric contention cancels), raw ms are indicative |
The uncapped render harness the canonical row said was still needed — now run. Unlike the 60-fps-capped canvas2d fixture (which read parity because both saturate the rAF ceiling), MotionMark ramps complexity with no cap, so it exposes the true render headroom: Kruze sustains −31.1% of Chrome's animated-element load on the same GPU. This closes the canonical row's open question and kills the "paint parity → render is fine" over-reach for good: the −31% here + Speedometer −32.7% + dom-churn −17% now triangulate to one conclusion — Kruze's deficit is in the OSR/embedding + main-thread path, not GPU paint compositing, and it is real and uncapped, not a software-GL artifact. Confirms the KRUZE-202 thesis direction (memory + main-thread, not the 199200201 paint-compositor rewrites). Interesting secondary: Kruze's FCP is 4× faster (76 vs 308 ms) — first paint is cheap; sustained throughput is where the embedding tax lands. JetStream deliberately NOT recorded: JetStream 2.x bundles WebKit-only *-inspector subtests (json-stringify-inspector → ReferenceError: obj is not defined) that hang any Chromium engine's run (isDone never flips) — and JetStream measures V8, identical between Kruze and Chrome (the control group), so it adds ~0 signal over the three deltas above. |
infranetdns — Herald authoritative ServeDNS (response cache, dns#021 Slice 1/5)
Anchor for the dns#021 response cache: serving a small multi-RR A pool (4 records, the GSLB-pool shape) through authoritative.Server.ServeDNS, comparing a cold resolve+render path against the warm cache-hit path. Harness: internal/authoritative/perf_test.go (collects a per-op latency sample → real percentiles, not Go-bench mean). Reproduce: go test ./internal/authoritative/ -run TestServeDNSPerfBaseline -v.
| date | component | bench | dataset | metric | value | target | pass | hardware | notes |
|---|---|---|---|---|---|---|---|---|---|
| 2026-06-22 | infranetdns | go test ./internal/authoritative/ -run TestServeDNSPerfBaseline -v |
4-RR A pool / 200 K serial ops | p50 | 5.29 µs | — | — | koder-laptop (Linux 6.12, i7-13700HX, 24 lcores, 63 GB) | Uncached: full resolve + RR render + (no DNSSEC) per query. |
| 2026-06-22 | infranetdns | (same) | (same) | p99 | 15.01 µs | — | — | koder-laptop | Uncached p99. |
| 2026-06-22 | infranetdns | (same) | (same) | throughput | 615 936 q/s | — | — | koder-laptop | Uncached, 24 goroutines (RunParallel-style). Per-core ≈ 25.7 K q/s. |
| 2026-06-22 | infranetdns | (same) | (same) | p50 | 473 ns | — | — | koder-laptop | Cache hit (dns#021 Slice 1): ~11× faster than uncached p50. |
| 2026-06-22 | infranetdns | (same) | (same) | p99 | 2.19 µs | — | — | koder-laptop | Cache hit p99: ~6.8× faster than uncached. |
| 2026-06-22 | infranetdns | (same) | (same) | throughput | 4 074 496 q/s | — | — | koder-laptop | Cache hit: ~6.6× the uncached throughput. First dns serving baseline; p99 target untracked until the GSLB control-plane (Slice 4, DNSP-gated) sets an SLO. |