RFC 001 — Koder Krep (search engine & CLI)

Status: DRAFT — initial scoping. Naming decided (krep, see §10). Module scaffolded at engines/search/krep/ (2026-05-02). 8/10 open questions resolved (§13). Now lives at its permanent home; the file at meta/docs/stack/rfcs/search-RFC-001-engine-and-cli.kmd is a symlink back here. Promotion to Approved status pending closure of items 9 (/k-bench reference dataset versions — depends on F1 setup).

1. Summary

This RFC proposes Koder Krep — a modern search engine and CLI for the Koder Stack, a from-scratch replacement for grep and ripgrep (rg) in all our agent loops, developer workflows, and human-facing search experiences (chat transcripts, log triage, code navigation).

The engine is a single library written in Koder Koda (Native AOT) that exposes search through multiple frontends: a CLI binary (krep), an MCP server, an LSP-style server, an HTTP/gRPC daemon, and bindings for DartGoRust. It supports three orthogonal search modes over the same engine: text (regex/literal), structural (AST per-language), and record (JSONL, NDJSON, structured logs, chat transcripts).

The non-negotiable success criterion is performance parity with rg in plain-text search, evaluated against a fixed reference benchmark suite per §6, with a two-gate progression that maps directly onto the policies/self-hosted-first.kmd status enum:

• F1 done / opt-in / status: stable — geomean ≤ 1.10× rg plus

  the structural criteria below. AI may prefer Krep case-by-case.

• Default flip / status: official — geomean ≤ 1.05× rg (matches

  policy default for hot-path G2). AI MUST prefer Krep.

The dual gate decouples the engineering finish-line of F1 from the policy threshold for becoming the Stack default — both are real milestones, the first faster, the second binding.

2. Motivation

2.1 What grep and rg get right

ripgrep is the current reference: SIMD-optimized matchers, RE2-style regex (no catastrophic backtracking), gitignore-aware walk, multi-thread by default, sensible UX. It's a ~10-year-tuned tool and the rational baseline. For most plain-text needs, it is excellent.

2.2 What we hit constantly that they don't solve

• AST-aware queries. "All call sites of KoderApp(...)" returns matches

  inside comments, doc strings, similarly-named functions. We end up filtering by hand. ast-grep and comby exist but are not drop-in, not unified across our languages (KL, Dart, Go, Rust), and not integrated with our other tooling.

• Multi-line as a default. Finding a TOML block, a JSON value spanning

  lines, a function body — all require (?s) flags, awkward -A 50 guesses, or shelling out to a structural tool.

• Search-and-replace as a first-class operation. rg doesn't replace.

  We pipe through sedsdxargs, lose preview/dry-run, and have no awareness of "don't replace inside string literals or comments".

• Result ranking. rg returns matches in walk order. For "where is X

  defined?" we get every mention equally weighted instead of "the one declaration first, the 50 references after".

• Scope-aware context. -A/-B with line counts is a guess. We want

  "the enclosing functionclassblock" — needs a parser, not a buffer.

• Record-aware search. Our agent loops, log pipelines, and chat

  transcripts are JSONL/NDJSON. Grep returns line matches, losing record boundaries (turns, log entries, transactions). We re-implement record reconstruction in every tool that consumes search output.

• Server mode. Each rg invocation pays startup + walk cost. In agent

  loops where Claude Code, Codex, Gemini call grep dozens of times per session, this is dominant overhead. A long-running daemon with a walker cache and optional incremental index turns repeated queries from ~50ms to sub-millisecond.

• First-class structured output and integration. rg --json exists but

  is a second-class citizen. We want NDJSON as the canonical output, with human rendering as a pretty-printer. We want native MCP and LSP, not wrappers.

2.3 Why now

Three convergent pressures:

1. Agent-driven development. Claude Code, Codex CLI, and Gemini are

   now primary consumers of grep in our workflow. Each call costs tokens (output) and permission prompts (in Claude Code). A grep designed for agent loops — server mode, MCP-native, structured-output-canonical — measurably reduces both.

2. Koder Koda flagship readiness. KL Native AOT is the official

   compilation target for performance-sensitive Stack components (koder-jet, kodec, etc.). A search engine is a textbook KL showcase: tight loops, SIMD, zero-alloc, IO-bound under load.

3. Reuse-first policy demands consolidation. Today every Stack

   component that searches text rolls its own thing on top of grep, rg, find, or ad-hoc Go regex. /k-search, /k-housekeep, /k-audit-landings, /k-evolve would all benefit from a shared library — and they need to feel native to the Stack, not shell out.

3. Non-goals

Equally important to declare upfront. This RFC explicitly does not propose:

• Beating rg on plain-text raw regex by ≥1.10×. The last 5–10% of

  perf is ~10 years of tuning by @BurntSushi and not worth chasing. Parity within geomean 1.10× is the bar (see §6).

• Wrapping or embedding rg for any release. No rg dependency at

  any point; design is pure-KL from day one. (Rejected option discussed and discarded during scoping conversation 2026-05-02.)

• A general-purpose code intelligence platform. No symbol resolution,

  no type inference, no go-to-definition. AST-aware search uses parsers for matching, not for semantic analysis. LSP-style server is for search queries from editors, not for replacing clangd/gopls.

• A full-text search platform (ZoektSourcegraphElasticsearch).

  Optional indexing exists for repeated-query speedup in server mode, but the default mode is index-less ad-hoc search like grep.

• A replacement for find. File enumeration is part of the engine but

  not the primary surface; find-style queries are out of scope for v1.

• A shell language. Composable pipelines exist (§7.5) but the syntax

  is flag-driven, not a new mini-language to learn.

4. Design principles

4.1 Engine = library; CLI = thin shell

The single source of truth is a library written in KL Native AOT. Every frontend (CLI, MCP server, LSP server, HTTP/gRPC daemon, language bindings) is a thin wrapper that calls into the library. There is no "CLI-only logic". This pattern matches the Koder Stack engine+product split (cf. project memory engine_product_pattern).

4.2 Three search modes, one engine

The engine exposes three orthogonal modes, each over the same indexing, walk, and output infrastructure:

• text — regex/literal text matching. Default mode. Compatible

  conceptually with grep/rg.

• structural — AST-aware queries via per-language parsers. "All

  function declarations matching name foo*", "all calls of bar in scope of class Baz", etc.

• record — record-oriented for line-delimited structured formats

  (JSONL, NDJSON, structured logs, chat transcripts). Returns whole records, not lines. Knows about turn boundaries in transcripts.

Modes are selected by flag (--mode=text|structural|record) or by auto-detection from input shape (extension/content sniff). They share the walk, output, and ranking infrastructure.

4.3 Streaming-first I/O

mmap for large files, read for small/streaming. Walk is parallel across CPUs with bounded queue depth. Output is streamed (no materialize all results before printing). Bounded memory: search runs over a 100 GB tree without loading the tree into memory.

4.4 Structured output is canonical

NDJSON is the engine's canonical output format. Every match emits a record with: file path, byte range, line number, line text, captures, mode-specific metadata (AST node kind for structural, record offset for record mode). The human-friendly TUI output is a pretty-printer over the NDJSON stream. This makes machine consumption (other tools, agents) the zero-friction path.

4.5 Server mode is first-class, not an afterthought

A daemon mode (koder-search serve) keeps the walker cache hot, supports optional incremental indexing, and serves queries over a local HTTP+gRPC socket. CLI invocations can transparently route to the daemon if it's running, falling back to standalone otherwise. Agent loops (MCP) talk to the daemon directly.

4.6 Hyperscale-first

Per policies/hyperscale-first.kmd: design for 10M files, 100 GB repositories, 1000 concurrent queries in server mode. Where two implementations have similar effort, choose the one that scales. This is not "we'll optimize later" — the architecture starts there.

5. Architecture overview

                ┌─────────────────────────────────────┐
                │  Frontends (thin shells)            │
                ├─────────────────────────────────────┤
                │  • CLI (k<slug>)                    │
                │  • MCP server                       │
                │  • LSP-style server                 │
                │  • HTTP / gRPC daemon               │
                │  • Bindings: Dart, Go, Rust, KL     │
                └─────────────────┬───────────────────┘
                                  │
                ┌─────────────────▼───────────────────┐
                │  Engine library (KL Native AOT)     │
                ├─────────────────────────────────────┤
                │  • Query planner                    │
                │  • Mode dispatch (text/struct/rec)  │
                │  • Result ranker & formatter        │
                └──┬───────────────┬──────────────────┘
                   │               │
        ┌──────────▼────┐  ┌───────▼─────────┐
        │ Match engines │  │ I/O subsystem   │
        ├───────────────┤  ├─────────────────┤
        │ • RE2-style   │  │ • Walker        │
        │   NFA→DFA     │  │   (gitignore,   │
        │ • Aho-Corasick│  │    parallel)    │
        │ • SIMD literal│  │ • mmap reader   │
        │ • AST parsers │  │ • Stream reader │
        │   (per-lang)  │  │ • Index         │
        │ • Record split│  │   (optional)    │
        └───────────────┘  └─────────────────┘

5.1 Match engines

• RE2-style NFA→DFA matcher. No backreferences in the high-perf path

  (matches rg's regex-automata design choice). Backreferences fall back to a slower path or are rejected with a clear message.

• Aho-Corasick for literal sets. When the pattern is a literal or

  alternation of literals, switch to AC. Same trick rg uses.

• SIMD literal scanning. AVX2/SSE2 on x86_64, NEON on aarch64. Used

  for the prefilter pass. Per-arch dispatch resolved at process start.

• AST parsers per language. v1 set: KL, Dart, Go, Rust, Python,

  JavaScript/TypeScript, Markdown, KMD. Extensible. Likely tree-sitter for breadth, with custom KL parser for KL itself.

• Record splitter. Format-aware split for JSONLNDJSONstructured

  log formats and for our chat transcript JSONL.

5.2 I/O subsystem

• Walker. Respects .gitignore, .koderignore, .ignore. Parallel

  across CPUs with work-stealing. File-type detection by extension and content sniff.

• mmap reader. For files above a threshold; bounded virtual memory

  budget per query.

• Stream reader. For stdin and small files; preserves streaming

  semantics for tail | search.

• Index (optional). Trigram-based, Zoekt-style. Off by default.

  Enabled per-tree via koder-search index <path>. Incremental update on filesystem changes (inotify/fsevents). Stored under ~/.cache/koder-search/ keyed by tree path.

5.3 Result ranker

Default ranking is walk-order (deterministic, no surprise). Optional flags:

• --rank=relevance — score by file type weight (source > doc > generated),

  match kind (definition > reference > comment string), match density.

• --rank=mtime — most recently modified first.
• --rank=size — smallest files first (often the best signal for "find

  the canonical definition").

Ranking happens after match collection but before output streaming; output is bounded by --limit=N to keep ranker memory finite.

6. Performance bar (two-gate progression)

Resolved 2026-05-02: the perf bar uses two gates, not one. Gate 1 marks F1 engineering completion (Krep becomes opt-in / status: stable); Gate 2 matches the policies/self-hosted-first.kmd G2 hot- path default of ≤ 1.05× and triggers the default-flip across the Stack (status: official). This decouples engineering completion from policy compliance without loosening the policy floor for default- by-default usage. Calibration discussion summary: option (b) tolerant 1.10× was chosen as the F1 finish line before self-hosted-first became canonical; option (c) dual-bar reconciles it with the policy default without slipping the F1 schedule.

All four structural criteria below (datasets, outlier cap, parity-or-win count) apply at both gates. Only the geomean threshold changes.

6.1 Gate 1 — F1 done / opt-in / status: stable (plain-text case)

When all four criteria pass on the reference benchmark suite via /k-bench:

1. Geomean ≤ 1.10× rg (average at most 10% slower).
2. No single benchmark > 1.5× rg (no catastrophic outlier).
3. ≥ 3 benchmarks at ≤ 1.0× rg (parity or wins) — proves the

   SIMD path, parallel walk, and matcher are pulling weight, not just an "ok average from luck".

4. Reference datasets, fixed and versioned:
   • Koder monorepo snapshot (heterogeneous languages, ~5 GB)
   • Linux kernel snapshot (massive C codebase)
   • Chromium snapshot (massive C++/JS codebase)
   • 10 GB synthetic JSONL (chat transcripts realistic shape)
   • "Pathological regex" set — patterns that exhibit catastrophic

     backtracking on PCRE but should not on RE2. We must match rg here (both engines avoid the trap).

Effect: koder.toml [self_hosted].status = "stable" for the plain-text case. AI may prefer Krep case-by-case. CLAUDE.md and command defaults stay on rg.

6.2 Gate 2 — Default flip / status: official (plain-text case)

Same four criteria, with one tightened threshold:

1. Geomean ≤ 1.05× rg (policy default for hot-path G2 in

   self-hosted-first.kmd).

2. No single benchmark > 1.5× rg (unchanged).
3. ≥ 3 benchmarks at ≤ 1.0× rg (unchanged).
4. Same reference datasets (unchanged).

Plus G3 + G5 from the policy:

• ≥ 3 stable releases without breaking changes (G3 stability).
• ≥ 2 production Stack consumers actively using Krep (G5).

Effect: koder.toml [self_hosted].status = "official" for the plain- text case. Default flips: krep becomes the default in CLAUDE.md, Claude Code config, /k-search, /k-housekeep, etc. AI MUST prefer Krep unless an explicit gate fails for the specific case.

6.3 Why dual-bar instead of single ≤ 1.05× from the start

A single ≤ 1.05× gate would:

• Inflate F1 from ~3-4 months to ~5-6 months focused (the marginal

  perf wins from 1.10× to 1.05× cost more than the previous 0× to 1.10×, by diminishing returns). No usable Krep in the meantime.

• Block any case-by-case opt-in (e.g. agents preferring Krep in agent

  loops where the new dimensions matter, even at 1.08× plain-text) until the strict bar is met.

The dual-bar lets F1 ship at ~3-4 months as stable, validates the architecture and dimensions in real use, and reserves the strict default-flip threshold for what it is in the policy: the gate to become the universal default, not the gate to exist.

Stricter geomean (always ≤ 1.0×) was discussed and rejected during scoping as unmeasurable in practice (some inputs always favor 10-year- tuned rg).

7. CLI surface (sketch)

Binary: krep (per binaries-and-cli/naming.kmd — k<slug> convention).

krep <pattern> [path...]          # default text mode, like rg
krep --mode=structural 'fn:foo'   # AST query
krep --mode=record '.level=ERROR' # JSONL record query
krep replace 'old' 'new' [path]   # search-and-replace with preview
krep scope --fn -p 'pattern'      # match + return enclosing function
krep serve                        # start daemon
krep index <path>                 # build optional index
krep bench                        # compare against rg on local repo

7.5 Composable pipeline

krep find 'KoderApp' --json \
  | krep narrow --scope fn \
  | krep filter --not-in-comments \
  | krep rank --relevance

Each subcommand reads NDJSON on stdin, writes NDJSON on stdout, with human pretty-printing only at terminal end. No new shell language.

8. Frontends

8.1 CLI

Standalone binary krep, KL Native AOT compiled, statically linked where possible. DebrpmAppImagemacOS pkgWindows MSI per specs/binaries-and-cli/naming.kmd. D-Bus ID dev.koder.krep, symlink /usr/local/bin/krep.

8.2 MCP server

Native MCP implementation. Exposes tools: search_text, search_struct, search_record, replace, scope_match. Direct integration with Claude Code (drop-in for built-in Grep tool), Codex CLI, Gemini.

8.3 LSP-style server

Speaks LSP-flavored JSON-RPC for editor integration (Zed, VSCode, Helix, Emacs). Provides "find references" / "find definitions" / "workspace symbol" semantics where they overlap with our search modes. Not a full LSP server — search subset only.

8.4 HTTP / gRPC daemon

ks serve opens on 127.0.0.1:9870 (TBD port) and Unix socket /run/user/$UID/koder-search.sock. CLI invocations transparently route to the daemon when running. Agents talk to the daemon directly to skip process startup cost.

8.5 Language bindings

• KL — native, the engine itself.
• Dart — for Flutter UIs (in-app search in Koder products).
• Go — for CLI tooling and existing Go components.
• Rust — for kodec and other Rust components, via FFI.

Bindings ship via the SDKs (engines/sdk/), not as separate distributions.

9. Module placement — decided

Following the shared name variant of the engine+product pattern (audience is the same — devs and agents — so engine and CLI share the brand krep, similar to kicon, kodec, kdb, jet):

• Engine + CLI: engines/search/krep/ (decided 2026-05-02).
  • Domain: engines/ (per RFC-003 — Krep is consumed by embedding +

    execution, not via API or UI).

  • Area: search/ (new top-level area under engines/).
  • Sector: krep.
  • Internal layout per RFC-006:
    • engine/ — the KL Native AOT library (the search engine itself).
    • app/cli/ — the krep binary frontend.
    • landing/ — landing page deployed to krep.koder.dev.
    • docs/ — RFC, ADRs, CHANGELOG, README.
    • backlog/ — pending/, in-progress/, done/.
    • koder.toml — module manifest including the [self_hosted]

      block (see §15).

• Why new area search/: Krep doesn't fit cleanly under existing

  engines/ areas (kodec/ is media codecs, lang/ is the Koder Language toolchain, sdk/ is consumer-side SDKs). Search-and- indexing is a distinct architectural concern with future components likely to join (e.g. an index daemon, query federation, semantic embeddings). Single-purpose area is future-proof.

• Daemon (long-running server frontend) ships as part of the same

  module under engines/search/krep/ exposed via krep serve. If it later grows into a shared multi-tenant service across hosts, it spins out to services/foundation/krep/ per RFC-003 §7 decision tree. This is a future migration, not a v1 concern.

• Future search-area neighbors (not part of this RFC): a dedicated

  index daemon (engines/search/index/), federation layer (engines/search/federation/), semantic search engine (engines/search/semantic/). Mentioned only to validate the area choice — not in scope here.

10. Naming — decided

Brand: krep (shared engine + CLI). Decided 2026-05-02 via Brand score session per specs/naming/brand-score.kmd.

• Engine: koder-krep (library + daemon)
• CLI binary: krep
• Domain: krep.koder.dev (future landing)
• D-Bus ID: dev.koder.krep

Rationale

Concept

"Krep into codetextrecords to find matches." Action verb, energetic, short, memorable. Pairs naturally with the existing Koder Stack name family (Tune, Talk, Pass, Keys, Eye, Koru, Take, Sky, Drive, Mosaic), all of which are short concrete nouns/verbs in the same register.

Why shared instead of split engine+CLI

Existing Stack precedent for shared single-name (engine + CLI under one brand): kicon, kodec, kdb-next, koder-jet. The split pattern (Kompass+Mosaic, Raven+Kmail, Keys+Pass) applies when engine is B2B infrastructure and product is B2C end-user — different audiences. For Krep, both engine and CLI address the same audience (devs and AI agents), so a single brand minimizes naming overhead and avoids brand fragmentation across our developer surface.

Collision check

• No major dev-tool collision in code search (DuckDuckGo Krep, Apache

  Hudi DiVE — both obscure or unrelated).

• No CLI command collision in PATH (krep is a Docker layer inspector

  by wagoodman/krep — niche, not a default install, and our binary is krep so there is no shell-level collision).

• Domain krep.koder.dev controlled.

Runners-up considered

• scour (Brand 71) — semantically more precise, less collision; lower

  Brand score overall.

• probe (Brand 71) — more abstract/technical, less energetic.
• sense, lens, vista, hawk — Brand 75–80 but each had a heavier

  pre-existing brand collision (Sense*, Google/Apple Lens, MS Vista, Hawk security tools).

• fish, dig, find (Brand 81–86 as raw scores) — fatal CLI

  collision with shellDNSUNIX tools regardless of k-prefix.

Full session log of 25 scored candidates is in the conversation transcript of 2026-05-02.

11. Phased roadmap

Phases gate on concrete deliverables, not time. Each phase ends with a release tag and /k-bench report.

F1 — Text mode + perf parity (estimated 3–4 months focused)

The hardest phase. Deliverable:

• Engine library with text mode end-to-end.
• CLI binary (one frontend), no daemon yet.
• RE2-style matcher in KL with SIMD literal prefilter.
• Aho-Corasick for literal sets.
• Parallel walker with gitignore awareness.
• NDJSON output canonical, human pretty-printer.
• /k-bench integration with all reference datasets from §6.

Gate to F1 done: §6.1 Gate 1 criteria met on /k-bench report (geomean ≤ 1.10× plus structural criteria). At this point the component flips to status: stable in the registry; AI may prefer Krep case-by-case but defaults stay on rg.

Subsequent perf tuning (in F2+ in parallel) closes §6.2 Gate 2 (geomean ≤ 1.05×). Together with G3 (≥3 stable releases) and G5 (≥2 prod consumers) it triggers status: official and the default-flip across the Stack.

F2 — Record mode (estimated 3–4 weeks)

• JSONL/NDJSON record splitter.
• Chat transcript turn-aware splitter (for /k-search).
• Structured log format awareness (jet/observe formats).
• Record-level operations (filter, project, sort).

F3 — Server mode + MCP (estimated 4–6 weeks)

• ks serve daemon with walker cache.
• HTTP+gRPC over local socket.
• CLI auto-routing to daemon when running.
• Native MCP server integrated with Claude Code, Codex, Gemini.
• Optional incremental trigram index.

F4 — Structural mode (estimated 6–8 weeks)

• Parser integration (tree-sitter for most languages, custom KL parser

  for KL).

• Structural query language (TBD; possibly ast-grep-compatible

  subset to ease adoption).

• Scope-aware context (--scope=fn|class|block).

F5 — Replace + LSP + bindings (estimated 4–6 weeks)

• Search-and-replace with preview, dry-run, structural-aware

  exclusions (don't touch comments / strings unless flagged).

• LSP-style server.
• DartGoRust bindings via SDKs.

Total realistic timeline: ~7–10 months to complete all five phases. F1 can ship as v1.0 standalone if needed, with subsequent phases as feature releases.

12. Risks

• Perf parity is harder than it looks. RE2-style matcher in a new

  language without 10 years of tuning may miss the bar. Mitigations: port the proven algorithms (not invent new ones), profile early and often, accept option (b) as the bar (geomean 1.10×, not strict ≤ 1.0×).

• AST parser maintenance. Tree-sitter grammars drift; per-language

  AST queries are surface area to maintain. Mitigation: ship structural mode with limited language set in F4, expand only when usage justifies.

• Daemon reliability. A long-running daemon adds operational

  complexity (zombie processes, stale caches, version mismatch with CLI). Mitigation: daemon is opt-in; CLI works standalone always; daemon failures fall back to standalone transparently.

• MCP ecosystem instability. The MCP spec is young; Claude Code and

  Codex implementations diverge. Mitigation: design MCP server as a thin wrapper around the engine library; rewriting it for spec changes is contained.

• Naming bikeshed. Naming session can drag. Mitigation: use the

  brand-score policy (objective scoring), time-box to one session.

13. Open questions

1. Module home for the engine. Resolved 2026-05-02: new

   area engines/search/, sector krep → engines/search/krep/. See §9.

2. Performance bar alignment with self-hosted-first.kmd G2.

   Resolved 2026-05-02: dual-gate progression — Gate 1 (§6.1) geomean ≤ 1.10× for status: stable, Gate 2 (§6.2) geomean ≤ 1.05× (policy default for hot-path) for status: official and default flip. Decouples F1 engineering finish-line from policy compliance for default-by-default usage. See §6 and §15.1.

3. Search-tooling capability sub-policy (G4). Resolved 2026-05-02:

   draft new policies/reuse/search-tooling-first.kmd sub-policy. Rationale: search-tooling has a distinct capability surface (MCP contract stability, structural-mode language coverage, daemon graceful-shutdown, index format versioning, NDJSON output contract) that does not fit cleanly under build-tooling-first (which covers determinism + idempotence for build pipelines). Krep is the seed consumer; future search-area components inherit. Sub-policy lands as a separate PR before F1 ratification, drafted in parallel with module scaffold.

4. KL standard library readiness for SIMD intrinsics. Investigation

   completed 2026-05-02:

   • AVX2/SSE on x86_64: Koder Koda already emits AVX2/SSE

     instructions. compiler_backend.kd contains 158 asm_emit lines referencing AVXSSE registers (ymmxmm) and instructions (vmovaps, vpaddd, movdqu, etc.). Runtime detects SSE4.2 via CPUID (line ~4490). SIMD intrinsics surface exists at AST level (simd.zero_i8x16, simd.f32x4, etc.). Auto-vectorizer hook exists (try_auto_vectorize_simd). Registry confirms simd is in gates_passed for Koder Koda's hot-path lib use. Gate G2 viable for x86_64.

   • NEON on aarch64: Gap. arm64_codegen.kd and

     arm64_encoder.kd contain zero NEON-related symbols (grep -E "neon|NEON|ld1|st1|fmla" returns 0). ARM64 backend today emits scalar arithmetic only.

   • Implication for Krep perf bar: rg uses NEON on aarch64

     (Apple Silicon, Linux ARM SBCs). Krep on aarch64 without NEON codegen will not hit Gate 1 (geomean ≤ 1.10×) on aarch64. Mitigation: scope F1's perf bar to x86_64 only; aarch64 parity becomes a follow-up gate in a NEON-codegen RFC against engines/lang/lang/. Add to RFC §6 datasets that the reference benchmark suite is x86_64; aarch64 parity tracked separately.

5. Tree-sitter integration. Resolved 2026-05-02: embed

   tree-sitter (C library) via FFI for F4 (structural mode). Rationale: ship velocity (tree-sitter has battle-tested grammars for KL, Dart, Go, Rust, Python, JS/TS, Markdown — much faster than writing parsers in KL). KL-native parsers tracked as a follow-up v2 RFC once the structural API stabilizes. Acceptable cost: a C-library FFI dependency, contained behind the backends/structural.kd interface and only enabled when --mode=structural is used.

6. Optional index storage location. Resolved 2026-05-02:

   ~/.cache/koder-krep/<tree-hash>/ for index storage. Rationale: follows XDG cache dir convention familiar to users; survives git clean; doesn't bloat repo working trees. <tree-hash> is the SHA-256 of the canonical tree path so multiple checkouts of the same repo share an index. Override via KODER_KREP_CACHE env var.

7. CLI backwards-compat with rg flags. Resolved 2026-05-02:

   yes, krep accepts rg's common flags directly (-i, -n, -A/-B/-C, -e, --type=..., plus the .gitignore/.ignore resolution rules). Required to satisfy G1 (feature parity) for the plain-text case-of-use. Krep-specific flags use long form (--mode=, --scope=, --rank=) to avoid colliding with rg short-flags. A formal compatibility table goes in engines/search/krep/docs/cli-compat.kmd once the module is scaffolded.

8. Adoption order across the Stack. Resolved 2026-05-02:

   ordered rollout:

   1. /k-search — first. Lowest risk: it operates over chat

      transcripts (JSONL), so it benefits primarily from Krep's record mode (F2). Plain-text perf bar (Gate 1) doesn't gate this adoption — record mode has its own gates.

   2. /k-housekeep audit walk — second. Read-only across the

      monorepo; failures observable and reversible.

   3. /k-audit-landings — third. Larger surface; needs Krep

      stable on diverse file types.

   4. /k-evolve ticket discovery — fourth. Heaviest

      integration; uses all three modes (textrecordstructural).

   Adoption #1 + #2 satisfy G5 (≥ 2 production consumers). After that, default-flip in CLAUDE.md per §6.2 Gate 2.

14. References

• policies/hyperscale-first.kmd — design principles
• policies/reuse-first.kmd — engine library reuse pattern
• policies/self-hosted-first.kmd — gates + decision protocol for Koder vs external
• policies/document-format.kmd — .kmd for new docs
• registries/self-hosted-pairs.md — registry of self-hosted ↔ external pairs
• specs/binaries-and-cli/naming.kmd — k<slug> binary naming
• specs/naming/brand-score.kmd — naming methodology
• RFC-003 (monorepo taxonomy) — module placement rules
• RFC-006 (sector internal layout) — engineapplanding structure
• RFC-009 (specs and policies location) — RFC hosting convention
• policies-RFC-002-self-hosted-first — design rationale for the self-hosted-first framework
• engine_product_pattern (project memory) — engine + product split
• koder_grep (project memory) — perf bar and adoption gate
• self_hosted_first (project memory) — gates + registry summary
• ripgrep source — reference for matcher and walker design

  (regex-automata, ignore, aho-corasick crates)

• ast-grep, comby — prior art for structural search
• Zoekt — prior art for trigram index in code search

15. Self-hosted-first alignment

Krep replaces an external dependency (rg / grep) used widely in the Stack and across all our agent loops. It is unambiguously a self-hosted ↔ external pair under policies/self-hosted-first.kmd and must declare a [self_hosted] block in its koder.toml once scaffolded, joining the registry at meta/docs/stack/registries/self-hosted-pairs.md.

15.1 Gate mapping (Krep ↔ rg/grep, plain-text case)

15.2 Status trajectory

15.3 Implications

• koder.toml [self_hosted] block at scaffold time — mandatory per

  the policy's anti-drift heuristic for components under engines/ with a koder--prefixed name.

• Per-case gate evaluation — Krep's status is not a single global

  value. Plain-text, record, and structural modes are evaluated independently, matching the policy's per-case-of-use semantics.

• CHANGELOG entry on first adoption — when the first production

  Stack consumer flips from rg to krep, log it per the policy's decision protocol question 3.

Status of ratification items (live tracker)

Next concrete steps for promotion DRAFT → Approved

1. Resolve items 5, 6, 7, 8 (eng + arch decisions requiring user input).
2. Scaffold module via /new-product or manual layout per RFC-006.
3. Write koder.toml with [self_hosted] block (status experimental,

   gates_pending listing all G1–G5).

4. Move this RFC to engines/search/krep/docs/rfcs/001-engine-and-cli.kmd

   with symlink-back at meta/docs/stack/rfcs/search-RFC-001-engine-and-cli.kmd.

5. Open F1 tracking ticket in engines/search/krep/backlog/pending/.
6. Lock /k-bench reference datasets (item 9) as part of F1 setup.
7. Re-circulate RFC for Approved status; bump frontmatter status field.