AI Memory — Long-term Agent Memory

• Area: Intelligence
• Path: services/ai/memory
• Kind: Self-hosted episodic memory + semantic recall (per user/tenant)
• Status: v0.0.3 — #007 kdb store swap shipped 2026-06-22. Production EpisodeStore (internal/store/kdb_store.go) persists metadata over kdb-pgwire (SQL) + embeddingsANN in kdb-vector; [store] backend="kdb". Homologated against real Postgres 17 (faithful pgwire) + go-vector InMemory on dev-linux-id. Previous: #008 re-scoped + shipped 2026-05-11. ServiceEmbedder speaks the real `servicesaiembedv1embedtext HTTP API; [recall.backend] = "service" enables semantic recall; InlineEmbedder` remains as the deterministic test fallback.

Role in the stack

memory is the foundation for persistent agents. Without a memory service, every product reinvents continuity (Kode CURRENT.md, Kortex indexing, agent ad-hoc state) and lose context across sessions. This sector fills that gap.

It is the Koder analog of Mem0MemGPTLetta and Anthropic's Memory Tool — self-hosted, tenant-isolated, integrated with services/ai/embed for semantic recall and infra/data/kdb (pgwire SQL for metadata + kdb-vector for the ANN index) for storage.

Features (v1 target)

• Episodic write/read API
• Semantic recall (top-K + score)
• Tenant isolation (handler-enforced)
• Forgetting policies: TTL, decay, explicit erase, GDPR sweep
• Pluggable embedding backend (inline ONNX or services/ai/embed)

Primary couplings

RFC and bootstrap

• RFC: memory-RFC-001-foundations.kmd — accepted 2026-05-09
• Bootstrap ticket: services/ai/backlog/done/129-memory-bootstrap.md
• Implementation tickets:
  • done/: 001 (OpenAPI), 002 (skeleton), 003 (tenant scope + audit), 004 (embed integration), 005 (forgettingTTLdecayeraseGDPR), 006 (real JWT via engines/sdk/go/auth.JWKSValidator), 008 (re-scoped — ServiceEmbedder HTTP client to services/ai/embed shipped 2026-05-11; BGE local ONNX moved to #012), 009 (decay auto-prune + quarantine), 010 (bench harness inline+memory)
  • done/: 007 (kdb store swap — KDBStore over kdb-pgwire + kdb-vector, shipped 2026-06-22; design corrected from the non-existent "kdb-doc" to kdb-next pgwire per RFC-001)
  • pending/: 011 (bench at 100K against kdb+BGE — now unblocked: 007+008 done), 012 (BGE local ONNX embedder — split from #008, blocked on services/ai/embed#009 ONNX runtime), 013 (kdb store slice 2 — live kdb-gateway vector e2e + hyperscale TTL/decay SQL pushdown + per-request tenant)

Recent changes

• 2026-06-24 (memory#014 — DEPLOYED; real-embedding recall loop CLOSED end-to-end) — koder-memory is running in the fleet (LXC embed@10.0.1.146:18080, systemd, co-located with services/ai/embed): [store] backend="memory" (in-process) + [recall] backend="service" → the real embed service + Koder ID JWT auth. Proof: stored 3 pt-BR episodes, recalled q="encontro para discutir verba de publicidade" → top-1 = the "orçamento de marketing" episode, score 0.7399, semantically above the unrelated catdatabase episodes (threshold-filtered). The whole AImemory product now works end-to-end on real multilingual embeddings (/k-arch-chosen service path: memory→embed HTTP, not in-process). Deploy-as-code in backend/deploy/. Production gaps (memory#014): embed-token refresh (the ServiceEmbedder uses a static token that expires), persistent [store]="kdb" (needs a deployed kdb-gateway; the store is already bench-validated #007/#013), and the service-account-vs-user identity model. Caps the embed arc (#017→#021) — real embeddings produced, served, authed, cached, and consumed by memory.
• 2026-06-24 (kdb#808 Slice 2 — vector + meta cross-request cache; latency target CRUSHED) — Slice 1 proved adjacency wasn't the bottleneck; the surviving per-node read_vector + is_tombstoned sled reads were. Slice 2 caches both across requests. Design (Quality > Speed, DRY): generalised the #059 slab-LRU into ByteLru<K,V> + a Weighable trait — three caches (adjacency / vector / meta) on one proven machine; the 15 existing cache tests pass verbatim. PersistentHnswIndex gained vec_cache + meta_cache Arcs (warm via Slice 1's registry); read_vector is L1→L2→KV, is_tombstoned is metacache→KV; write-through invalidation on insertdeletewipe. Re-benched: 2K p50 109 → 6.7 ms (~16×); 100K acceptance p50 19.6 ms (5× under) / p99 45.0 ms (11× under) — RFC target (p50<100msp99<500ms) MET at both 2K and 100K, closing kdb#808 + memory#011. Throughput 9.7 → 133.8 qs (2K) / 47.6 qs (100K); populate 15 → 73 eps (2K), 28 ep/s at 100K scale. Homologated dev-linux-kdb: 91 kdb-vector lib tests (3 new cache + `slice2nodecachescoherentafterwarm`) + the gateway coherence regressions. #807 (filtered-HNSW) is now an optional compounding follow-up, not a blocker.
• 2026-06-24 (kdb#808 Slice 1 — warm-handle registry; perf reframe) — The kdb-gateway vector service was stateless per RPC, rebuilding a fresh PersistentHnswIndex (and a fresh empty adjacency cache) every call, so the #059 cache never survived a request. Slice 1 adds a gateway-resident IndexRegistry that keeps one handle warm per index identity (built once under a write lock → exactly one shared Arc<HnswCache> per key, so the existing write-through invalidation stays coherent). Homologated on dev-linux-kdb: 3 unit tests + a 2-case coherence regression (tests/regression/vector_warm_cache_coherence_808.rs — post-warm insertsdeletes must still be observed). Re-benched: p50 109 → 105.7 ms (only ≈3 %) — decisive finding that adjacency was NOT the bottleneck: at ef=128, ~256 per-node read_vector+is_tombstoned sled readsquery survive. Real latency levers are kdb#808 Slice 2 (cross-request vector+meta cache, enabled by this registry) + #807 (filtered-HNSW). Also found the bench SQL side can't run on kdb-pgwire (ANY($1::text[]) encode gap → kdb#809); uses real PG17.
• 2026-06-23 (#011 bench harness — kdb store benched; perf finding) — bench/main.go gained --store=memory|kdb + --embedder=inline|service + make bench-100k. Benching the real kdb-vector gateway (RELEASE, RTT 0.058 ms) at 2K gave p50 109 ms (✗ the < 100 ms RFC target). Finding: the #007 store swap is functionally correct + storage-scalable but latency-bound by per-request sled node reads — PersistentHnswIndex::search rebuilds its VectorCache per call (no cross-request cache), amplified by the #806 filter over-fetch (ef=128). The 100K acceptance run is blocked-by kdb#808 (in-memory index/node cache) + kdb#807 (filtered-HNSW). Baseline in perf-baseline.md.
• 2026-06-23 (#013 criterion 3 — per-request tenant; #013 COMPLETE) — One memory instance now serves many tenants. The authenticated tenant flows auth.Identity.TenantID → a server middleware (tenantContextMW) → store.ContextWithTenant(ctx) → KDBStore.tenantFor(ctx), scoping every SQL query (request path + sweeper scans) and falling back to the configured [store] tenant for non-request callers. Episode.Tenant is populated on read; ListTenants + the TenantLister capability let the TTLdecay sweepers iterate every tenant (no leak). The store imports no auth (the composition root bridges, avoiding an auth→handler→store cycle). Integration-tested vs real PG17 (cross-tenant GetSearch/Delete isolation; ListTenants). #013 complete (all 3 criteria). Next: #011 bench @100K.
• 2026-06-23 (#013 criterion 2 — TTL/decay SQL pushdown) — The TTL + decay sweepers now scan only candidates instead of pulling every tenant row each pass. KDBStore gained ListExpired (WHERE expires_at < now) and ListDecayCandidates (flagged OR GREATEST(last_accessed, created_at) < cutoff OR overridden-user — provably complete; the sweeper re-evaluates each exactly). The policy sweepers fast-path onto the optional ExpiredListerDecayCandidateLister seams (MemoryStore keeps the full-scan fallback). Removes the only app-side full-table scan (hyperscale-first). Integration-tested vs real PG17 (77). Remaining #013: per-request tenant threading.
• 2026-06-22 (#013 live kdb-gateway vector e2e — /k-go, slice 2 partial) — Ran the internal/store integration suite against a live kdb-gateway (vector gRPC 10.0.1.227:50061) for the first time, metadata→real PG17. The test is parameterized (MEMORY_KDB_TEST_VECTOR_GATEWAY → vector.Open, else InMemory). The live run surfaced + fixed a real gateway bug (kdb#806): vector_service.rs::nearest applied the metadata filter after the top-k truncation, so a usertenant-scoped Search returned fewer than k results when the globally-nearest vectors belonged to other users — recall silently degraded for every metadata-scoped vector search. Fix = oversample (k*10, min 128) before filtering, then truncate; Rust regression test + registry entry; HNSW predicate pushdown for highly-selective filters tracked as kdb#807. memory#013 e2e now *5 green*against the real gateway. Remaining #013: hyperscale TTL/decay SQL pushdown + per-request tenant.
• 2026-06-22 (#007 kdb store swap — /k-go memory, slice 1) — Shipped the production EpisodeStore backed by kdb-next (RFC-001 unified data plane). internal/store/kdb_store.go splits each episode: metadatacontentmutable lifecycle fields + point-getdelete-by-user → kdb-pgwire (`sdkgo-pgwire, pgx/v5) SQL table memory_episodes; embeddings + ANN → **kdb-vector** (sdkgo-vector), joined by ULID↔fnv64a(ULID)`. Design correction: the ticket named "kdb-doc" (no such component) — the canonical row store is kdb-next over pgwire. The vector store is ANN-only, which is why the two-backend split is required (it can't serve GetDeleteAllscan). Added ListAll so the TTL+decay sweepers keep working; [store] backend="kdb" config; buildStore wiring (vector.Open + Migrate on boot); `cmdkoder-memory-migrate one-shot loader. **Homologated on dev-linux-id (Go 1.25, PG17.10):** build+vet+unit green; integration suite green against **real Postgres 17 + go-vector InMemory** (round-trip+bump, user-scoping, search ranking/threshold/topK + no-recall-bump, delete idempotency, DeleteAll, ListAll). Found+fixed a latent compile break in sdk/go-pgwire (BulkLoader.source *pgx.CopyFromRows — a func used as a type in pgx v5; forensic kdb#805). Residuals: live kdb-gateway vector e2e + hyperscale TTL pushdown → #013; bench @100K → #011` (unblocked).
• 2026-05-11 (#008 re-scope + ship) — ServiceEmbedder rewritten from a 3-line stub returning ErrEmbedServiceUnavailable to a full HTTP client of services/ai/embed/v1/embed/text (~165 lines). Options pattern: WithEmbedderBearer, WithEmbedderIntent, WithEmbedderModel, WithEmbedderTimeout. Dim verification on every response prevents silent contract drift. [recall.backend] = "service" now consumes the new config: embed_url, embed_token, embed_model, embed_dim, embed_timeout_ms. 10 tests SE1-SE10 (happy, 503, 401, bearer, dim mismatch, count mismatch, empty endpoint, empty inputs, model override, timeout). InlineEmbedder retained as test/bench fallback. Closes the AI cross-sector loop: memory now joins cache #003 semantic + embed #007 HTTP cache as a consumer of an AI sibling sector. Original BGE-local-ONNX path split off as #012 (blocked-by embed #009).

Self-hosted-first analysis (5 gates)

Performance baseline

Latest bench numbers (and per-metric trend) live in registries/perf-baseline.md. Re-run via make bench-full from services/ai/memory/backend/.