AI Embed — Unified Embeddings Foundation

• Area: Intelligence
• Path: services/ai/embed
• Kind: Embedding generation foundation (text + image + audio; intent-based model selection)
• Status: v0.0.7 — #013 binary naming alignment shipped 2026-05-11. Daemon binary renamed kembed → koder-embed; worker kembed-worker → koder-embed-worker; CLI kembed-admin → kembed. Matches the cache precedent (koder-cache daemon + kcache CLI). systemd units updated; feature surface unchanged from v0.0.6. Previous v0.0.6: #007 HTTP cache adapter consuming services/ai/cache#008 KV API.

Role in the stack

embed consolidates embeddings server-side. Today every consumer ships its own model (kortex, search, recsys, agents) — no shared cache, inconsistent dimensions, no consolidated billing. Migrating to a better model means rewriting each consumer.

The headline design choice: intent-based selection. Consumers query by intent (text-match-search, text-classify, image-match) and the registry picks the model. Swapping a model is a registry change, not a consumer rewrite.

It is the Koder analog of OpenAI Embeddings, Voyage, Cohere Embed and Jina — self-hosted via BGE / E5 / CLIP / CLAP on GPU runtime, with proxy fallback to OpenAI / Voyage / Cohere through services/ai/gateway for capability gaps.

Boundary vs neighbors

• infra/data/kdb-vector is the vector store (search, ANN); embed is the producer.
• services/ai/cache is the embedding cache backend (keyed by model+input hash).
• services/ai/runtime hosts the model weights.

Features (v1 target)

• Text embeddings: BGE-multilingual default (pt+en), BGE-large-en opt-in, E5, Instructor
• Image embeddings: CLIP-ViT-L/14 + SigLIP for cross-modal
• Audio embeddings: CLAP for audio-text cross-modal
• Intent registry: 6 default intents, per-tenant override
• Batch coalescing: 10ms window OR 64-input cap for throughput
• Cache integration: hits don't decrement quota

Primary couplings

RFC and bootstrap

• RFC: embed-RFC-001-foundations.kmd — accepted 2026-05-09
• Bootstrap ticket: services/ai/backlog/done/121-embed-bootstrap.md
• API contract (normative): api-v1.kmd + intents.toml — shipped 2026-05-11 via #001 (5 endpoints, intent-over-model design, 7 registered models across textimageaudio, error map)
• Implementation tickets:
  • done/: 001 (API contract + intent registry), 002 (Go server skeleton + coalescer), 003 (quotas + cache + audit + admin API), 004 (split into #009-#012), 006 (kembed-admin CLI), 007 (HTTP cache adapter — consumes cache#008), 008 (superseded by #010), 010 (real tokenizers — WordPiece pure-Go), 013 (binary naming alignment) — all 2026-05-11
  • pending/: 005 (image+audio CLIP/CLAP), 009 (ONNX runtime + LocalONNXWorker — CGO), 011 (GPU LXC s.embed.gpu provisioning), 012 (weights + MTEB benchmark — blocked-by #009 + #011)

Recent changes

• 2026-06-24 (#021 — DEV deploy LIVE) — embed is deployed + running in the fleet. LXC embed @ 10.0.1.146 on s.khost1: koder-embed server (:18089) + koder-embed-worker (:18091, runtime=onnx:cgo, real multilingual XLM-R model + SentencePiece) under systemd; onnxruntime + model provisioned to /opt/onnxruntime + /var/lib/koder-embed/models/. Validated in-fleet: curl /v1/embed/text → HTTP 200, pt-BR cosine 0.9782/0.1152. Registered in koder-service-ports.toml. Public embed.koder.dev deliberately deferred: dev_tokens_enabled accepts any dev-<t>-<u> bearer → public exposure would be an open API (security.kmd); embed is internal (its consumer memory reaches 10.0.1.146:18089 internally), so the high-blast-radius Jet edit + DNS wait on prod JWKS auth + a real external consumer. Reproducible recipe in backend/deploy/ (provision-onnx.sh + units + config.example.toml). Remaining (#021): prod auth, then public proxy + memory prod config + services/ai/cache.
• 2026-06-24 (service-path e2e + /k-arch) — Production topology validated: memory → embed service → real embeddings. /k-arch ratified the service path over an in-process embedder: a memory-local ONNX model is the exact anti-pattern embed exists to eliminate (no shared cache, inconsistent dims, no consolidated billing, model-swap = consumer rewrite). services/ai/memory#012 (in-process) re-gated owner-decision; the offline case is a koder-embed-worker sidecar, not a fork. End-to-end proof (off-laptop dev-linux-id): the real memory.recall.ServiceEmbedder → live koder-embed /v1/embed/text (Bearer dev-<tenant>-<user>, {inputs,intent}) → HTTP → koder-embed-worker (onnxcgo) → SentencePiece + real ONNX (paraphrase-multilingual-MiniLM, mean) → pt-BR cosine(paraphrase)0.9782 / unrelated0.1152, identical to the direct worker test (the HTTP wire preserves the vectors). Two processes: server (plain build) + worker (-tags onnxcgo + libonnxruntime.so), wired via [[workers]][[onnx_models]][[tokenizers]] + a [registry] intents map. Remaining = #021 (real deployment to devprd + prod auth + point memory at the endpoint + `servicesai/cache`). +1 live regression test (env-gated).
• 2026-06-24 (#019 ship) — SentencePiece Unigram tokenizer (pure Go) → real multilingual (pt-BR) embeddings. The worker was WordPiece-only (BERTBGE-en); the multilingual embedders Koder products need (bge-m3, multilingual-E5, paraphrase-multilingual-MiniLM) all use SentencePiece Unigram (XLM-R, 250k vocab). `internaltokenizersentencepiece.go (FromSentencePieceJSON, rejects non-Unigram) loads a HF tokenizer.json and runs the XLM-R pipeline: **NFKC** normalize (approximation of the reference "Precompiled" charmap — byte-exact deferred to **#020**) → **WhitespaceSplit + Metaspace** (▁ per word) → **Unigram Viterbi** (DP maximizing summed log-prob scores; unmatched runes → unk) → s … <s> specials (default, WithSPSpecialTokens(false) opt-out). Wired into worker.LoadTokenizers as kind = "sentencepiece" (vocab_path → tokenizer.json). **Verified off-laptop (dev-linux-id):** unit (TestSentencePieceUnigramViterbiAndSpecials + non-Unigram rejection) + go vet green; onnxcgo e2e with real **paraphrase-multilingual-MiniLM-L12-v2** (mean pooling) on **pt-BR** through worker.Embed → **cosine(paraphrase)=0.9782 / unrelated=0.1152** — excellent semantic separation, the NFKC approximation is more than sufficient. +2 tests. Unblocks quality multilingual embeddings via the embed service + clears memory#012`'s SentencePiece blocker. Follow-on #020 (byte-exact normalizer parity).
• 2026-06-24 (#018 ship) — koder-embed-worker serves real embeddings on CPU (wires #017's Session into the worker). New [[onnx_models]] config (config.OnnxModelEntry: model_path, pooling, intra_op_threads) — deployment binding, mirrors [[tokenizers]] (the path is infra, not public catalog). buildBackend builds map[id]worker.ONNXModelConfig and passes worker.NewONNXAdapterFactory(map) to NewLocalONNXWorker; the factory resolves per-model model_path + pooling (poolKindFor: "cls"→PoolCLS), and under the real build (onnx.BuildHasONNX) a model with no entry is a hard error (modelpath required — mirrors the strict-tokenizer contract). *wo correctness gaps found+fixed that a naive "pass modelpath" would have shipped broken: (A) the encodeTokenizer bridge fed raw WordPiece ids with no [CLS][SEP]*→ CLS pooling read the wrong vector at position 0; fixed at root — WordPiece.Encode now wraps [CLS] … [SEP] by default (HF add_special_tokens=True, ids from vocab, WithSpecialTokens(false) opt-out); (B) BERTBGE graphs require token_type_ids → Run errored "Missing Input"; GenericPoolingAdapter now feeds all-zero token_type_ids (the Session forwards only graph inputs it has, harmless for E5). Build: the worker built -tags onnxcgo needs libonnxruntime.so at runtime (CGO_CFLAGS=-I<ort>/include CGO_LDFLAGS="-L<ort>/lib -lonnxruntime" LD_LIBRARY_PATH=<ort>/lib; onnxruntime-linux-x64-1.20.1 prebuilt). Verified off-laptop (dev-linux-id): defaultmock + go vet green; onnxcgo + real bge-small-en-v1.5 — TestLocalONNXWorkerRealBGEEmbeddings paraphrase 0.8455 / unrelated 0.3924 through worker.Embed, matching the raw #017 Session exactly (full tokenizer→factory→Session→adapter chain). +3 tests (2 mock unit + 1 onnxcgo e2e). Follow-on: real models.toml in the deployISO pipeline, multilingual SentencePiece, memory#012 consumer, CUDA EP (#011).
• 2026-06-24 (#017 ship) — Real onnxruntime CGO Session on CPU — the runtime is no longer mock-only. #009 shipped just the onnx.Session interface + pure-Go MockSession + a skeleton loader_cgo.go returning ErrNotImplemented; the real CGO bindings were deferred "to land with #011 (GPU LXC)". That conflated two independent axes — CPU small-model inference (bge-small dim384; the literal memory#012 target "batch=32 <200ms on CPU"; platform linux x86-64 already fixed) vs GPU large-model serving (BGE-large p50<50ms; needs #011 + CUDA EP). #017 implements the CPU slice (the onnx.Session interface already abstracts the device → CUDA later is an EP swap, not a rewrite; stays sequenced with #011). loader_cgo.go (//go:build onnxcgo) wraps the onnxruntime C API (OrtApi function-pointer table) via C-preamble helpers: ort_open (env + session opts + CPU mem, intra-op threads, graph-opt ALL, all OrtStatus checked), graph name introspection, and a stateful RunCtx that copies int64 input buffers (CreateTensorWithData doesn't copy → copy outlives Run) → Run → float32 outputs. realSession implements Session (LoadRunClose, mutex + finalizer); scope v1 = int64 inputs / float32 outputs (the text-path dtypes). Default !onnxcgo build unchanged (mock) → normal CI needs no C library. Validated off-laptop (dev-linux-id, onnxruntime 1.20.1 CPU, real BAAI/bge-small-en-v1.5): tokenize via WordPiece, CLS-pool + L2-norm → paraphrase cosine 0.8455 ranks far above unrelated 0.3924 (>0.7) — the runtime is real. Unblocks the whole Stack's real-embedding capability (incl. memory#012). Follow-ons: #018 (wire real Session into LocalONNXWorker + real-model config), SentencePiece for multilingual (XLM-R), memory#012 consumer, CUDA EP (#011). +1 real-model regression test (build-tag + env gated).
• 2026-06-02 (#015 ship) — Real per-model tokenizers wired into the ONNX worker. worker.LoadTokenizers(cfg.Tokenizers) builds the text.Tokenizer set from [[tokenizers]] via an encodeTokenizer bridge (internal/tokenizer.Tokenizer single-text Encode→[]int → batch input_ids+attention_mask for the adapter, padding handled by GenericPoolingAdapter). Strict: a wordpiece entry with a missing vocab is a startup error (tokenizer: <id> not found at <path>), not a silent passthrough — passthrough now only backs models with no entry (stubmock). Wired into buildBackend's "onnx" path, replacing the empty placeholder. Only WordPiece (vocab.txt) is wired — SentencePieceBPE (tokenizer.json) still need those tokenizers added to internal/tokenizer. #016 (Instructor prefix) is blocked on a registry-schema gap (no Model.Family/"instructor" source) — deferred. +4 tests.
• 2026-05-11 (#013 ship) — Binary naming aligned with cache precedent. Daemon binary: kembed → koder-embed (matches cmd/koder-embed/ + systemd unit). Worker: kembed-worker → koder-embed-worker. CLI: kembed-admin → kembed (now matches the binaries-and-cli/naming.kmd "binary = aliases[0] || slug" rule). systemd units' ExecStart paths updated to /usr/local/bin/koder-embed[-worker]. Zero behaviour change (pure rename); 65+ tests stay green. Breaking for any operator running pre-v0.0.7 packages on the same host — /usr/local/bin/kembed now refers to the CLI, not the daemon.
• 2026-05-11 (#007 ship) — HTTP cache adapter shipped. internal/cache/http_adapter.HTTPCache consumes services/ai/cache#008 public KV API (PUT/GET/DELETE /v1/cache/kv/{tenant}/{key}) with circuit breaker (5 consecutive 5xx/network errors → open for 30s, single probe re-closes on success), per-call timeout (default 1s), TTL via ?ttl_seconds= (default 30 days, clamped server-side), defensive empty-BaseURL → NoOp. Wire format JSON {vector: [...], usage: {text_tokens, image_count, audio_seconds}}. 9 tests (HC1-HC9) including bearer header propagation, URL-encoding of |-keys, 404=clean-miss (no trip), trip+cooldown reopen, local Stats accumulation. [cache.backend] = "http" selects it; [cache.http] config section in TOML.
• 2026-05-11 (#006 ship + #007 blocker) — kembed-admin CLI shipped (cmd/kembed/, bin/kembed-admin). Subcommands stats, quotas {list,set,delete}, models. Mirrors kcache pattern but binary named kembed-admin (not kembed) to avoid clash with the daemon binary kembed from #002 — naming alignment tracked as #013. Cross-sector discovery: #007 (HTTP cache adapter) was blocked because services/ai/cache exposes no public KV PUTGET — only admin + middleware Wrap. New ticket `servicesai/cache#008 (public KV HTTP API) opened; embed #007` re-stated as blocked-by until that lands.
• 2026-05-11 (#004 split + #010 ship) — #004 was bundling code + tokenizers + GPU LXC + weights + benchmarks; split into 4 sub-tickets (#009 ONNX runtime + LocalONNXWorker (CGO), #010 real tokenizers (this commit), #011 GPU LXC s.embed.gpu provisioning, #012 weights + MTEB benchmark). #010 shipped: internal/tokenizer/ with BERT-canonical WordPiece in pure Go (greedy longest-prefix, ## continuations, CJK rune-per-piece, lowercase toggle, max-chars fallback), Whitespace fallback, Registry modelID→Tokenizer index, [[tokenizers]] TOML config, handler swaps approxTokens → tokenizerFor(model.ID).Count. Quota billing now charges in target-model tokens. SentencePiece deferred to #009/#012. #008 closed as superseded-by #010. +13 tests (T1-T12 unit + I11 integration).
• 2026-05-11 (#003) — Multi-tenant quotas + per-input cache + audit. internal/quota.Enforcer with day-rolling UTC counters for text-tokens / image-count / audio-seconds, per-tenant override that shadows the default, AllowedModels + BatchMaxSize checks, ReserveRefund clamp-at-zero. `internalcache with NoOp + Memory (LRU bound, tenant-scoped key tuple). internalaudit slog wrapper. Admin API GETPOSTDELETE /v1embedquotas[{tenant}] + GET v1embed/stats (admin-gated). Cache hits skip quota by construction (handler reserves only the miss bundle). Estimated-vs-actual reconciliation refunds overshoots post-dispatch. +26 tests (Q1-Q10, C1-C6, I1-I10). TTL deferred to #007` HTTP cache adapter.
• 2026-05-11 (#002) — Go server skeleton shipped (backend/cmd/koder-embed + backend/cmd/koder-embed-worker). Wire format: HTTPJSON server↔worker (`POST /v1workerembed); gRPC deferred. Coalescer: per-(modality, model_id) queue, 10 ms window OR 64-input cap. Stub worker uses xxh3-seeded deterministic vectors (real ONNX in #004`#005). Auth + identity context split into internal/identctx/ so handlers can read the principal without an auth↔handler import cycle. 24 tests pass (R1-R7, C1-C7, H1-H8, S1-S2).

Self-hosted-first analysis (5 gates)