Koder Konsul — `dev/eye`

On-device UI analysis bridge for Android. Replaces most screenshot-to-LLM round-trips (_{1 500 image tokens each) with a local JSON API (}200 tokens) that agents query via adb forward. From v0.2 agents can also inject gestures, making Eye a full inspect-and-drive bridge.

Role in the stack

Sits alongside dev/bridge and dev/kterm as a device-facing developer tool. Unlike those, Eye runs on the device — it is the on-device half of a split inspector.

Primary couplings

Interfaces

Local HTTP on 127.0.0.1:9876 (reach via adb forward tcp:9876 tcp:9876). All routes except /healthz require Authorization: Bearer <token>.

Observation

Capture & OCR

Injection (requires toggle ON in app)

Rate limiting

Token-bucket per-path rate limiter (RateLimiter.kt):

Exceeded requests get 429 Too Many Requests with Retry-After header and error ID EYE-RL-429-001.

gRPC transport (:9877)

Alongside the Ktor HTTP server, Eye runs a gRPC sibling on :9877 (EyeGrpcServer.kt, proto contracts in proto/koder/eye/v1/). All 13 RPCs across 5 services (HealthScreenActionEventMirror) are bound and delegate to the same in-process services the HTTP routes use (#033).

• Auth — bearer token via the gRPC authorization metadata key,

  mirroring HTTP Authorization: Bearer.

• Rate limiting (#041) — RateLimitInterceptor maps each unary RPC

  to its mirror HTTP path and shares the same RateLimiter buckets, so HTTP + gRPC draw from one combined budget. Healthz and the server-streaming RPCs are exempt; limit hit → RESOURCE_EXHAUSTED + retry-after-ms trailer.

• TLS symmetry (#042) — in LAN mode (#031) the gRPC server binds the

  same CertBootstrap cert as HTTPS, so /info.fingerprint_sha256 pins both channels; plaintext for adb forward.

• /info (#043) — exposes grpc_port / grpc_host / grpc_tls

  alongside the HTTP bind state.

Advanced config (in-app)

EyeConfig exposes four parameters via SharedPreferences, editable in the app's "Advanced" section:

EventLog.resize() is called live when event cap changes.

Host-side CLI: konsul

products/dev/konsul/cli/konsul — single-file Python 3 script, zero dependencies.

konsul healthz | text | buttons | screen | diff <hash>
konsul text --since-last           # diff vs cached last screen hash (TTL 5 min)
konsul windows | capture [file.png] | ocr [--region x,y,w,h]
konsul watch [--since N]           # SSE event stream
konsul tap <x> <y> | tap --id <view_id>
konsul scroll --dir down | type [--id <view_id>] <text>
konsul back | home | recents
konsul setup                        # adb forward + smoke test
konsul fanout <command>             # run read command on all ADB-connected devices
konsul token [--set <token>]
konsul --grpc <read-command>        # route over gRPC :9877 (needs grpcio)
konsul --https [--host <ip>] <cmd>  # TLS + self-signed cert pinning (LAN, #050)

The optional --grpc flag (#034) routes the cleanly-mapping read commands (healthzscreentextbuttonswindows/capture) over the gRPC server; grpcio is imported lazily only on that path so the default HTTP transport stays stdlib-only (zero-dep guarantee preserved). Other commands transparently fall back to HTTP. Vendored stubs in cli/proto/ (regenerate with make grpc-stubs).

--https (#050) pins the server's self-signed cert by fingerprint: a TOFU bootstrap fetches /info, reads fingerprint_sha256, asserts it matches the cert actually presented, and refuses any later connection whose cert SHA-256 differs (stdlib ssl+hmac, zero-dep preserved). Use it for LAN access (--host <device-ip>); the adb forward loopback default stays plaintext. The client pin is unit-tested device-free; the on-device negative test (a mismatched cert rejected against a live server) is gated on a device session and a working TLS server (#060, LAN-TLS engine swap).

Token saved to ~/.config/koder-konsul/token. Screen hash cache at ~/.cache/koder-konsul/<serial>/last-hash. Coloured table output by default; --json for raw.

Status

v0.2.1 (2026-04-23) — verified on Galaxy S26 Ultra (Android 16).

Autonomy layer (RFC-003) — from sensor to autonomous on-device agent

Iris is evolving from inspector to autonomous on-device agent (RFC-003, 2026-06). The layer is least-privilege by construction: a capability/consent spine gates every power — each capability is OFF by default, time-boxed (TTL), revocable, with an audit trail (the "god-mode" of unrestricted access + standing credential/geo aggregation was rejected by design).

• Injection default-OFF (#075): ActionService.injectionEnabled delegates to

  Capabilities.isGranted(INJECTION) — /tap,/type,… act only while granted (fixes the prior default-ON least-privilege gap).

• Interactive mirror (#077): the mirror page routes a browser click/wheel

  through /tap_at`scroll_at, which hit-test the a11y tree (nodeAtPoint → nearest clickable/scrollable ancestor) and fire ACTIONCLICK/ACTIONSCROLL — so clicking the espelho actuates a real widget even where coordinate gestures are no-ops (the S26). Emulator-proven end-to-end (/tap_at` flips a Compose Switch).

• First-run onboarding (#078): when the AccessibilityService is off, a

  prominent step-by-step card (incl. the sideload "Allow restricted settings" step) guides the user through the OS consent and stays until configured — the 1.5s status poll auto-dismisses it once Accessibility is on. Guides the OS gates rather than bypassing them (the least-privilege answer to "enable everything"); closes the silent-no-UI gap of #058. S26-verified live.

• Privilege tiers (RFC-003 §3): T0 node-action (today) → T2 privileged

  extension (hub#408, real INJECT_EVENTS — dispatchGesture is a no-op on the owner's S26, see #065) → elevated bridge (#107, done): Shizuku (Part A — shell/root uid via a bound UserService, no root / no device-owner, so viable on the daily S26 with no MDM footprint) and Device-Owner (Part B — T3, silent runtime-perm grant → POLICY_FIXED, emulator/dedicated-device only since banking/gov.br detect MDM). Neither grants signature INJECT_EVENTS (stays in hub#408/root) nor auto-enables Accessibility. T4 root = dedicated rooted Pixel (future, owner asset). Kodix-system-app (T5) still future/gated.

• Debug-bridge transport (#068): reuses Koder Remote (infra/net/remote

  relay-server) for NAT-traversing adb over Wi-Fi without manual IPportcode (decided via /k-arch); not a bespoke overlay.

• 29 unit tests across the 5 cores; all build-verified on the

  ci-runner-android Android build host (/opt/android-sdk, JDK 21).

• Naming: kept Koder Konsul (Íris = messenger goddess / longa manus + the

  iris as a light-regulating aperture = gated access) — RFC-003 §5.

Design references

• RFC-001 — On-device UI analysis bridge for AI agents
• RFC-003 — Koder Konsul: de sensor a agente autônomo on-device
• meta/docs/stack/specs/errors/user-facing-messages.kmd — all EYE-* / IRIS-* error IDs