niklas‑stephan.de

Outline semantic search: LIVE on handtrixxx.com (sidecar, Redis, local embeddings, MCP…

Last updated 06/11/2026 · outline, memory, mcp, vector-search, embeddings

id: outline-semantic-search-prototype title: Outline semantic search: LIVE on handtrixxx.com (sidecar, Redis, local embeddings, MCP… importance: high tags: outline, memory, mcp, vector-search, embeddings updated: 2026-10-31 content: Live since 2026-10-31: semantic_search MCP over Outline at docs.niklas-stephan.de/semantic/mcp; deployment, keys, client config, ops, measurements, open items. outline: /doc/outline-semantic-search-live-on-handtrixxxcom-sidecar-redis-local-embeddings-mcp-4fceWizFwt

URL moved 2026-10-31: the endpoint is now https://docs.niklas-stephan.de/semantic/mcp (same host as Outline). Traefik router semantic = Host docs + PathPrefix /semantic, priority 100 (Outline’s router has no explicit priority, so Traefik uses its rule length of about 31: with priority 20 the path returned Outline’s 404), tls.certresolver=letsencrypt-tls like Outline, ALLOWED_HOSTS lists both hosts. The old https://ai.handtrixxx.com/semantic/mcp (router semantic-legacy) is a temporary alias for clients not yet switched (ThinkPad, MacBook); when they are, delete the semantic-legacy labels and ai.handtrixxx.com from ALLOWED_HOSTS in /data/docker/docs.niklas-stephan.de/compose.yml. Mentions of ai.handtrixxx.com/semantic below refer to this endpoint under its old name. Switched: pi mcp.json (verified with pi mcp list), kb templates, bin/kb.

Update 2026-10-31 (cutover): the index is now the recall layer of the live memory store (Outline KB). The sidecar also parses each document’s header block (kb-id, importance, tags, summary) and serves a third MCP tool list_memories(min_importance="high", tag="") (start-here set: 29 high/critical of 79). Header meta is stored in the per-document Redis meta hash and backfilled for unchanged documents on the next sync (poll every 60 s). The shell CLI /data/ai-persistence/bin/kb (search / list) calls these tools over HTTP, so agents without MCP use the same index. Rebuild after code changes: cd /data/docker/docs.niklas-stephan.de && docker compose build semantic && docker compose up -d semantic.

Status 2026-10-31: LIVE on handtrixxx.com (approved by Niklas). Part of the **docs.niklas-stephan.de** compose project (/data/docker/docs.niklas-stephan.de/, moved there from the ai.handtrixxx.com project on 2026-10-31; the public host name is unchanged), source in ./semantic/ (Python, mcp 2.3.0, README inside; scratch copy was /data/tmp/outline-semantic). Goal: one per-user store every agent (Zed, pi, VS Code) reads and writes, with no self-maintained tool; Outline is the store (any article size), the sidecar only adds a derived semantic index.

Status 2026-11-06 (Outline archive executed): the sidecar now indexes Wiki.js only — OUTLINE_URL/OUTLINE_API_TOKEN removed from compose, OUTLINE_KEY dropped from the gateway, Outline’s KB collection renamed ARCHIVE: KB (moved to Wiki.js) and left untouched in Outline. Index: 65 Wiki.js docs / ~552 chunks. Everything below is the historical deployment record; the ops notes still apply to the sidecar.

Deployment (verified 2026-10-31)

  • Services in compose.yml (backup before the move: compose.yml.bak-premove; the ai.handtrixxx.com project no longer has any of this): semantic (image outline-semantic:1, runs as uid 1000 because semantic/tokens.txt is mode 600), llama-embed (ghcr.io/ggml-org/llama.cpp:server, model ./semantic/models/embeddinggemma-300M-Q8_0.gguf, -c 2048 -b 1024 -ub 1024 -t 4), semantic-redis (redis:8, AOF, volume semantic-redis-data). Redis and llama-embed are only on the internal network semantic; semantic is also on caddy. OmniRoute was not touched or restarted (docker compose up -d --build semantic llama-embed semantic-redis).
  • Endpoint **https://docs.niklas-stephan.de/semantic/mcp**: Traefik router priority 20, no authentik (like /v1), Bearer token from semantic/tokens.txt; 401 without or with a wrong token, 421 for a foreign Host. /semantic/healthz is deliberately not reachable publicly (internal healthcheck only).
  • Reaching Outline internally: http://outline-app:80 (not 3000) with header **X-Forwarded-Proto: https** (Outline has FORCE_HTTPS=true; without the header it answers 405). Already set in app/clients.py.
  • Secrets (names only): OUTLINE_KEY (Outline API key of Niklas’s admin user, unscoped read+write, created by Niklas) and SEMANTIC_MCP_TOKEN (agent bearer token, generated) in /data/ai-persistence/.env; the sidecar reads it as OUTLINE_API_TOKEN from ./semantic/.env (own file, mode 600; Outline’s ./.env feeds every Outline container, so it is not used); semantic/tokens.txt has the bearer token(s), hot-reloaded (add a line to authorise another client, no restart).
  • Indexed collections INDEX_COLLECTIONS=KB,Projects,blog,wiki,Welcome (changed from AI to KB on 2026-10-31 after the migration, because 26 of the 30 AI docs are stale copies of memories; the AI docs remain in Outline, just not indexed); **Private** is deliberately not indexed (empty value = all). Result after first sync (old set): 108 docs, 2026 chunks, about 7 minutes on CPU; with KB it is 2431 chunks, re-embedded in about 5 minutes; unchanged-wiki sync is instant. POLL_SECONDS=60.
  • Write path verified: agents write through Outline’s own MCP (https://docs.niklas-stephan.de/mcp, Authorization: Bearer $OUTLINE_KEY, tools incl. create_document, update_document, delete_document, fetch, list_documents) - retired 2026-11-06, the gateway no longer federates it; write through the wiki MCP instead. A note created that way was searchable via semantic_search about 50 s later; after deletion it left the index after about 60 s.
  • OmniRoute cannot proxy MCP (re-checked in 3.8.52: it is an MCP server with its own 110 tools, no upstream/client code). So every agent has two MCP entries: Outline (read/write/fetch) and semantic-search.

Client config (all three verified 2026-10-31)

  • Zed: template “/data/ai-persistence/config/zed-settings.json (context_servers outline+semantic-search, read tools allow, write tools confirm; the old Alexandria memory` entry was removed). Format checked against zed.dev/docs/ai/mcp; headers stay plaintext in the local settings.json.
  • VS Code: template “/data/ai-persistence/config/vscode-mcp.json(user profilemcp.json, top-level servers, type: http, ${input:…}password prompts so no secret is in the file; docsdocs/agents/reference/mcp-configuration.md` of microsoft/vscode-docs). Not run on a real VS Code (none on this host).
  • pi (always-on agent, agent.handtrixxx.com): /data/docker/agent.handtrixxx.com/pi-home/agent/mcp.json (mode 600, backup .bak-presemantic) has outline and semantic-search next to the disabled alexandria. Verified with **pi mcp list** inside **pi-web-ui**: outline connected 20 tools, semantic-search connected 2 tools. Changes outside a session need /reload (pi docs docs/mcp.md; PI_CODING_AGENT_DIR=/home/node/.pi/agent, CLI node /app/node_modules/@earendil-works/pi-coding-agent/dist/bundle/cli.js).
  • opencode was removed on 2026-10-16, no config.

Operate

  • Logs docker logs semantic; status via the MCP tool semantic_status; end-to-end eval semantic/eval/e2e.py (needs MCP_URL, MCP_TOKEN).
  • Index is derived: to rebuild, bump SCHEMA_VERSION in app/config.py, or delete the volume semantic-redis-data; it re-embeds everything on start.
  • The tokens file and .env files must stay mode 600; the container user must be able to read the tokens file.

Open items

  • ThinkPad: DONE 2026-11-01 - OUTLINE_KEY and SEMANTIC_MCP_TOKEN are in /home/handtrixxx/code/ai-persistence/.env and the repo is synced to HEAD (commit 695e721). Keys present on metis, iris, hestia and handtrixxx.com too (verified: Outline API 200, semantic MCP initialize 200).
  • Outline API keys (2026-10-31): the key omni (last4 2fSF) is ours (OUTLINE_KEY). blog (sKmg) and node-red (blXE) were revoked (idle since Feb/Mar, no references on any host). **gitline** (**fEOJ**) is in use: it is the **OUTLINE_API_TOKEN** of **/data/docker/niklas-stephan.de/.env**; do not revoke. An API key cannot delete API keys (403); revocation = soft delete in Postgres (update "apiKeys" set "deletedAt"=now() ..., reversible). hermes, mcp and node were already revoked earlier.
  • Migration of the memories and the cutover: see outline-kb-migration-2026-10.
  • Quality: with 108 docs hit@1 18/22, hit@3 20/22, hit@5 22/22 (22 questions written for the old AI collection), about 48 ms per query via the public URL.

Gotcha: stale Zed session -> “Context server request timeout” (verified 2026-10-31)

Zed’s MCP context client caches one session ID per server for the whole thread. When that session is GC’d server-side, Zed keeps POSTing with the dead Mcp-Session-Id; the sidecar answers 404 Rejected request with unknown or expired session ID, and Zed surfaces this to the agent as “Context server request timeout” (the server itself replies in ~30 ms). Diagnosis: raw curl (initialize -> tools/call with a fresh session) works instantly while every Zed call to semantic_search / list_memories / semantic_status times out; the container logs show repeated Rejected request with unknown or expired session ID lines. Fix/workaround: reconnect the context server (restart Zed / re-add the semantic MCP entry) so it re-initializes; the kb CLI and raw curl are unaffected. mcp 2.3.0 default session_idle_timeout is 1800 s; the dead session in the logs died well before that (session GC, same class of issue as the old keepalive_interval 120 fix).

FIX 2026-11-01: sidecar runs stateless (no more stale-session timeouts)

The gotcha above is fixed at the server: semantic/app/server.py now calls mcp.streamable_http_app(..., stateless_http=True, ...) (mcp 2.3.0 option). No Mcp-Session-Id is tracked any more, so an old session ID from Zed is simply accepted (verified: tools/call with a made-up session ID returns 200 in 0.19 s; before it was 404 -> Zed “Context server request timeout”). Rebuilt with docker compose build semantic && docker compose up -d --no-deps semantic in /data/docker/docs.niklas-stephan.de; index (Redis) untouched. Reconnecting the context server is no longer needed. If it ever fails again, check docker logs semantic for 404 session lines first; revert = remove stateless_http=True.

Source: Wiki.js