niklas‑stephan.de

Homelab backlog / TODO (single list)

Last updated 05/11/2026 · migrated-from-outline, backlog

One backlog for the whole lab (handtrixxx.com, metis, iris, hestia). The old open-todos list was merged into this document on 2026-11-01 — check here, nowhere else. Done items are removed (Outline revision history is the log).

Gateway & agent rollout (2026-10-31 cutover, see ai-gateway-omniroute)

  • Finish the OmniRoute key rollout: the old LITELLM_KEY_* are dead. Niklas must paste OMNIROUTE_KEY_ZED_MACBOOK / OMNIROUTE_KEY_ZED_THINKPAD into Zed (Settings → AI → LLM Providers → Metis) on the MacBook and ThinkPad, and add the OMNIROUTE_KEY_* lines to /home/handtrixxx/code/ai-persistence/.env on the ThinkPad (metis, iris, handtrixxx.com, hestia already done). After a few days read headroom-omni /stats and compare the saving with the old LiteLLM-path 18.9 %; the ai-omni DNS record is unused; docker image rm the old docker.litellm.ai/berriai/litellm and postgres:16 (+ delete ai.handtrixxx.com/db, litellm-retired/) once rollback is no longer wanted.
  • Paste the Zed RustySqueezer compaction settings on the MacBook and ThinkPad (rustysqueezer model entry + agent.compaction_model, templates in config/zed-settings*.json), then check docker logs rustysqueezer for [usage] after the first real /compact.
  • OmniRoute dashboard password login is disabled (OIDC only) → admin-API scripts need a browser auth_token cookie. (Compression decision is closed: Headroom stays, OmniRoute compression off.)

Security & hardening (handtrixxx.com)

  • fail2ban or CrowdSec (requested 2026-10-09). Neither installed; exposed: SSH 2222, container SSH 2223, SMTP/IMAP, 80/443. Hetzner Robot firewall limits ports but doesn’t block brute-force. First confirm SSH is key-only (PasswordAuthentication no). Iris/metis only if ever exposed (currently LAN/tailnet only).
  • Pin **:latest** image tags for tailscale, headscale, gluetun, socat, adguard (Traefik pinned to v3.7). Version jumps change behaviour (e.g. metis nftables detection).
  • Weak **nc -z** health checks remain on handtrixxx.com forwarders (forward-llama, forward-homeassistant, forward-musicassistant); iris ones use real wget checks (see headscale-tailscale-reference).

Monitoring & ops

  • Monitoring + alerting (deferred 2026-10-09). Nothing like Uptime Kuma/healthchecks/netdata runs anywhere; the 2026-10-08 tailnet outage went unnoticed. Idea: Uptime Kuma on iris checking iris forwarder ports, home./music.handtrixxx.com, tailnet IPs .1/.2/.3; alert via mail/ntfy.
  • Watch backup status: /data/ai-persistence/bin/backup-status (per host) or /data/ai-persistence/bin/backup-overview (global, live ETA). The Backrest UI does not show logs for a running job. Verify every plan has a first snapshot (handtrixxx media needs ~2 days over the home uplink).
  • Hestia disk monitoring: SMART works on the Pi (smartctl -d sat -a /dev/sda); add a periodic check/alert (smartd or cron), plus a free-space alert on /srv/backup.
  • Reboot test of all three servers (handtrixxx.com, metis, iris), then verify tailnet (headscale nodes list), forwarders healthy, Jellyfin reachable. Hestia reboot tested 2026-10-21 (mount race fixed, see pi-backup-server).
  • Arcane automation decision: autoUpdate, imageEventWatcherEnabled, autoHealEnabled, imageAutoPatchEnabled are all false on all four environments; Arcane only mails update reports. Enabling redeploy/auto-heal needs Niklas’ explicit ok.
  • Gitea OCI registry for self-built images (approved 2026-10-26, not done): dedicated Gitea bot account + token → GITEA_TOKEN in /data/ai-persistence/.env (Niklas to create, e.g. arcane-bot). Then push the local-only images (headroom-proxy:0.41.0, pi-web-ui:1, pihandtrixxxcom-pi-agent:latest, llamahandtrixxxcom-llama:latest) and register in Arcane via POST /api/container-registries with registryType: "generic" (type cannot be changed later; Arcane currently has zero registries).
  • Confirm test mail arrived in mail@niklas-stephan.de (sent 2026-10-26 via Arcane env 0; only Niklas can verify — agent key cannot read that mailbox).
  • Vaultwarden: recurring outbound certificate expired (~231 days) — target unidentified; low priority.
  • Frigate camera “CNA” (iris): ffmpeg “Unable to read frames” ~19x/day — camera flaky/offline? Check camera/power/wifi.

Home Assistant (iris)

  • HA llama conversation entry: still points at 127.0.0.1 and is unavailable; repoint to http://192.168.68.50:8000/v1 (llama is on metis, listens on 0.0.0.0:8000). Wyoming voice stack moved to iris 2026-10-23 and works.
  • HA Assist pipeline (German): local LLM agent + “Prefer handling commands locally”, faster-whisper de, piper de_DE-thorsten-medium; wake word empty until a voice satellite exists. Wyoming host must be exactly 127.0.0.1 (HA host network). Expose only 10–30 entities to the model.
  • Voice satellite hardware (HA Voice PE, ESP32-S3-BOX-3, or Android Companion App as free test) + openWakeWord wiring.
  • Delete the stopped HA leftovers on metis (/data/docker/HomeAssistant/, config-backup-20261021-pre-iris.tar.gz, orphan home-assistant:stable image) after a safe period; never start a second HA instance. Decide the metis Backrest plan scope (it still backs up the old HA copy; iris’s iris-data covers the live config).
  • go2rtc ports on iris: HA’s go2rtc wants 8554/8555, which clash with Frigate’s published ports; watch the log. Check the HA companion apps’ internal URL (now http://192.168.68.63:8123) and Apple Home HomeKit bridges after the move.

Misc

  • Searxng engines (DuckDuckGo captcha, Brave rate limit) — optional tuning.
  • Metis **/etc/resolv.conf** is static (NM resolvconf hook fails); consider resolvconf/systemd-resolved or NM dns=default + rc-manager=file.
  • Optional: benchmark one fast MoE (Gemma 4 26B-A4B QAT or Qwen3.6-35B-A3B) via llama-swap as a swap-in; cannot run concurrently with the prod model (~44–48 GiB GTT).
  • Optional: hestia keys/ssh alias on iris and handtrixxx.com, clone /data/ai-persistence on hestia only if an agent session will run there.
  • Low priority: lightweight messenger/phone access to an agent (WhatsApp bridge removed with Hermes 2026-10-15; a Telegram bot would be the alternative).

Dead ends — do not pursue

  • Alexandria memory MCP (mcp.handtrixxx.com, memory-system) and its roadmap slices: discontinued and torn down on purpose 2026-10-25; Outline KB is the memory store (memory-system). The repo survives on GitHub only if work ever resumes.
  • The git-era memories/ directory, kb-sync cron and OpenClaw wiring: removed 2026-11-01/2026-10-23; history in unified-memory-kb-architecture and outline-kb-migration-2026-10 (archived).
  • Headscale DB backup as standalone cron (handled by the backup project); automatic TS_AUTHKEY drift check (declined 2026-10-09); IPv6 forwarding on handtrixxx.com (dropped).

Done (condensed; details in topic memories)

Unattended upgrades on all servers; iris forwarder health checks; cdfox removed from Traefik; gluetun IP pinned; metis tailscale nftables (headscale-tailscale-reference); Hermes + OpenCode agents removed (2026-10-15/16), Zed-direct is the agent; urbackup retired 2026-10-19 (Backrest + restic on hestia, decided 2026-10-19); HA moved metis→iris 2026-10-21; OmniRoute replaced LiteLLM 2026-10-31.

Source: Wiki.js