diff --git a/.claude/agents/documentation-keeper.md b/.claude/agents/documentation-keeper.md index ad1e086..6130dc7 100644 --- a/.claude/agents/documentation-keeper.md +++ b/.claude/agents/documentation-keeper.md @@ -142,6 +142,7 @@ When creating documentation: 2. Full content in markdown format 3. Mermaid diagrams embedded within the documentation 4. Summary of what was documented and any follow-up actions needed +5. Update the change logs When providing information to other agents: 1. The specific information requested diff --git a/.claude/settings.local.json b/.claude/settings.local.json index 4cad768..a7f5fa1 100644 --- a/.claude/settings.local.json +++ b/.claude/settings.local.json @@ -36,7 +36,8 @@ "Bash(docker version:*)", "Bash(/tmp/mcpo-config.json << 'EOF'\n{\n \"servers\": []\n}\nEOF)", "Bash(/tmp/mcpo-config.json << 'EOF'\n{\n \"mcpServers\": {}\n}\nEOF)", - "Bash(/tmp/mcpo-config.json << 'EOF'\n{\n \"mcpServers\": {\n \"example\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-everything\"],\n \"env\": {}\n }\n }\n}\nEOF)" + "Bash(/tmp/mcpo-config.json << 'EOF'\n{\n \"mcpServers\": {\n \"example\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-everything\"],\n \"env\": {}\n }\n }\n}\nEOF)", + "Bash(sleep 20 && sshpass -p '&Rip8a115' ssh -o StrictHostKeyChecking=no glyptic@192.168.86.100 \"docker ps --filter name=n8n --format 'table {{.Names}}\\\\t{{.Status}}' && echo '---' && docker logs n8n --tail 15 2>&1\")" ] } } diff --git a/CLAUDE.md b/CLAUDE.md index 961e1de..3aa9c4a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -36,6 +36,7 @@ hf-pdocker-01 (23GB RAM, i7-7700T, NVMe, HD 630 GPU) -- RESTORED (2026-03-08, ha ├── [COMPOSE] Farm Manager API (:8888) -- dashboard + API server ├── [COMPOSE] Farm Agent (:8889) ├── [COMPOSE] Rongorongo decipherment engine -- 2 containers (rongorongo-web :8181, rongorongo-api). Server-synced state at /mnt/docker-data/configs/rongorongo/data. Traefik basicAuth on rongorongo.hobbs.farm. +├── [COMPOSE] LinkedStorm (:8300 pipeline, :8310 api, cron no-port) -- whole-home audio assistant. VAD+Whisper STT+speaker-ID. Deployed 2026-05-28. Public: linkedstorm.hobbs.farm -> :8310. ├── [REMOVED] VoiceCoach -- moved to GPU machine (192.168.86.150) 2026-04-04 ├── [COMPOSE] Node-exporter (:9100) -- standalone with pid:host ├── [COMPOSE] Telegraf -- metrics collection to InfluxDB @@ -58,6 +59,7 @@ hf-pdocker-02 (7.6GB RAM, i7-6700T, HDD) ├── [COMPOSE] Wiki.js + wikijs-db (PostgreSQL) ├── [COMPOSE] Vaultwarden, Actual Budget, MCPO, Trivy ├── [COMPOSE] n8n + n8n-db (PostgreSQL) -- workflow automation +├── [COMPOSE] LifeOS (:8095) -- personal life/goals/habits tracker (PWA). Local mode: Flask+SQLite, no Supabase. LAN-only (no auth in backend). AI companion -> Ollama on .150. ├── [COMPOSE] Farm Agent (:8889) ├── [COMPOSE] Node-exporter (:9100) -- standalone with pid:host ├── [COMPOSE] Telegraf -- metrics collection to InfluxDB @@ -140,8 +142,9 @@ compose/ # Standalone Compose definitions (deployed per-node) - Home Assistant: `yoda.hobbs.farm` - Vaultwarden: `vault.hobbs.farm` - Gitea: `git.hobbs.farm` -- VoiceCoach: `coach.hobbs.farm` (LiveKit WebRTC: `voice.hobbs.farm`) +- FRIDAY Voice: `coach.hobbs.farm` (replaced VoiceCoach 2026-07-05; runs on .150 :8320) - Rongorongo: `rongorongo.hobbs.farm` +- LinkedStorm: `linkedstorm.hobbs.farm` ### Internal Services by Node @@ -167,7 +170,8 @@ compose/ # Standalone Compose definitions (deployed per-node) - Farm Manager: :8888 (dashboard + API) - Farm Agent: :8889 - Rongorongo: :8181 (rongorongo-web nginx; reverse-proxies /api/* to rongorongo-api Flask backend on the internal `rongorongo` bridge net). State + images at /mnt/docker-data/configs/rongorongo/data/. Public URL gated by Traefik basicAuth (creds in Vaultwarden). -- ~~VoiceCoach~~ -- moved to GPU machine (192.168.86.150) 2026-04-04 +- LinkedStorm: :8300 (pipeline/VAD+STT+speaker-ID), :8310 (API/dashboard), cron (no port). Public: linkedstorm.hobbs.farm -> :8310. Source: /opt/linkedstorm (rsync from WSL). Data: /mnt/docker-data/configs/linkedstorm/. STT via Speaches GPU machine :8000, LLM via Ollama :11434 (qwen2.5:14b). +- ~~VoiceCoach~~ -- fully decommissioned 2026-07-05, replaced by FRIDAY Voice on .150 (see GPU Machine section + memory `friday-voice`) **hf-pdocker-02 (192.168.86.100):** - Farm Agent: :8889 @@ -179,6 +183,7 @@ compose/ # Standalone Compose definitions (deployed per-node) - Actual Budget: :5006 - MCPO: :8001 - Trivy: :4954 +- LifeOS: :8095 (host) -> :8080 (container). LAN-only, deliberately NOT behind Traefik: the Flask backend is single-user with NO auth. Image `lifeos:local` built on-node from /opt/lifeos (upstream https://github.com/lifeos-app/lifeos, pinned 3009bf6). SQLite at /var/lib/lifeos/data.db on LOCAL disk (NFS breaks SQLite WAL locking). Compose: compose/hf-pdocker-02/lifeos/. Note: host 8080 is taken by unifi-os-server. **bart (192.168.86.167):** - Farm Agent: :8889 @@ -190,17 +195,14 @@ compose/ # Standalone Compose definitions (deployed per-node) - Jellyseerr: :5055 - OpenVAS: :9392 -**GPU Machine (192.168.86.150, RTX 3090, 15GB RAM):** -- Ollama: :11434 (llama3.1:8b for VoiceCoach LLM, qwen2.5:14b, others) -- [COMPOSE] VoiceCoach (all services co-located): - - voicecoach-livekit: :7880/:7881, UDP 50000-50100 (public at voice.hobbs.farm) - - voicecoach-speaches-gpu: :8000 (GPU Whisper STT + Kokoro TTS, CUDA float16) - - voicecoach-speechbrain: :8001 (emotion analysis) - - voicecoach-token-server: :8080 (token + settings API) - - voicecoach-pipecat: pipeline agent (LLM via local Ollama llama3.1:8b) - - voicecoach-web: :8085 (public at coach.hobbs.farm) - - voicecoach-openclaw: :3100 (idle, kept for future Claude API routing) -- Compose: ~/voicecoach-speaches/docker-compose.yml +**GPU Machine (192.168.86.150, RTX 3090):** +- **NOTE (2026-07-05):** .150 is actually the user's **Windows workstation** (runs the WSL/Claude Code session), not a headless Linux server. It runs **Docker Desktop** (WSL2 backend, RTX 3090 GPU passthrough working) alongside **UniFi OS Server** (Podman). Docker Desktop panics if UniFi/Podman claims `\\.\pipe\docker_engine` first — start Docker Desktop before UniFi OS Server. See memory `docker-desktop-unifi-podman-conflict`. +- **FRIDAY Voice replaced VoiceCoach (2026-07-05).** VoiceCoach fully decommissioned (v1+v2 removed from hf-pdocker-01; voice.hobbs.farm route removed). See memory `friday-voice`. +- [COMPOSE] **FRIDAY Voice stack (all on .150 Docker Desktop, via `docker.exe -c desktop-linux`):** + - **voicecoach-speaches-gpu**: :8000 — GPU Whisper large-v3-turbo (STT) + Kokoro-82M af_nova (TTS), CUDA float16. Compose: `compose/gpu-machine/voicecoach-speaches/`. + - **hermes**: `nousresearch/hermes-agent:latest` gateway — FRIDAY's reasoning agent. Settings transferred from WSL `~/.hermes` into named volume `hermes-data`. Compose: `compose/gpu-machine/hermes/`. + - **friday-voice**: :8320 — push-to-talk console. Calls speaches (`host.docker.internal:8000`) + hermes (`docker exec hermes hermes -Q` via wrapper). Public: **coach.hobbs.farm** → :8320. Compose: `compose/gpu-machine/friday-voice/`. +- Ollama: :11434 (qwen2.5:14b, Hermes-4.3-36B-GGUF, others) ### Port Conflict Notes @@ -297,3 +299,7 @@ All persistent data on NAS at `/mnt/docker-data/`: 9. **Guacamole schema**: App image pinned to `guacamole/guacamole:1.6.0` + `guacd:1.6.0` with Watchtower disabled. DB schema was reset to fresh 1.6.0 on 2026-05-20 (old schema was non-standard). Recreate connections after any DB reset. 10. **Gitea image**: Pinned to `gitea/gitea:1.26.1` with Watchtower disabled (2026-05-20). Watchtower previously pulled broken `:latest` images causing missing git binary errors. 11. **Home Assistant reverse proxy**: Config at `/mnt/docker-data/configs/homeassistant/configuration.yaml` (no hyphen). Must have `http: use_x_forwarded_for: true` + `trusted_proxies: [172.16.0.0/12, 127.0.0.1]` or proxied requests return 400. +12. **hf-pdocker-01 memory pressure (2026-07-05)**: Node runs hot (~15/23GB, chronic swap). Mitigations applied: `vm.swappiness` lowered 60→10 (persisted in `/etc/sysctl.d/99-swappiness.conf`), and VoiceCoach (v1+v2, ~10 containers) fully removed (replaced by FRIDAY Voice on .150). Frigate is the main consumer (~9GB/12GB). Frigate has no semantic-search/face/LPR features enabled. +13. **Docker Desktop vs UniFi OS Server pipe conflict (Windows box .150, 2026-07-05)**: Both fight over `\\.\pipe\docker_engine`; Docker Desktop's backend nil-panics ("backend exited before becoming ready") if Podman wins. Fix: let Docker Desktop (auto-starts at login) claim the pipe first, launch UniFi OS Server manually after. Full detail in memory `docker-desktop-unifi-podman-conflict`. +14. **MySQL InnoDB corruption on NFS**: `semaphore-db` (mysql:8.0 on NFS) hit system-tablespace page corruption and crash-looped 3789× (2026-07-11), same class as the Guacamole-on-NFS issue (#9). Recovered in-place via `--innodb-force-recovery=1` → mysqldump → fresh volume → reimport (DB was empty, no data lost); admin recreated with `semaphore user add`. Any MySQL DB on NFS is at risk — prefer LOCAL storage like guacamole-db. See memory `semaphore-db-nfs-corruption`. +15. **litellm hangs after DB blip**: litellm does NOT self-recover from a transient `litellm-db` disconnect — it loops on "Prisma DB reconnect failed" forever; needs manual `docker restart litellm`. Its healthcheck was fixed from `curl` (absent from image, permanent false-unhealthy) to a python urllib check (2026-07-11). diff --git a/Rongorongo/RONGORONGO_DEPLOY.md b/Rongorongo/RONGORONGO_DEPLOY.md new file mode 100644 index 0000000..2699c38 --- /dev/null +++ b/Rongorongo/RONGORONGO_DEPLOY.md @@ -0,0 +1,322 @@ +# RONGORONGO_DEPLOY.md +# Rongorongo Decipherment Engine — Claude Code Deployment Guide +# Target: hobbs.farm Docker Swarm infrastructure +# Subdomain: rongorongo.hobbs.farm + +--- + +## CONTEXT + +This document instructs Claude Code to build, push, and deploy the Rongorongo +Decipherment Engine as a containerized static web app to the hobbs.farm Swarm. + +The app is a single-file HTML/JS application served by nginx. It calls the +Anthropic API directly from the browser. No backend, no database. All session +state is stored in browser localStorage. + +Infrastructure assumptions (from existing hobbs.farm stack): +- Docker Swarm: nodes hf-pdocker-01, hf-pdocker-02, bart +- Traefik reverse proxy with wildcard *.hobbs.farm TLS via Cloudflare DNS +- External overlay network: traefik-public +- Registry: registry.hobbs.farm (if local) OR use Docker Hub as fallback +- Portainer CE available for stack management +- DNS: AdGuard Home with hairpin NAT for *.hobbs.farm + +--- + +## REPOSITORY STRUCTURE + +After cloning or creating the project, verify this structure exists: + +``` +rongorongo/ +├── Dockerfile +├── nginx.conf +├── stack.yml +├── frontend/ +│ └── index.html +└── RONGORONGO_DEPLOY.md (this file) +``` + +--- + +## STEP 1 — VERIFY PREREQUISITES + +Run these checks before proceeding. Stop and report any failures. + +```bash +# Confirm Docker and Swarm context +docker info | grep -E "Swarm:|Manager:|Nodes:" + +# Confirm traefik-public network exists +docker network ls | grep traefik-public + +# Confirm Traefik is running +docker service ls | grep traefik + +# Test registry connectivity (if using local registry) +curl -s https://registry.hobbs.farm/v2/_catalog || echo "Registry unreachable — will use build-and-load method" + +# Confirm you are on a Swarm manager node +docker node ls +``` + +If registry.hobbs.farm is unreachable, use the ALTERNATIVE DEPLOYMENT method +at the bottom of this document. + +--- + +## STEP 2 — BUILD THE IMAGE + +```bash +cd rongorongo/ + +# Build image +docker build -t registry.hobbs.farm/rongorongo:latest . + +# Verify image was created +docker images | grep rongorongo +``` + +Expected output: image tagged `registry.hobbs.farm/rongorongo:latest`, size +should be ~25–40MB (nginx:alpine base + small HTML file). + +--- + +## STEP 3 — PUSH TO REGISTRY + +```bash +# If registry requires auth: +docker login registry.hobbs.farm + +# Push +docker push registry.hobbs.farm/rongorongo:latest +``` + +If the push fails due to registry issues, fall through to ALTERNATIVE DEPLOYMENT. + +--- + +## STEP 4 — DEPLOY STACK + +```bash +# Deploy or update the stack +docker stack deploy -c stack.yml rongorongo --with-registry-auth + +# Watch rollout +docker service ps rongorongo_rongorongo --no-trunc +``` + +Wait for the service to show `Running` status. This typically takes 15–30 seconds. + +```bash +# Confirm healthy +docker service ls | grep rongorongo +curl -s http://localhost/health # or via container exec +``` + +--- + +## STEP 5 — DNS VERIFICATION + +The stack.yml assumes `rongorongo.hobbs.farm` resolves to the Traefik entrypoint. + +```bash +# Check AdGuard Home has the record (or wildcard covers it) +nslookup rongorongo.hobbs.farm + +# Test internal access +curl -sk https://rongorongo.hobbs.farm/health +``` + +If using AdGuard Home for internal DNS and the wildcard *.hobbs.farm is already +configured for hairpin NAT, no additional DNS entry is needed. + +--- + +## STEP 6 — SMOKE TEST + +Open https://rongorongo.hobbs.farm in a browser and verify: +1. App loads with "Rongorongo · Decipherment Engine" title +2. Sidebar navigation works (Image Corpus, Glyph Catalog, etc.) +3. Enter an Anthropic API key in the sidebar footer input +4. Upload a test image to the corpus +5. Navigate to Pattern Analyzer and run an analysis +6. Confirm AI response appears in the result box + +--- + +## UPDATING THE APP + +When index.html is modified: + +```bash +cd rongorongo/ + +# Rebuild +docker build -t registry.hobbs.farm/rongorongo:latest . + +# Push +docker push registry.hobbs.farm/rongorongo:latest + +# Force service update (pull new image) +docker service update --image registry.hobbs.farm/rongorongo:latest --force rongorongo_rongorongo + +# Watch update +docker service ps rongorongo_rongorongo +``` + +--- + +## ROLLBACK + +If something breaks after an update: + +```bash +docker service rollback rongorongo_rongorongo +docker service ps rongorongo_rongorongo +``` + +--- + +## TEARDOWN + +To remove the service without destroying data (there is no persistent volume): + +```bash +docker stack rm rongorongo +``` + +--- + +## ALTERNATIVE DEPLOYMENT (no registry) + +Use this if registry.hobbs.farm is unavailable. This method builds the image +directly on the Swarm manager and uses `docker service create` instead of +`stack deploy`. + +```bash +# On the Swarm manager node: +cd rongorongo/ + +# Build locally +docker build -t rongorongo:latest . + +# Create service manually (adjust Traefik labels as needed) +docker service create \ + --name rongorongo_rongorongo \ + --network traefik-public \ + --label "traefik.enable=true" \ + --label "traefik.docker.network=traefik-public" \ + --label "traefik.http.routers.rongorongo.rule=Host(\`rongorongo.hobbs.farm\`)" \ + --label "traefik.http.routers.rongorongo.entrypoints=websecure" \ + --label "traefik.http.routers.rongorongo.tls=true" \ + --label "traefik.http.routers.rongorongo.tls.certresolver=cloudflare" \ + --label "traefik.http.services.rongorongo.loadbalancer.server.port=80" \ + --replicas 1 \ + --restart-condition on-failure \ + rongorongo:latest + +# Monitor +docker service ps rongorongo_rongorongo +``` + +NOTE: With this method, to update you must rebuild locally and run: +```bash +docker service update --image rongorongo:latest --force rongorongo_rongorongo +``` + +This only works if your current node is the one running the replica. If you +have a multi-node swarm, you should set up the registry to avoid placement issues. + +--- + +## PORTAINER DEPLOYMENT (optional GUI method) + +If you prefer Portainer CE: + +1. Navigate to https://portainer.hobbs.farm (or your Portainer URL) +2. Select the Swarm endpoint +3. Go to Stacks → Add Stack +4. Name: `rongorongo` +5. Paste the contents of stack.yml into the web editor +6. Set `registry.hobbs.farm/rongorongo:latest` as the image + (you still need to build and push first) +7. Click Deploy the stack + +--- + +## ENVIRONMENT NOTES + +The app does NOT use environment variables or Docker secrets. The Anthropic API +key is entered by the user at runtime and stored in their browser's localStorage. +This is intentional — the app is designed to be used by a single user (Jeremy) +and no server-side key storage is needed or appropriate. + +If you want to pre-populate the API key for convenience (only if the deployment +is strictly private/internal), you can inject it at build time by modifying +frontend/index.html before building: + +Find this line: +```javascript +function getApiKey() { + return document.getElementById('apiKeyInput').value.trim() +``` + +And prepend a hardcoded fallback (NOT recommended for any externally accessible URL). + +--- + +## TRAEFIK ENTRYPOINT NAMES + +The stack.yml assumes: +- HTTP entrypoint name: `web` +- HTTPS entrypoint name: `websecure` +- Cert resolver name: `cloudflare` + +These match the standard hobbs.farm Traefik configuration. If your Traefik uses +different names, edit the labels in stack.yml before deploying. + +--- + +## TROUBLESHOOTING + +**Service won't start:** +```bash +docker service ps rongorongo_rongorongo --no-trunc +docker inspect $(docker ps -q --filter name=rongorongo) 2>/dev/null | grep -A5 '"State"' +``` + +**Traefik not routing:** +```bash +# Check Traefik logs for rongorongo +docker service logs traefik_traefik 2>&1 | grep -i rongorongo | tail -20 +# Confirm labels are applied +docker service inspect rongorongo_rongorongo | grep -A2 Labels +``` + +**TLS cert not issuing:** +```bash +# Wildcard cert should already cover rongorongo.hobbs.farm +# If not, check Cloudflare DNS API token has Zone:Read and DNS:Edit +docker service logs traefik_traefik 2>&1 | grep -i "acme\|certificate\|cloudflare" | tail -20 +``` + +**API calls blocked (CORS/CSP):** +The nginx.conf includes a Content-Security-Policy header that allows +`connect-src https://api.anthropic.com`. If you see CORS errors in the browser +console, verify the CSP header is present: +```bash +curl -I https://rongorongo.hobbs.farm/ | grep -i content-security +``` + +**App loads but AI calls fail:** +- Check API key is entered correctly in the sidebar +- Anthropic API key must start with `sk-ant-` +- Confirm outbound HTTPS to api.anthropic.com is not blocked by firewall rules +- The app sends `anthropic-dangerous-direct-browser-access: true` header which + is required for direct browser API access + +--- + +## END OF DEPLOYMENT GUIDE diff --git a/Rongorongo/_extracted/rongorongo/Dockerfile b/Rongorongo/_extracted/rongorongo/Dockerfile new file mode 100644 index 0000000..ff84aa3 --- /dev/null +++ b/Rongorongo/_extracted/rongorongo/Dockerfile @@ -0,0 +1,18 @@ +FROM nginx:1.25-alpine + +# Remove default nginx static content +RUN rm -rf /usr/share/nginx/html/* + +# Copy app +COPY frontend/index.html /usr/share/nginx/html/index.html + +# Copy nginx config +COPY nginx.conf /etc/nginx/conf.d/default.conf + +# Healthcheck +HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \ + CMD wget -qO- http://localhost/health || exit 1 + +EXPOSE 80 + +CMD ["nginx", "-g", "daemon off;"] diff --git a/Rongorongo/_extracted/rongorongo/deploy/RONGORONGO_DEPLOY.md b/Rongorongo/_extracted/rongorongo/deploy/RONGORONGO_DEPLOY.md new file mode 100644 index 0000000..2699c38 --- /dev/null +++ b/Rongorongo/_extracted/rongorongo/deploy/RONGORONGO_DEPLOY.md @@ -0,0 +1,322 @@ +# RONGORONGO_DEPLOY.md +# Rongorongo Decipherment Engine — Claude Code Deployment Guide +# Target: hobbs.farm Docker Swarm infrastructure +# Subdomain: rongorongo.hobbs.farm + +--- + +## CONTEXT + +This document instructs Claude Code to build, push, and deploy the Rongorongo +Decipherment Engine as a containerized static web app to the hobbs.farm Swarm. + +The app is a single-file HTML/JS application served by nginx. It calls the +Anthropic API directly from the browser. No backend, no database. All session +state is stored in browser localStorage. + +Infrastructure assumptions (from existing hobbs.farm stack): +- Docker Swarm: nodes hf-pdocker-01, hf-pdocker-02, bart +- Traefik reverse proxy with wildcard *.hobbs.farm TLS via Cloudflare DNS +- External overlay network: traefik-public +- Registry: registry.hobbs.farm (if local) OR use Docker Hub as fallback +- Portainer CE available for stack management +- DNS: AdGuard Home with hairpin NAT for *.hobbs.farm + +--- + +## REPOSITORY STRUCTURE + +After cloning or creating the project, verify this structure exists: + +``` +rongorongo/ +├── Dockerfile +├── nginx.conf +├── stack.yml +├── frontend/ +│ └── index.html +└── RONGORONGO_DEPLOY.md (this file) +``` + +--- + +## STEP 1 — VERIFY PREREQUISITES + +Run these checks before proceeding. Stop and report any failures. + +```bash +# Confirm Docker and Swarm context +docker info | grep -E "Swarm:|Manager:|Nodes:" + +# Confirm traefik-public network exists +docker network ls | grep traefik-public + +# Confirm Traefik is running +docker service ls | grep traefik + +# Test registry connectivity (if using local registry) +curl -s https://registry.hobbs.farm/v2/_catalog || echo "Registry unreachable — will use build-and-load method" + +# Confirm you are on a Swarm manager node +docker node ls +``` + +If registry.hobbs.farm is unreachable, use the ALTERNATIVE DEPLOYMENT method +at the bottom of this document. + +--- + +## STEP 2 — BUILD THE IMAGE + +```bash +cd rongorongo/ + +# Build image +docker build -t registry.hobbs.farm/rongorongo:latest . + +# Verify image was created +docker images | grep rongorongo +``` + +Expected output: image tagged `registry.hobbs.farm/rongorongo:latest`, size +should be ~25–40MB (nginx:alpine base + small HTML file). + +--- + +## STEP 3 — PUSH TO REGISTRY + +```bash +# If registry requires auth: +docker login registry.hobbs.farm + +# Push +docker push registry.hobbs.farm/rongorongo:latest +``` + +If the push fails due to registry issues, fall through to ALTERNATIVE DEPLOYMENT. + +--- + +## STEP 4 — DEPLOY STACK + +```bash +# Deploy or update the stack +docker stack deploy -c stack.yml rongorongo --with-registry-auth + +# Watch rollout +docker service ps rongorongo_rongorongo --no-trunc +``` + +Wait for the service to show `Running` status. This typically takes 15–30 seconds. + +```bash +# Confirm healthy +docker service ls | grep rongorongo +curl -s http://localhost/health # or via container exec +``` + +--- + +## STEP 5 — DNS VERIFICATION + +The stack.yml assumes `rongorongo.hobbs.farm` resolves to the Traefik entrypoint. + +```bash +# Check AdGuard Home has the record (or wildcard covers it) +nslookup rongorongo.hobbs.farm + +# Test internal access +curl -sk https://rongorongo.hobbs.farm/health +``` + +If using AdGuard Home for internal DNS and the wildcard *.hobbs.farm is already +configured for hairpin NAT, no additional DNS entry is needed. + +--- + +## STEP 6 — SMOKE TEST + +Open https://rongorongo.hobbs.farm in a browser and verify: +1. App loads with "Rongorongo · Decipherment Engine" title +2. Sidebar navigation works (Image Corpus, Glyph Catalog, etc.) +3. Enter an Anthropic API key in the sidebar footer input +4. Upload a test image to the corpus +5. Navigate to Pattern Analyzer and run an analysis +6. Confirm AI response appears in the result box + +--- + +## UPDATING THE APP + +When index.html is modified: + +```bash +cd rongorongo/ + +# Rebuild +docker build -t registry.hobbs.farm/rongorongo:latest . + +# Push +docker push registry.hobbs.farm/rongorongo:latest + +# Force service update (pull new image) +docker service update --image registry.hobbs.farm/rongorongo:latest --force rongorongo_rongorongo + +# Watch update +docker service ps rongorongo_rongorongo +``` + +--- + +## ROLLBACK + +If something breaks after an update: + +```bash +docker service rollback rongorongo_rongorongo +docker service ps rongorongo_rongorongo +``` + +--- + +## TEARDOWN + +To remove the service without destroying data (there is no persistent volume): + +```bash +docker stack rm rongorongo +``` + +--- + +## ALTERNATIVE DEPLOYMENT (no registry) + +Use this if registry.hobbs.farm is unavailable. This method builds the image +directly on the Swarm manager and uses `docker service create` instead of +`stack deploy`. + +```bash +# On the Swarm manager node: +cd rongorongo/ + +# Build locally +docker build -t rongorongo:latest . + +# Create service manually (adjust Traefik labels as needed) +docker service create \ + --name rongorongo_rongorongo \ + --network traefik-public \ + --label "traefik.enable=true" \ + --label "traefik.docker.network=traefik-public" \ + --label "traefik.http.routers.rongorongo.rule=Host(\`rongorongo.hobbs.farm\`)" \ + --label "traefik.http.routers.rongorongo.entrypoints=websecure" \ + --label "traefik.http.routers.rongorongo.tls=true" \ + --label "traefik.http.routers.rongorongo.tls.certresolver=cloudflare" \ + --label "traefik.http.services.rongorongo.loadbalancer.server.port=80" \ + --replicas 1 \ + --restart-condition on-failure \ + rongorongo:latest + +# Monitor +docker service ps rongorongo_rongorongo +``` + +NOTE: With this method, to update you must rebuild locally and run: +```bash +docker service update --image rongorongo:latest --force rongorongo_rongorongo +``` + +This only works if your current node is the one running the replica. If you +have a multi-node swarm, you should set up the registry to avoid placement issues. + +--- + +## PORTAINER DEPLOYMENT (optional GUI method) + +If you prefer Portainer CE: + +1. Navigate to https://portainer.hobbs.farm (or your Portainer URL) +2. Select the Swarm endpoint +3. Go to Stacks → Add Stack +4. Name: `rongorongo` +5. Paste the contents of stack.yml into the web editor +6. Set `registry.hobbs.farm/rongorongo:latest` as the image + (you still need to build and push first) +7. Click Deploy the stack + +--- + +## ENVIRONMENT NOTES + +The app does NOT use environment variables or Docker secrets. The Anthropic API +key is entered by the user at runtime and stored in their browser's localStorage. +This is intentional — the app is designed to be used by a single user (Jeremy) +and no server-side key storage is needed or appropriate. + +If you want to pre-populate the API key for convenience (only if the deployment +is strictly private/internal), you can inject it at build time by modifying +frontend/index.html before building: + +Find this line: +```javascript +function getApiKey() { + return document.getElementById('apiKeyInput').value.trim() +``` + +And prepend a hardcoded fallback (NOT recommended for any externally accessible URL). + +--- + +## TRAEFIK ENTRYPOINT NAMES + +The stack.yml assumes: +- HTTP entrypoint name: `web` +- HTTPS entrypoint name: `websecure` +- Cert resolver name: `cloudflare` + +These match the standard hobbs.farm Traefik configuration. If your Traefik uses +different names, edit the labels in stack.yml before deploying. + +--- + +## TROUBLESHOOTING + +**Service won't start:** +```bash +docker service ps rongorongo_rongorongo --no-trunc +docker inspect $(docker ps -q --filter name=rongorongo) 2>/dev/null | grep -A5 '"State"' +``` + +**Traefik not routing:** +```bash +# Check Traefik logs for rongorongo +docker service logs traefik_traefik 2>&1 | grep -i rongorongo | tail -20 +# Confirm labels are applied +docker service inspect rongorongo_rongorongo | grep -A2 Labels +``` + +**TLS cert not issuing:** +```bash +# Wildcard cert should already cover rongorongo.hobbs.farm +# If not, check Cloudflare DNS API token has Zone:Read and DNS:Edit +docker service logs traefik_traefik 2>&1 | grep -i "acme\|certificate\|cloudflare" | tail -20 +``` + +**API calls blocked (CORS/CSP):** +The nginx.conf includes a Content-Security-Policy header that allows +`connect-src https://api.anthropic.com`. If you see CORS errors in the browser +console, verify the CSP header is present: +```bash +curl -I https://rongorongo.hobbs.farm/ | grep -i content-security +``` + +**App loads but AI calls fail:** +- Check API key is entered correctly in the sidebar +- Anthropic API key must start with `sk-ant-` +- Confirm outbound HTTPS to api.anthropic.com is not blocked by firewall rules +- The app sends `anthropic-dangerous-direct-browser-access: true` header which + is required for direct browser API access + +--- + +## END OF DEPLOYMENT GUIDE diff --git a/Rongorongo/_extracted/rongorongo/frontend/index.html b/Rongorongo/_extracted/rongorongo/frontend/index.html new file mode 100644 index 0000000..3a8b08c --- /dev/null +++ b/Rongorongo/_extracted/rongorongo/frontend/index.html @@ -0,0 +1,2057 @@ + + + + + +Rongorongo · Decipherment Engine + + + + + + +
+ + +
+
+ Rongorongo + Decipherment Engine +
+
+
+ Corpus 0 images · + Glyphs 0 · + Anchors 0 +
+ + +
+
+ + + + + +
+ + +
+ + +
+ +
𓀠
+
Drop images here or click to browse
+
JPG · PNG · WEBP · Multiple files supported
+
+ +
+ +
+
Suggested Sources
+
+
— Barthel's Grundlagen zur Entzifferung der Osterinselschrift (1958) — 26 authenticated objects
+
— Fischer's Rongorongo: The Easter Island Script (1997)
+
— Musée de l'Homme, Paris · Smithsonian NMNH · Bishop Museum, Honolulu
+
— Santiago Staff tablet (highest glyph density, ~2320 glyphs)
+
+
+
+ + +
+ + +
+
+
Select Image to Annotate
+ +
+ +
+
+ + + Draw a box around a glyph +
+
+
+ + +
+
+ + +
+
+
+ +
+
Glyph Registry
+
+
No glyphs tagged yet
+
+ +
+
+
+ + +
+ + +
+
Analysis Mode
+
+ + + + + + +
+ +
+ +
+
+
+ +
+ ◉ ANALYSIS OUTPUT +
Upload images and run an analysis to begin
+
+ + +
+ + +
+ + +
+
+
+ + +
+
+ + +
+
+ +
+ +
+ ◇ SEQUENCE MATCHES +
Load multiple tablet images then run the matcher
+
+
+ + +
+ + +
+
Rapa Nui Phoneme Inventory (Known)
+
+ CV structure dominant. Consonants: p t k m n ŋ h r v. Vowels: a e i o u.
+ Most frequent syllables (corpus analysis): ko · ma · ta · ra · ka · ha · na · te · ai · oa +
+ +
+ + +
+ + +
+ + +
+
+
+
Define New Anchor
+
+ + +
+
+ + +
+
+
+ + +
+
+ + +
+
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+ +
+
Anchor Registry
+
+
No anchors defined
+
+
+
+
+ + +
+ + +
+
+
+ + +
+
+ + +
+
+
+ + +
+ +
+ +
+ ⊕ PROPAGATION OUTPUT +
Define anchors and select one to simulate propagation
+
+
+ + +
+ + +
+
Select Anchors to Test
+
+
No anchors defined. Go to Anchor Lab first.
+
+
+
+ +
+ + + + +
+
+
+ + +
+
+ +
+ + + +
+ ✦ FEASIBILITY REPORT +
Define and select anchors, then run tests
+
+ + +
+ + +
+ + +
+
Run feasibility tests to populate confidence tracking
+
+
+ + +
+ + +
+ +
+
Ask Against Scholarship
+ + +
+ 📜 SCHOLARSHIP ANALYSIS +
Ask a question about existing decipherment scholarship
+
+
+
+ + +
+ + +
+
+
+ + +
+
+ + +
+
+
+ + +
+
+ + +
+
+ + +
+ +
+
+ +
+ + + + + diff --git a/Rongorongo/_extracted/rongorongo/nginx.conf b/Rongorongo/_extracted/rongorongo/nginx.conf new file mode 100644 index 0000000..dafc9b6 --- /dev/null +++ b/Rongorongo/_extracted/rongorongo/nginx.conf @@ -0,0 +1,41 @@ +server { + listen 80; + server_name _; + + root /usr/share/nginx/html; + index index.html; + + # Security headers + add_header X-Frame-Options "SAMEORIGIN" always; + add_header X-Content-Type-Options "nosniff" always; + add_header X-XSS-Protection "1; mode=block" always; + add_header Referrer-Policy "strict-origin-when-cross-origin" always; + + # CSP — allows Anthropic API calls and Google Fonts + add_header Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; font-src 'self' https://fonts.gstatic.com; img-src 'self' data: blob:; connect-src 'self' https://api.anthropic.com; worker-src blob:;" always; + + # Gzip + gzip on; + gzip_types text/html text/css application/javascript; + gzip_min_length 1024; + + # Health endpoint + location /health { + access_log off; + return 200 "ok\n"; + add_header Content-Type text/plain; + } + + # SPA fallback + location / { + try_files $uri $uri/ /index.html; + expires -1; + add_header Cache-Control "no-store, no-cache, must-revalidate"; + } + + # Cache static assets aggressively if you add any + location ~* \.(js|css|png|jpg|jpeg|gif|ico|woff2)$ { + expires 1y; + add_header Cache-Control "public, immutable"; + } +} diff --git a/Rongorongo/_extracted/rongorongo/stack.yml b/Rongorongo/_extracted/rongorongo/stack.yml new file mode 100644 index 0000000..2ba60cd --- /dev/null +++ b/Rongorongo/_extracted/rongorongo/stack.yml @@ -0,0 +1,48 @@ +version: "3.8" + +services: + rongorongo: + image: registry.hobbs.farm/rongorongo:latest + networks: + - traefik-public + deploy: + replicas: 1 + restart_policy: + condition: on-failure + delay: 10s + max_attempts: 3 + update_config: + parallelism: 1 + delay: 10s + failure_action: rollback + labels: + - "traefik.enable=true" + - "traefik.docker.network=traefik-public" + + # HTTP → HTTPS redirect + - "traefik.http.routers.rongorongo-http.rule=Host(`rongorongo.hobbs.farm`)" + - "traefik.http.routers.rongorongo-http.entrypoints=web" + - "traefik.http.routers.rongorongo-http.middlewares=redirect-to-https" + + # HTTPS router + - "traefik.http.routers.rongorongo.rule=Host(`rongorongo.hobbs.farm`)" + - "traefik.http.routers.rongorongo.entrypoints=websecure" + - "traefik.http.routers.rongorongo.tls=true" + - "traefik.http.routers.rongorongo.tls.certresolver=cloudflare" + + # Service + - "traefik.http.services.rongorongo.loadbalancer.server.port=80" + + # Middlewares + - "traefik.http.routers.rongorongo.middlewares=rongorongo-headers" + - "traefik.http.middlewares.rongorongo-headers.headers.customResponseHeaders.X-Robots-Tag=noindex,nofollow" + healthcheck: + test: ["CMD", "wget", "-qO-", "http://localhost/health"] + interval: 30s + timeout: 5s + retries: 3 + start_period: 10s + +networks: + traefik-public: + external: true diff --git a/Rongorongo/files.zip b/Rongorongo/files.zip new file mode 100644 index 0000000..0893e65 Binary files /dev/null and b/Rongorongo/files.zip differ diff --git a/Rongorongo/rongorongo-deploy.zip b/Rongorongo/rongorongo-deploy.zip new file mode 100644 index 0000000..558a900 Binary files /dev/null and b/Rongorongo/rongorongo-deploy.zip differ diff --git a/Rongorongo/rongorongo-v3.html b/Rongorongo/rongorongo-v3.html new file mode 100644 index 0000000..3a8b08c --- /dev/null +++ b/Rongorongo/rongorongo-v3.html @@ -0,0 +1,2057 @@ + + + + + +Rongorongo · Decipherment Engine + + + + + + +
+ + +
+
+ Rongorongo + Decipherment Engine +
+
+
+ Corpus 0 images · + Glyphs 0 · + Anchors 0 +
+ + +
+
+ + + + + +
+ + +
+ + +
+ +
𓀠
+
Drop images here or click to browse
+
JPG · PNG · WEBP · Multiple files supported
+
+ +
+ +
+
Suggested Sources
+
+
— Barthel's Grundlagen zur Entzifferung der Osterinselschrift (1958) — 26 authenticated objects
+
— Fischer's Rongorongo: The Easter Island Script (1997)
+
— Musée de l'Homme, Paris · Smithsonian NMNH · Bishop Museum, Honolulu
+
— Santiago Staff tablet (highest glyph density, ~2320 glyphs)
+
+
+
+ + +
+ + +
+
+
Select Image to Annotate
+ +
+ +
+
+ + + Draw a box around a glyph +
+
+
+ + +
+
+ + +
+
+
+ +
+
Glyph Registry
+
+
No glyphs tagged yet
+
+ +
+
+
+ + +
+ + +
+
Analysis Mode
+
+ + + + + + +
+ +
+ +
+
+
+ +
+ ◉ ANALYSIS OUTPUT +
Upload images and run an analysis to begin
+
+ + +
+ + +
+ + +
+
+
+ + +
+
+ + +
+
+ +
+ +
+ ◇ SEQUENCE MATCHES +
Load multiple tablet images then run the matcher
+
+
+ + +
+ + +
+
Rapa Nui Phoneme Inventory (Known)
+
+ CV structure dominant. Consonants: p t k m n ŋ h r v. Vowels: a e i o u.
+ Most frequent syllables (corpus analysis): ko · ma · ta · ra · ka · ha · na · te · ai · oa +
+ +
+ + +
+ + +
+ + +
+
+
+
Define New Anchor
+
+ + +
+
+ + +
+
+
+ + +
+
+ + +
+
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+ +
+
Anchor Registry
+
+
No anchors defined
+
+
+
+
+ + +
+ + +
+
+
+ + +
+
+ + +
+
+
+ + +
+ +
+ +
+ ⊕ PROPAGATION OUTPUT +
Define anchors and select one to simulate propagation
+
+
+ + +
+ + +
+
Select Anchors to Test
+
+
No anchors defined. Go to Anchor Lab first.
+
+
+
+ +
+ + + + +
+
+
+ + +
+
+ +
+ + + +
+ ✦ FEASIBILITY REPORT +
Define and select anchors, then run tests
+
+ + +
+ + +
+ + +
+
Run feasibility tests to populate confidence tracking
+
+
+ + +
+ + +
+ +
+
Ask Against Scholarship
+ + +
+ 📜 SCHOLARSHIP ANALYSIS +
Ask a question about existing decipherment scholarship
+
+
+
+ + +
+ + +
+
+
+ + +
+
+ + +
+
+
+ + +
+
+ + +
+
+ + +
+ +
+
+ +
+ + + + + diff --git a/compose/bart/media/docker-compose.yml b/compose/bart/media/docker-compose.yml index 7823ef2..5d75a8f 100644 --- a/compose/bart/media/docker-compose.yml +++ b/compose/bart/media/docker-compose.yml @@ -114,7 +114,7 @@ services: - /mnt/docker-data/configs/nzbhydra:/config ports: - "5076:5076" - mem_limit: 512M + mem_limit: 768M healthcheck: test: ["CMD-SHELL", "curl -f http://localhost:5076/ || exit 1"] interval: 30s diff --git a/compose/gpu-machine/friday-voice/Dockerfile.hermes b/compose/gpu-machine/friday-voice/Dockerfile.hermes new file mode 100644 index 0000000..f112f97 --- /dev/null +++ b/compose/gpu-machine/friday-voice/Dockerfile.hermes @@ -0,0 +1,17 @@ +# Overlay on the friday-voice app image that adds the docker CLI + a wrapper so +# the container can reach the hermes agent via `docker exec hermes hermes ...`. +# Runs as root to access the mounted /var/run/docker.sock. +FROM friday-voice:latest + +USER root + +# Static docker *client* only (~small), for `docker exec` into the hermes container. +RUN curl -fsSL https://download.docker.com/linux/static/stable/x86_64/docker-27.5.1.tgz \ + | tar -xz --strip-components=1 -C /usr/local/bin docker/docker \ + && chmod +x /usr/local/bin/docker \ + && docker --version + +COPY hermes-wrap /usr/local/bin/hermes-wrap +RUN chmod +x /usr/local/bin/hermes-wrap + +# Keep the base image's CMD (uvicorn friday_voice.app:app :8320). diff --git a/compose/gpu-machine/friday-voice/docker-compose.yml b/compose/gpu-machine/friday-voice/docker-compose.yml new file mode 100644 index 0000000..303da35 --- /dev/null +++ b/compose/gpu-machine/friday-voice/docker-compose.yml @@ -0,0 +1,49 @@ +# FRIDAY Voice on .150 (Docker Desktop) — replaces VoiceCoach. +# STT/TTS via local GPU speaches (:8000). Reasoning via the hermes container +# (invoked through the docker-exec wrapper). Runs as root for socket access. +services: + friday-voice: + build: + context: . + dockerfile: Dockerfile.hermes + image: friday-voice-hermes:latest + container_name: friday-voice + restart: unless-stopped + user: "0:0" + ports: + - "8320:8320" + volumes: + - friday-voice-data:/data + - /var/run/docker.sock:/var/run/docker.sock + environment: + - DATA_DIR=/data + - SPEACHES_URL=http://host.docker.internal:8000 + - HERMES_COMMAND=/usr/local/bin/hermes-wrap + - HERMES_SOURCE=friday-voice + - WHISPER_MODEL=deepdml/faster-whisper-large-v3-turbo-ct2 + - TTS_MODEL=speaches-ai/Kokoro-82M-v1.0-ONNX + - TTS_VOICE=af_nova + - SPEAKER_CONFIRMED_THRESHOLD=0.85 + - SPEAKER_PROBABLE_THRESHOLD=0.70 + extra_hosts: + - "host.docker.internal:host-gateway" + labels: + - "com.centurylinklabs.watchtower.enable=false" + healthcheck: + test: ["CMD", "curl", "-fsS", "http://127.0.0.1:8320/health"] + interval: 30s + timeout: 10s + retries: 3 + start_period: 20s + logging: + driver: "json-file" + options: + max-size: "20m" + max-file: "3" + +volumes: + friday-voice-data: + +networks: + default: + name: friday-voice diff --git a/compose/gpu-machine/friday-voice/hermes-wrap b/compose/gpu-machine/friday-voice/hermes-wrap new file mode 100644 index 0000000..717b3ca --- /dev/null +++ b/compose/gpu-machine/friday-voice/hermes-wrap @@ -0,0 +1,5 @@ +#!/bin/sh +# friday-voice invokes: hermes-wrap chat -q "" --source friday-voice +# We forward into the hermes container and add -Q (quiet) so only the final +# response text is returned (no banner/spinner/box art) for clean TTS. +exec docker exec hermes hermes "$@" -Q diff --git a/compose/hf-pdocker-01/ai-agents/docker-compose.yml b/compose/hf-pdocker-01/ai-agents/docker-compose.yml index 56b3607..9168712 100644 --- a/compose/hf-pdocker-01/ai-agents/docker-compose.yml +++ b/compose/hf-pdocker-01/ai-agents/docker-compose.yml @@ -36,6 +36,7 @@ services: - HOME=/home/glyptic - USER=glyptic - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY} + - GEMINI_API_KEY=${GEMINI_API_KEY} - WIKIJS_API_KEY=${WIKIJS_API_KEY} - OPENCLAW_STATE_DIR=/home/glyptic/.spirit - OPENCLAW_CONFIG_PATH=/home/glyptic/.spirit/openclaw.json @@ -43,6 +44,8 @@ services: - /mnt/infrastructure/spirit:/home/glyptic/.spirit - /var/run/docker.sock:/var/run/docker.sock - /mnt/infrastructure/spirit/ssh:/root/.ssh:ro + - /mnt/docker-data/configs/prometheus:/prometheus-config + - /mnt/docker-data/configs/alertmanager:/alertmanager-config command: ["sh", "-c", "mkdir -p /home/glyptic && node --max-old-space-size=1500 /usr/local/lib/node_modules/openclaw/dist/index.js gateway --allow-unconfigured"] mem_limit: 3G memswap_limit: 3G @@ -52,74 +55,3 @@ services: max-size: "50m" max-file: "3" - bob: - image: margaret-openclaw:latest - container_name: bob - restart: unless-stopped - network_mode: host - user: "0:0" - environment: - - NODE_ENV=production - - HOME=/home/glyptic - - USER=glyptic - - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY} - - OPENCLAW_STATE_DIR=/home/glyptic/.bob - - OPENCLAW_CONFIG_PATH=/home/glyptic/.bob/openclaw.json - volumes: - - /mnt/infrastructure/bob:/home/glyptic/.bob - command: ["sh", "-c", "mkdir -p /home/glyptic && node --max-old-space-size=1500 /usr/local/lib/node_modules/openclaw/dist/index.js gateway --allow-unconfigured"] - mem_limit: 2G - memswap_limit: 2G - logging: - driver: "json-file" - options: - max-size: "50m" - max-file: "3" - - clippy: - image: margaret-openclaw:latest - container_name: clippy - restart: unless-stopped - network_mode: host - user: "0:0" - environment: - - NODE_ENV=production - - HOME=/home/glyptic - - USER=glyptic - - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY} - - OPENCLAW_STATE_DIR=/home/glyptic/.clippy - - OPENCLAW_CONFIG_PATH=/home/glyptic/.clippy/openclaw.json - volumes: - - /mnt/infrastructure/clippy:/home/glyptic/.clippy - command: ["sh", "-c", "mkdir -p /home/glyptic && node --max-old-space-size=1500 /usr/local/lib/node_modules/openclaw/dist/index.js gateway --allow-unconfigured"] - mem_limit: 2G - memswap_limit: 2G - logging: - driver: "json-file" - options: - max-size: "50m" - max-file: "3" - - travis: - image: travis-nemoclaw:latest - container_name: travis - restart: unless-stopped - network_mode: host - user: "0:0" - environment: - - NODE_ENV=production - - HOME=/home/glyptic - - USER=glyptic - - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY} - - OPENCLAW_STATE_DIR=/home/glyptic/.travis - - OPENCLAW_CONFIG_PATH=/home/glyptic/.travis/openclaw.json - volumes: - - /mnt/infrastructure/travis:/home/glyptic/.travis - command: ["sh", "-c", "mkdir -p /home/glyptic && openclaw plugins install /opt/nemoclaw 2>/dev/null; node --max-old-space-size=1500 /usr/local/lib/node_modules/openclaw/dist/index.js gateway --allow-unconfigured"] - mem_limit: 2G - memswap_limit: 2G - logging: - driver: "json-file" - options: - max-size: "50m" - max-file: "3" diff --git a/compose/hf-pdocker-01/litellm/docker-compose.yml b/compose/hf-pdocker-01/litellm/docker-compose.yml index 8451968..d89f955 100644 --- a/compose/hf-pdocker-01/litellm/docker-compose.yml +++ b/compose/hf-pdocker-01/litellm/docker-compose.yml @@ -17,7 +17,7 @@ services: litellm-db: condition: service_healthy healthcheck: - test: ["CMD-SHELL", "curl -f http://localhost:4000/health || exit 1"] + test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:4000/health/liveliness',timeout=5).status==200 else 1)\""] interval: 30s timeout: 10s retries: 3 diff --git a/compose/hf-pdocker-01/openwebui/docker-compose.yml b/compose/hf-pdocker-01/openwebui/docker-compose.yml deleted file mode 100644 index c058e0b..0000000 --- a/compose/hf-pdocker-01/openwebui/docker-compose.yml +++ /dev/null @@ -1,29 +0,0 @@ -services: - open-webui: - image: ghcr.io/open-webui/open-webui:main - container_name: open-webui - restart: unless-stopped - ports: - - "3010:8080" - env_file: - - .env - environment: - - TZ=America/Chicago - - OLLAMA_BASE_URL=http://192.168.86.196:11434 - volumes: - - /mnt/docker-data/configs/openwebui:/app/backend/data - mem_limit: 1G - memswap_limit: 1G - labels: - - "com.centurylinklabs.watchtower.enable=false" - healthcheck: - test: ["CMD-SHELL", "curl -sf http://localhost:8080/health || exit 1"] - interval: 30s - timeout: 10s - retries: 3 - start_period: 60s - logging: - driver: "json-file" - options: - max-size: "50m" - max-file: "3" diff --git a/compose/hf-pdocker-02/n8n/docker-compose.yml b/compose/hf-pdocker-02/n8n/docker-compose.yml index 386a655..3048ac8 100644 --- a/compose/hf-pdocker-02/n8n/docker-compose.yml +++ b/compose/hf-pdocker-02/n8n/docker-compose.yml @@ -6,24 +6,13 @@ services: ports: - "5678:5678" environment: - - DB_TYPE=postgresdb - - DB_POSTGRESDB_HOST=n8n-db - - DB_POSTGRESDB_PORT=5432 - - DB_POSTGRESDB_DATABASE=n8n - - DB_POSTGRESDB_USER=n8n - - DB_POSTGRESDB_PASSWORD=${N8N_DB_PASSWORD} - - N8N_PORT=5678 - GENERIC_TIMEZONE=America/Chicago - - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY} + - N8N_SECURE_COOKIE=false + - NODE_OPTIONS=--max-old-space-size=1500 volumes: - - /mnt/docker-data/configs/n8n:/home/node/.n8n - depends_on: - n8n-db: - condition: service_healthy - networks: - - n8n-internal - mem_limit: 512M - memswap_limit: 512M + - n8n_data:/home/node/.n8n + mem_limit: 2g + memswap_limit: 2g healthcheck: test: ["CMD-SHELL", "wget -q --spider http://localhost:5678/healthz || exit 1"] interval: 30s @@ -36,35 +25,6 @@ services: max-size: "50m" max-file: "3" - n8n-db: - image: postgres:15-alpine - container_name: n8n-db - restart: unless-stopped - user: "1024:100" - environment: - - POSTGRES_DB=n8n - - POSTGRES_USER=n8n - - POSTGRES_PASSWORD=${N8N_DB_PASSWORD} - volumes: - - /mnt/docker-data/databases/n8n-db:/var/lib/postgresql/data - networks: - - n8n-internal - labels: - - "com.centurylinklabs.watchtower.enable=false" - mem_limit: 256M - memswap_limit: 256M - healthcheck: - test: ["CMD-SHELL", "pg_isready -U n8n -d n8n"] - interval: 30s - timeout: 10s - retries: 3 - start_period: 30s - logging: - driver: "json-file" - options: - max-size: "50m" - max-file: "3" - -networks: - n8n-internal: - driver: bridge +volumes: + n8n_data: + external: true diff --git a/compose/hf-pdocker-02/netclaw/.env.example b/compose/hf-pdocker-02/netclaw/.env.example deleted file mode 100644 index e4cd0c2..0000000 --- a/compose/hf-pdocker-02/netclaw/.env.example +++ /dev/null @@ -1,2 +0,0 @@ -ANTHROPIC_API_KEY=sk-ant-... -MATRIX_ACCESS_TOKEN=syt_... diff --git a/compose/hf-pdocker-02/netclaw/Dockerfile b/compose/hf-pdocker-02/netclaw/Dockerfile deleted file mode 100644 index a88373d..0000000 --- a/compose/hf-pdocker-02/netclaw/Dockerfile +++ /dev/null @@ -1,38 +0,0 @@ -FROM node:20-bookworm - -# Install system dependencies -RUN apt-get update && apt-get install -y \ - python3 python3-pip python3-venv python3-full \ - git curl jq openssh-client iputils-ping net-tools \ - && rm -rf /var/lib/apt/lists/* - -# Install uv (Python package manager used by some MCP servers) -RUN curl -LsSf https://astral.sh/uv/install.sh | sh -ENV PATH="/root/.local/bin:$PATH" - -# Install OpenClaw globally -RUN npm install -g openclaw@latest - -# Clone NetClaw -WORKDIR /opt/netclaw -RUN git clone https://github.com/automateyournetwork/netclaw.git . - -# Create openclaw home directory -RUN mkdir -p /root/.openclaw/workspace - -# Copy NetClaw workspace config into openclaw home -RUN cp -r workspace/skills /root/.openclaw/workspace/ 2>/dev/null || true -RUN cp config/openclaw.json /root/.openclaw/openclaw.json 2>/dev/null || true - -# Install Python dependencies for core MCP servers -RUN python3 -m venv /opt/venv -ENV PATH="/opt/venv/bin:$PATH" -RUN pip install --no-cache-dir pyats genie netmiko paramiko requests pyyaml jinja2 2>/dev/null || true - -# Create entrypoint script -COPY entrypoint.sh /opt/entrypoint.sh -RUN chmod +x /opt/entrypoint.sh - -EXPOSE 1179 - -ENTRYPOINT ["/opt/entrypoint.sh"] diff --git a/compose/hf-pdocker-02/netclaw/docker-compose.yml b/compose/hf-pdocker-02/netclaw/docker-compose.yml deleted file mode 100644 index 0bf1114..0000000 --- a/compose/hf-pdocker-02/netclaw/docker-compose.yml +++ /dev/null @@ -1,20 +0,0 @@ -services: - netclaw: - build: - context: . - dockerfile: Dockerfile - container_name: netclaw-aaron - restart: unless-stopped - environment: - - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY} - - MATRIX_ACCESS_TOKEN=${MATRIX_ACCESS_TOKEN} - - MATRIX_HOMESERVER=http://192.168.86.192:8008 - volumes: - - /mnt/docker-data/configs/netclaw:/root/.openclaw/data - mem_limit: 2G - memswap_limit: 2G - logging: - driver: "json-file" - options: - max-size: "50m" - max-file: "3" diff --git a/compose/hf-pdocker-02/netclaw/entrypoint.sh b/compose/hf-pdocker-02/netclaw/entrypoint.sh deleted file mode 100644 index 7e2c67f..0000000 --- a/compose/hf-pdocker-02/netclaw/entrypoint.sh +++ /dev/null @@ -1,72 +0,0 @@ -#!/bin/bash -set -e - -# Configure OpenClaw environment -mkdir -p /root/.openclaw - -# Write .env with API key -cat > /root/.openclaw/.env << EOF -ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY} -EOF - -# Write openclaw config for gateway mode -cat > /root/.openclaw/openclaw.json << 'OCEOF' -{ - "mode": "gateway", - "provider": "anthropic", - "model": "claude-sonnet-4-6", - "fallbackModel": "claude-haiku-4-5-20251001", - "name": "Aaron", - "identity": "Aaron - NetClaw AI Network Engineering Agent" -} -OCEOF - -# Copy workspace skills if they exist -if [ -d /opt/netclaw/workspace/skills ]; then - cp -r /opt/netclaw/workspace/skills /root/.openclaw/workspace/ 2>/dev/null || true -fi - -# Copy personality/identity docs -for doc in SOUL.md AGENTS.md IDENTITY.md USER.md TOOLS.md; do - if [ -f /opt/netclaw/$doc ]; then - cp /opt/netclaw/$doc /root/.openclaw/ 2>/dev/null || true - fi -done - -# Send Matrix DM to Jeremy to announce availability -if [ -n "$MATRIX_ACCESS_TOKEN" ] && [ -n "$MATRIX_HOMESERVER" ]; then - echo "Sending Matrix DM to Jeremy..." - - # Create a direct message room with Jeremy - ROOM_RESPONSE=$(curl -sf -X POST "${MATRIX_HOMESERVER}/_matrix/client/r0/createRoom" \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer ${MATRIX_ACCESS_TOKEN}" \ - -d "{ - \"is_direct\": true, - \"invite\": [\"@jeremy:hobbs.farm\"], - \"preset\": \"trusted_private_chat\" - }" 2>/dev/null || echo "") - - ROOM_ID=$(echo "$ROOM_RESPONSE" | python3 -c "import sys,json; print(json.load(sys.stdin).get('room_id',''))" 2>/dev/null || echo "") - - if [ -n "$ROOM_ID" ]; then - # Send the message - curl -sf -X PUT "${MATRIX_HOMESERVER}/_matrix/client/r0/rooms/${ROOM_ID}/send/m.room.message/$(date +%s%N)" \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer ${MATRIX_ACCESS_TOKEN}" \ - -d "{ - \"msgtype\": \"m.text\", - \"body\": \"Hey Jeremy - Aaron here. I'm online and ready to help with network engineering tasks. I'm running NetClaw with CCIE-level networking skills. Ask me about device health checks, routing analysis, security audits, or anything network-related.\" - }" 2>/dev/null || echo "Failed to send DM" - echo "DM sent to @jeremy:hobbs.farm in room $ROOM_ID" - else - echo "Warning: Could not create DM room with Jeremy. Response: $ROOM_RESPONSE" - fi -fi - -# Start OpenClaw gateway -echo "Starting NetClaw (Aaron) gateway..." -exec openclaw gateway 2>&1 || { - echo "OpenClaw gateway failed to start, keeping container alive for debugging..." - tail -f /dev/null -} diff --git a/compose/hf-pdocker-02/traefik/dynamic-routes.yml b/compose/hf-pdocker-02/traefik/dynamic-routes.yml index d69dd8e..c434d2b 100644 --- a/compose/hf-pdocker-02/traefik/dynamic-routes.yml +++ b/compose/hf-pdocker-02/traefik/dynamic-routes.yml @@ -5,6 +5,8 @@ http: service: yoda entryPoints: - websecure + middlewares: + - security-headers-allow-frames tls: certResolver: letsencrypt @@ -13,6 +15,8 @@ http: service: mess entryPoints: - websecure + middlewares: + - security-headers tls: certResolver: letsencrypt @@ -21,6 +25,8 @@ http: service: mess entryPoints: - websecure + middlewares: + - security-headers tls: certResolver: letsencrypt @@ -29,6 +35,8 @@ http: service: wellknown entryPoints: - websecure + middlewares: + - security-headers tls: certResolver: letsencrypt @@ -37,6 +45,8 @@ http: service: solo entryPoints: - websecure + middlewares: + - security-headers-allow-frames tls: certResolver: letsencrypt @@ -45,6 +55,8 @@ http: service: vault entryPoints: - websecure + middlewares: + - security-headers tls: certResolver: letsencrypt @@ -53,9 +65,41 @@ http: service: git entryPoints: - websecure + middlewares: + - security-headers tls: certResolver: letsencrypt + middlewares: + security-headers: + headers: + stsSeconds: 31536000 + stsIncludeSubdomains: true + stsPreload: true + contentTypeNosniff: true + frameDeny: true + referrerPolicy: "strict-origin-when-cross-origin" + permissionsPolicy: "camera=(), microphone=(), geolocation=(), payment=()" + browserXssFilter: true + customResponseHeaders: + X-Powered-By: "" + Server: "" + + security-headers-allow-frames: + headers: + stsSeconds: 31536000 + stsIncludeSubdomains: true + stsPreload: true + contentTypeNosniff: true + frameDeny: false + customFrameOptionsValue: "SAMEORIGIN" + referrerPolicy: "strict-origin-when-cross-origin" + permissionsPolicy: "camera=(), microphone=(), geolocation=(), payment=()" + browserXssFilter: true + customResponseHeaders: + X-Powered-By: "" + Server: "" + services: yoda: loadBalancer: diff --git a/compose/hf-pdocker-02/vaultwarden/docker-compose.yml b/compose/hf-pdocker-02/vaultwarden/docker-compose.yml index 0b6f706..a4beb95 100644 --- a/compose/hf-pdocker-02/vaultwarden/docker-compose.yml +++ b/compose/hf-pdocker-02/vaultwarden/docker-compose.yml @@ -6,6 +6,7 @@ services: environment: - DOMAIN=https://vault.hobbs.farm - SIGNUPS_ALLOWED=false + - ADMIN_TOKEN=${ADMIN_TOKEN} volumes: - /mnt/docker-data/configs/vaultwarden:/data ports: diff --git a/docs/superpowers/plans/2026-05-28-linkedstorm-implementation.md b/docs/superpowers/plans/2026-05-28-linkedstorm-implementation.md new file mode 100644 index 0000000..057ffc6 --- /dev/null +++ b/docs/superpowers/plans/2026-05-28-linkedstorm-implementation.md @@ -0,0 +1,3187 @@ +# LinkedStorm Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build and deploy LinkedStorm — 3 Docker containers (pipeline, api, cron) that capture household speech via browser WebSocket, transcribe with Whisper, identify speakers with resemblyzer, and serve a searchable daily log dashboard. + +**Architecture:** Browser tabs stream 250ms audio chunks over WebSocket to `linkedstorm-pipeline` (port 8300), which runs Silero VAD, calls Whisper STT + resemblyzer concurrently, matches voice embeddings against SQLite, and writes utterances. `linkedstorm-api` (port 8310) serves the FastAPI dashboard. `linkedstorm-cron` runs nightly Ollama summaries. All three mount the same NAS volume for SQLite + WAV clips. + +**Tech Stack:** Python 3.12, FastAPI, uvicorn, SQLite WAL+FTS5, resemblyzer, silero-vad, httpx, livekit-api, pytest, pytest-asyncio, vanilla JS, Docker Compose. + +**Code repo:** `http://192.168.86.100:3003/jeremy/linkedstorm` (clone to `/mnt/c/Users/Jeremy/projects/linkedstorm`) +**Compose repo:** `/mnt/c/Users/Jeremy/projects/docker/compose/hf-pdocker-01/linkedstorm/` + +--- + +## File Map + +``` +/mnt/c/Users/Jeremy/projects/linkedstorm/ +├── pyproject.toml +├── pytest.ini +├── .env.example +├── app/ +│ ├── shared/ +│ │ ├── __init__.py +│ │ ├── config.py # Pydantic Settings — all env vars +│ │ ├── database.py # SQLite thread-local connection, schema init +│ │ └── models.py # Dataclasses: Room, Speaker, Utterance, etc. +│ ├── pipeline/ +│ │ ├── __init__.py +│ │ ├── main.py # FastAPI app: ws://:8300/ws/{room_id} + /internal/rooms/{id}/mute +│ │ ├── ingestion.py # RoomVADBuffer: per-room VAD accumulator +│ │ ├── processor.py # process_utterance: Whisper + resemblyzer + DB write +│ │ └── recognition.py # embed_audio, find_best_speaker_match, pack/unpack embedding +│ ├── api/ +│ │ ├── __init__.py +│ │ ├── main.py # FastAPI app: mounts all routes, serves static/ +│ │ ├── routes/ +│ │ │ ├── __init__.py +│ │ │ ├── utterances.py # GET/POST /api/utterances, /tag, /dismiss, /clip +│ │ │ ├── speakers.py # GET/POST /api/speakers +│ │ │ ├── rooms.py # GET /api/rooms, POST /api/rooms/{id}/mute +│ │ │ ├── search.py # GET /api/search?q= +│ │ │ └── rag.py # POST /api/ask, GET /api/summary/{date} +│ │ └── static/ +│ │ ├── index.html # Dashboard SPA (5 tabs) +│ │ ├── capture.html # Minimal capture page (MediaRecorder + WS) +│ │ └── app.js # Dashboard JS +│ └── cron/ +│ ├── __init__.py +│ ├── main.py # Sleep-loop entrypoint: runs all jobs at midnight +│ ├── daily_summary.py # Fetch utterances → Ollama → daily_summaries row +│ ├── cluster_unknowns.py # Cluster unknown embeddings into Unknown #N groups +│ └── purge_clips.py # Delete WAV files older than CLIP_RETENTION_DAYS +├── tests/ +│ ├── conftest.py # Shared fixtures: in-memory SQLite settings +│ ├── test_database.py +│ ├── test_recognition.py +│ ├── test_ingestion.py +│ ├── test_processor.py +│ ├── test_cron.py +│ └── routes/ +│ ├── conftest.py # TestClient fixture +│ ├── test_utterances.py +│ ├── test_speakers.py +│ ├── test_rooms.py +│ ├── test_search.py +│ └── test_rag.py +├── pipeline/ +│ └── Dockerfile +├── api/ +│ └── Dockerfile +└── cron/ + └── Dockerfile + +/mnt/c/Users/Jeremy/projects/docker/ +├── compose/hf-pdocker-01/linkedstorm/ +│ ├── docker-compose.yml +│ └── .env.example +└── stacks/traefik/dynamic/linkedstorm.yml +``` + +--- + +## Task 1: Clone repo and scaffold + +**Files:** `pyproject.toml`, `pytest.ini`, `.env.example`, all `__init__.py` files + +- [ ] **Clone the linkedstorm repo locally** + +```bash +git clone http://192.168.86.100:3003/jeremy/linkedstorm /mnt/c/Users/Jeremy/projects/linkedstorm +cd /mnt/c/Users/Jeremy/projects/linkedstorm +``` + +- [ ] **Create `pyproject.toml`** + +```toml +[build-system] +requires = ["setuptools>=68"] +build-backend = "setuptools.backends.legacy:build" + +[project] +name = "linkedstorm" +version = "0.1.0" +requires-python = ">=3.12" +dependencies = [ + "fastapi==0.115.0", + "uvicorn[standard]==0.30.6", + "pydantic-settings==2.4.0", + "httpx==0.27.2", + "websockets==13.1", + "resemblyzer==0.1.1.dev0", + "silero-vad==5.1.2", + "torch==2.4.1", + "numpy==1.26.4", + "livekit-api==0.7.2", + "scipy==1.14.1", +] + +[project.optional-dependencies] +dev = [ + "pytest==8.3.3", + "pytest-asyncio==0.24.0", + "pytest-mock==3.14.0", + "httpx==0.27.2", +] + +[tool.setuptools.packages.find] +where = ["."] +include = ["app*"] +``` + +- [ ] **Create `pytest.ini`** + +```ini +[pytest] +asyncio_mode = auto +testpaths = tests +``` + +- [ ] **Create `.env.example`** + +```bash +DB_PATH=/data/linkedstorm.db +CLIPS_DIR=/data/clips +SPEACHES_URL=http://192.168.86.150:8000 +OLLAMA_URL=http://192.168.86.150:11434 +OLLAMA_MODEL=qwen2.5:14b +LIVEKIT_URL=ws://192.168.86.150:7880 +LIVEKIT_API_KEY=devkey +LIVEKIT_API_SECRET=devsecret +PIPELINE_INTERNAL_URL=http://linkedstorm-pipeline:8300 +CLIP_RETENTION_DAYS=30 +SPEAKER_KNOWN_THRESHOLD=0.85 +SPEAKER_AMBIGUOUS_THRESHOLD=0.60 +``` + +- [ ] **Create all `__init__.py` files** + +```bash +mkdir -p app/shared app/pipeline app/api/routes app/api/static app/cron +mkdir -p tests/routes pipeline api cron +touch app/__init__.py app/shared/__init__.py app/pipeline/__init__.py +touch app/api/__init__.py app/api/routes/__init__.py +touch app/cron/__init__.py +touch tests/__init__.py tests/routes/__init__.py +``` + +- [ ] **Install dev dependencies** + +```bash +pip install -e ".[dev]" +``` + +- [ ] **Commit** + +```bash +git add . +git commit -m "feat: scaffold project structure" +git push origin master +``` + +--- + +## Task 2: Shared config and models + +**Files:** `app/shared/config.py`, `app/shared/models.py` + +- [ ] **Write `app/shared/config.py`** + +```python +from pydantic_settings import BaseSettings + + +class Settings(BaseSettings): + db_path: str = "/data/linkedstorm.db" + clips_dir: str = "/data/clips" + speaches_url: str = "http://192.168.86.150:8000" + ollama_url: str = "http://192.168.86.150:11434" + ollama_model: str = "qwen2.5:14b" + livekit_url: str = "ws://192.168.86.150:7880" + livekit_api_key: str = "devkey" + livekit_api_secret: str = "devsecret" + pipeline_internal_url: str = "http://linkedstorm-pipeline:8300" + clip_retention_days: int = 30 + speaker_known_threshold: float = 0.85 + speaker_ambiguous_threshold: float = 0.60 + vad_silence_ms: int = 600 + vad_min_speech_ms: int = 500 + + model_config = {"env_file": ".env", "env_file_encoding": "utf-8"} + + +def get_settings() -> Settings: + return Settings() +``` + +- [ ] **Write `app/shared/models.py`** + +```python +from dataclasses import dataclass +from datetime import datetime +from typing import Optional + + +@dataclass +class Room: + id: int + name: str + device_label: Optional[str] + is_active: bool + last_seen: Optional[datetime] + created_at: datetime + + +@dataclass +class Speaker: + id: int + name: str + created_at: datetime + updated_at: datetime + + +@dataclass +class VoiceEmbedding: + id: int + speaker_id: int + embedding: bytes # packed float32 + audio_sample: Optional[bytes] + created_at: datetime + + +@dataclass +class Utterance: + id: int + room_id: int + speaker_id: Optional[int] + transcript: str + embedding: Optional[bytes] + match_status: str # known | unknown | ambiguous | unreviewed + match_confidence: Optional[float] + start_time: datetime + end_time: datetime + audio_clip_path: Optional[str] + livekit_room: Optional[str] + created_at: datetime + + +@dataclass +class DailySummary: + id: int + date: str # YYYY-MM-DD + summary_text: str + generated_at: datetime +``` + +- [ ] **Commit** + +```bash +git add app/shared/ +git commit -m "feat: add shared config and models" +git push origin master +``` + +--- + +## Task 3: Database layer (TDD) + +**Files:** `app/shared/database.py`, `tests/conftest.py`, `tests/test_database.py` + +- [ ] **Write failing tests first — `tests/conftest.py`** + +```python +import pytest +import tempfile +import os +from app.shared.config import Settings +from app.shared.database import init_schema, get_db + + +@pytest.fixture +def settings(tmp_path): + db_file = str(tmp_path / "test.db") + clips_dir = str(tmp_path / "clips") + os.makedirs(clips_dir, exist_ok=True) + s = Settings( + db_path=db_file, + clips_dir=clips_dir, + speaches_url="http://localhost:9999", + ollama_url="http://localhost:9999", + livekit_api_key="test", + livekit_api_secret="test", + ) + init_schema(s) + return s + + +@pytest.fixture +def db(settings): + return get_db(settings) +``` + +- [ ] **Write `tests/test_database.py`** + +```python +def test_schema_creates_rooms_table(db): + rows = db.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='rooms'").fetchall() + assert len(rows) == 1 + + +def test_schema_creates_utterances_fts(db): + rows = db.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='utterances_fts'").fetchall() + assert len(rows) == 1 + + +def test_insert_and_fetch_room(db): + db.execute("INSERT INTO rooms (name) VALUES (?)", ("Kitchen",)) + db.commit() + row = db.execute("SELECT name FROM rooms WHERE name='Kitchen'").fetchone() + assert row["name"] == "Kitchen" + + +def test_insert_speaker_and_embedding(db): + db.execute("INSERT INTO speakers (name) VALUES (?)", ("Jeremy",)) + db.commit() + speaker = db.execute("SELECT id FROM speakers WHERE name='Jeremy'").fetchone() + db.execute( + "INSERT INTO voice_embeddings (speaker_id, embedding) VALUES (?, ?)", + (speaker["id"], b"\x00\x01\x02\x03"), + ) + db.commit() + emb = db.execute("SELECT speaker_id FROM voice_embeddings").fetchone() + assert emb["speaker_id"] == speaker["id"] + + +def test_fts_trigger_on_utterance_insert(db): + db.execute("INSERT INTO rooms (name) VALUES (?)", ("Living Room",)) + db.commit() + room = db.execute("SELECT id FROM rooms WHERE name='Living Room'").fetchone() + db.execute( + "INSERT INTO utterances (room_id, transcript, match_status, start_time, end_time) VALUES (?,?,?,?,?)", + (room["id"], "hello world test", "unreviewed", "2026-01-01 10:00:00", "2026-01-01 10:00:03"), + ) + db.commit() + results = db.execute("SELECT rowid FROM utterances_fts WHERE transcript MATCH 'hello'").fetchall() + assert len(results) == 1 +``` + +- [ ] **Run tests — expect FAIL (database.py does not exist)** + +```bash +cd /mnt/c/Users/Jeremy/projects/linkedstorm +pytest tests/test_database.py -v +``` + +Expected: `ModuleNotFoundError: No module named 'app.shared.database'` + +- [ ] **Write `app/shared/database.py`** + +```python +import sqlite3 +import threading +from pathlib import Path +from app.shared.config import Settings + +_local = threading.local() + + +def get_db(settings: Settings) -> sqlite3.Connection: + if not hasattr(_local, "conn") or _local.conn is None: + conn = sqlite3.connect(settings.db_path, check_same_thread=False) + conn.row_factory = sqlite3.Row + conn.execute("PRAGMA journal_mode=WAL") + conn.execute("PRAGMA foreign_keys=ON") + _local.conn = conn + return _local.conn + + +def init_schema(settings: Settings) -> None: + Path(settings.clips_dir).mkdir(parents=True, exist_ok=True) + db = get_db(settings) + db.executescript(""" + CREATE TABLE IF NOT EXISTS rooms ( + id INTEGER PRIMARY KEY, + name TEXT NOT NULL UNIQUE, + device_label TEXT, + is_active BOOLEAN DEFAULT 0, + last_seen DATETIME, + created_at DATETIME DEFAULT CURRENT_TIMESTAMP + ); + + CREATE TABLE IF NOT EXISTS speakers ( + id INTEGER PRIMARY KEY, + name TEXT NOT NULL, + created_at DATETIME DEFAULT CURRENT_TIMESTAMP, + updated_at DATETIME DEFAULT CURRENT_TIMESTAMP + ); + + CREATE TABLE IF NOT EXISTS voice_embeddings ( + id INTEGER PRIMARY KEY, + speaker_id INTEGER NOT NULL REFERENCES speakers(id) ON DELETE CASCADE, + embedding BLOB NOT NULL, + audio_sample BLOB, + created_at DATETIME DEFAULT CURRENT_TIMESTAMP + ); + + CREATE TABLE IF NOT EXISTS utterances ( + id INTEGER PRIMARY KEY, + room_id INTEGER NOT NULL REFERENCES rooms(id), + speaker_id INTEGER REFERENCES speakers(id), + transcript TEXT NOT NULL, + embedding BLOB, + match_status TEXT NOT NULL DEFAULT 'unreviewed', + match_confidence REAL, + start_time DATETIME NOT NULL, + end_time DATETIME NOT NULL, + audio_clip_path TEXT, + livekit_room TEXT, + created_at DATETIME DEFAULT CURRENT_TIMESTAMP + ); + + CREATE VIRTUAL TABLE IF NOT EXISTS utterances_fts USING fts5( + transcript, content=utterances, content_rowid=id + ); + + CREATE TRIGGER IF NOT EXISTS utterances_ai AFTER INSERT ON utterances BEGIN + INSERT INTO utterances_fts(rowid, transcript) VALUES (new.id, new.transcript); + END; + + CREATE TRIGGER IF NOT EXISTS utterances_au AFTER UPDATE OF transcript ON utterances BEGIN + INSERT INTO utterances_fts(utterances_fts, rowid, transcript) + VALUES ('delete', old.id, old.transcript); + INSERT INTO utterances_fts(rowid, transcript) VALUES (new.id, new.transcript); + END; + + CREATE TRIGGER IF NOT EXISTS utterances_ad AFTER DELETE ON utterances BEGIN + INSERT INTO utterances_fts(utterances_fts, rowid, transcript) + VALUES ('delete', old.id, old.transcript); + END; + + CREATE TABLE IF NOT EXISTS daily_summaries ( + id INTEGER PRIMARY KEY, + date DATE NOT NULL UNIQUE, + summary_text TEXT NOT NULL, + generated_at DATETIME DEFAULT CURRENT_TIMESTAMP + ); + """) + db.commit() +``` + +- [ ] **Run tests — expect PASS** + +```bash +pytest tests/test_database.py -v +``` + +Expected: `4 passed` + +- [ ] **Commit** + +```bash +git add app/shared/database.py tests/conftest.py tests/test_database.py +git commit -m "feat: add database schema with WAL mode and FTS5 triggers" +git push origin master +``` + +--- + +## Task 4: Voice recognition module (TDD) + +**Files:** `app/pipeline/recognition.py`, `tests/test_recognition.py` + +- [ ] **Write `tests/test_recognition.py`** + +```python +import numpy as np +import struct +import pytest +from unittest.mock import patch, MagicMock + + +def test_pack_unpack_roundtrip(): + from app.pipeline.recognition import pack_embedding, unpack_embedding + original = np.array([0.1, 0.2, 0.3, -0.5], dtype=np.float32) + packed = pack_embedding(original) + recovered = unpack_embedding(packed) + np.testing.assert_allclose(recovered, original, rtol=1e-6) + + +def test_cosine_similarity_identical(): + from app.pipeline.recognition import cosine_similarity + v = np.array([1.0, 0.0, 0.0], dtype=np.float32) + assert cosine_similarity(v, v) == pytest.approx(1.0) + + +def test_cosine_similarity_orthogonal(): + from app.pipeline.recognition import cosine_similarity + a = np.array([1.0, 0.0], dtype=np.float32) + b = np.array([0.0, 1.0], dtype=np.float32) + assert cosine_similarity(a, b) == pytest.approx(0.0) + + +def test_cosine_similarity_zero_vector(): + from app.pipeline.recognition import cosine_similarity + a = np.array([0.0, 0.0], dtype=np.float32) + b = np.array([1.0, 0.0], dtype=np.float32) + assert cosine_similarity(a, b) == 0.0 + + +def test_find_best_match_known(): + from app.pipeline.recognition import find_best_speaker_match, pack_embedding + ref = np.array([1.0, 0.0, 0.0], dtype=np.float32) + query = np.array([0.98, 0.2, 0.0], dtype=np.float32) + query /= np.linalg.norm(query) + ref /= np.linalg.norm(ref) + status, sid, conf = find_best_speaker_match( + query, [(1, pack_embedding(ref))], known_threshold=0.85, ambiguous_threshold=0.60 + ) + assert status == "known" + assert sid == 1 + assert conf >= 0.85 + + +def test_find_best_match_ambiguous(): + from app.pipeline.recognition import find_best_speaker_match, pack_embedding + ref = np.array([1.0, 0.0, 0.0], dtype=np.float32) + # cos similarity ~0.707 (45 degrees) + query = np.array([1.0, 1.0, 0.0], dtype=np.float32) + query /= np.linalg.norm(query) + ref /= np.linalg.norm(ref) + status, sid, conf = find_best_speaker_match( + query, [(1, pack_embedding(ref))], known_threshold=0.85, ambiguous_threshold=0.60 + ) + assert status == "ambiguous" + assert sid == 1 + + +def test_find_best_match_unknown(): + from app.pipeline.recognition import find_best_speaker_match, pack_embedding + ref = np.array([1.0, 0.0, 0.0], dtype=np.float32) + query = np.array([0.0, 1.0, 0.0], dtype=np.float32) # orthogonal = 0 similarity + status, sid, conf = find_best_speaker_match( + query, [(1, pack_embedding(ref))], known_threshold=0.85, ambiguous_threshold=0.60 + ) + assert status == "unknown" + assert sid is None + + +def test_find_best_match_empty_embeddings(): + from app.pipeline.recognition import find_best_speaker_match + query = np.array([1.0, 0.0], dtype=np.float32) + status, sid, conf = find_best_speaker_match(query, []) + assert status == "unknown" + assert sid is None + + +def test_find_best_match_takes_max_per_speaker(): + from app.pipeline.recognition import find_best_speaker_match, pack_embedding + # Speaker 1 has two embeddings — one poor, one good + ref_bad = np.array([0.0, 1.0, 0.0], dtype=np.float32) + ref_good = np.array([1.0, 0.0, 0.0], dtype=np.float32) + query = np.array([1.0, 0.0, 0.0], dtype=np.float32) + stored = [(1, pack_embedding(ref_bad)), (1, pack_embedding(ref_good))] + status, sid, conf = find_best_speaker_match( + query, stored, known_threshold=0.85, ambiguous_threshold=0.60 + ) + assert status == "known" + assert sid == 1 + + +def test_embed_audio_calls_resemblyzer(mocker): + from app.pipeline.recognition import embed_audio + mock_wav = np.zeros(16000, dtype=np.float32) + mock_emb = np.ones(256, dtype=np.float32) + mocker.patch("app.pipeline.recognition.preprocess_wav", return_value=mock_wav) + mock_encoder = MagicMock() + mock_encoder.embed_utterance.return_value = mock_emb + mocker.patch("app.pipeline.recognition.get_encoder", return_value=mock_encoder) + result = embed_audio(b"\x00" * 100) + assert result.shape == (256,) + mock_encoder.embed_utterance.assert_called_once_with(mock_wav) +``` + +- [ ] **Run to confirm failure** + +```bash +pytest tests/test_recognition.py -v +``` + +Expected: `ModuleNotFoundError: No module named 'app.pipeline.recognition'` + +- [ ] **Write `app/pipeline/recognition.py`** + +```python +import struct +import io +import numpy as np +from resemblyzer import VoiceEncoder, preprocess_wav + +_encoder: VoiceEncoder | None = None + + +def get_encoder() -> VoiceEncoder: + global _encoder + if _encoder is None: + _encoder = VoiceEncoder() + return _encoder + + +def embed_audio(wav_bytes: bytes) -> np.ndarray: + """Extract 256-dim d-vector from raw WAV bytes.""" + wav = preprocess_wav(io.BytesIO(wav_bytes)) + return get_encoder().embed_utterance(wav) + + +def pack_embedding(embedding: np.ndarray) -> bytes: + n = len(embedding) + return struct.pack(f"{n}f", *embedding) + + +def unpack_embedding(blob: bytes) -> np.ndarray: + n = len(blob) // 4 + return np.array(struct.unpack(f"{n}f", blob), dtype=np.float32) + + +def cosine_similarity(a: np.ndarray, b: np.ndarray) -> float: + norm_a = np.linalg.norm(a) + norm_b = np.linalg.norm(b) + if norm_a == 0 or norm_b == 0: + return 0.0 + return float(np.dot(a, b) / (norm_a * norm_b)) + + +def find_best_speaker_match( + utterance_embedding: np.ndarray, + stored_embeddings: list[tuple[int, bytes]], + known_threshold: float = 0.85, + ambiguous_threshold: float = 0.60, +) -> tuple[str, int | None, float | None]: + """ + Returns (match_status, speaker_id, confidence). + Groups by speaker_id, takes MAX cosine similarity per speaker. + """ + if not stored_embeddings: + return "unknown", None, None + + best_by_speaker: dict[int, float] = {} + for speaker_id, blob in stored_embeddings: + ref = unpack_embedding(blob) + sim = cosine_similarity(utterance_embedding, ref) + if speaker_id not in best_by_speaker or sim > best_by_speaker[speaker_id]: + best_by_speaker[speaker_id] = sim + + best_id = max(best_by_speaker, key=best_by_speaker.__getitem__) + best_sim = best_by_speaker[best_id] + + if best_sim >= known_threshold: + return "known", best_id, best_sim + elif best_sim >= ambiguous_threshold: + return "ambiguous", best_id, best_sim + else: + return "unknown", None, best_sim +``` + +- [ ] **Run tests — expect PASS** + +```bash +pytest tests/test_recognition.py -v +``` + +Expected: `9 passed` + +- [ ] **Commit** + +```bash +git add app/pipeline/recognition.py tests/test_recognition.py +git commit -m "feat: voice recognition — resemblyzer embed + cosine similarity matching" +git push origin master +``` + +--- + +## Task 5: VAD ingestion buffer (TDD) + +**Files:** `app/pipeline/ingestion.py`, `tests/test_ingestion.py` + +- [ ] **Write `tests/test_ingestion.py`** + +```python +import numpy as np +import pytest +from datetime import datetime, timezone +from unittest.mock import patch, MagicMock + + +def _make_chunk(samples: int = 4000, amplitude: float = 0.0) -> bytes: + """Create a fake PCM int16 chunk.""" + audio = (np.ones(samples) * amplitude * 32767).astype(np.int16) + return audio.tobytes() + + +@pytest.fixture +def buf(mocker): + """RoomVADBuffer with mocked VAD model.""" + mocker.patch( + "app.pipeline.ingestion.vad_prob", + side_effect=lambda chunk: 0.9 if np.frombuffer(chunk, np.int16).max() > 100 else 0.1, + ) + from app.pipeline.ingestion import RoomVADBuffer + return RoomVADBuffer(silence_threshold_ms=600, min_speech_ms=500, chunk_ms=250) + + +def test_silence_produces_no_segment(buf): + ts = datetime(2026, 1, 1, tzinfo=timezone.utc) + for _ in range(10): + result = buf.process_chunk(_make_chunk(amplitude=0.0), ts) + assert result is None + + +def test_speech_then_silence_produces_segment(buf): + ts = datetime(2026, 1, 1, tzinfo=timezone.utc) + # 4 speech chunks = 1 second of speech + for _ in range(4): + result = buf.process_chunk(_make_chunk(amplitude=0.5), ts) + assert result is None # still accumulating + # 3 silence chunks = 750ms silence (> 600ms threshold) + segment = None + for _ in range(3): + segment = buf.process_chunk(_make_chunk(amplitude=0.0), ts) + assert segment is not None + assert isinstance(segment, tuple) + audio_bytes, start_time = segment + assert len(audio_bytes) > 0 + + +def test_short_speech_discarded(buf): + ts = datetime(2026, 1, 1, tzinfo=timezone.utc) + # 1 speech chunk = 250ms (below 500ms min) + buf.process_chunk(_make_chunk(amplitude=0.5), ts) + # Now silence to trigger emission + result = None + for _ in range(3): + result = buf.process_chunk(_make_chunk(amplitude=0.0), ts) + assert result is None # discarded as too short + + +def test_start_time_recorded(buf): + ts = datetime(2026, 5, 1, 12, 0, 0, tzinfo=timezone.utc) + buf.process_chunk(_make_chunk(amplitude=0.5), ts) + for _ in range(3): + result = buf.process_chunk(_make_chunk(amplitude=0.0), ts) + # Should have started at ts + # (segment would be discarded due to being too short in this test, + # so test start_time tracking via internal state) + # Reset and do a proper speech segment + from app.pipeline.ingestion import RoomVADBuffer + pass # covered by test_speech_then_silence_produces_segment +``` + +- [ ] **Run to confirm failure** + +```bash +pytest tests/test_ingestion.py -v +``` + +Expected: `ModuleNotFoundError: No module named 'app.pipeline.ingestion'` + +- [ ] **Write `app/pipeline/ingestion.py`** + +```python +import asyncio +import numpy as np +import torch +from datetime import datetime, timezone +from typing import Optional + +# Load Silero VAD model once at module level +_vad_model = None +_vad_utils = None + + +def _load_vad(): + global _vad_model, _vad_utils + if _vad_model is None: + _vad_model, _vad_utils = torch.hub.load( + repo_or_dir="snakers4/silero-vad", + model="silero_vad", + force_reload=False, + trust_repo=True, + ) + return _vad_model + + +def vad_prob(chunk: bytes, sample_rate: int = 16000) -> float: + """Return speech probability (0-1) for a PCM int16 chunk.""" + model = _load_vad() + audio = np.frombuffer(chunk, dtype=np.int16).astype(np.float32) / 32768.0 + tensor = torch.FloatTensor(audio) + with torch.no_grad(): + prob = model(tensor, sample_rate).item() + return prob + + +class RoomVADBuffer: + """ + Accumulates audio chunks for a single room. + Emits (audio_bytes, start_time) when a speech segment ends. + """ + + SPEECH_THRESHOLD = 0.5 + + def __init__( + self, + silence_threshold_ms: int = 600, + min_speech_ms: int = 500, + chunk_ms: int = 250, + sample_rate: int = 16000, + ): + self.silence_threshold_ms = silence_threshold_ms + self.min_speech_ms = min_speech_ms + self.chunk_ms = chunk_ms + self.sample_rate = sample_rate + self._buffer: list[bytes] = [] + self._speaking = False + self._silence_ms = 0 + self._start_time: Optional[datetime] = None + + def process_chunk( + self, chunk: bytes, timestamp: datetime + ) -> Optional[tuple[bytes, datetime]]: + """ + Returns (audio_bytes, start_time) when a complete speech segment is ready. + Returns None while accumulating or during silence. + """ + prob = vad_prob(chunk) + + if prob >= self.SPEECH_THRESHOLD: + if not self._speaking: + self._speaking = True + self._start_time = timestamp + self._buffer = [] + self._buffer.append(chunk) + self._silence_ms = 0 + elif self._speaking: + self._buffer.append(chunk) # include trailing silence + self._silence_ms += self.chunk_ms + if self._silence_ms >= self.silence_threshold_ms: + audio = b"".join(self._buffer) + start = self._start_time + self._buffer = [] + self._speaking = False + self._silence_ms = 0 + self._start_time = None + # Calculate duration (int16 = 2 bytes per sample) + duration_ms = (len(audio) / 2 / self.sample_rate) * 1000 + if duration_ms >= self.min_speech_ms: + return audio, start + + return None + + +# Per-room registry: room_id -> (buffer, muted_flag) +_rooms: dict[str, dict] = {} + + +def get_or_create_room(room_id: str, silence_ms: int, min_speech_ms: int) -> dict: + if room_id not in _rooms: + _rooms[room_id] = { + "buffer": RoomVADBuffer(silence_threshold_ms=silence_ms, min_speech_ms=min_speech_ms), + "muted": False, + } + return _rooms[room_id] + + +def mute_room(room_id: str) -> None: + if room_id in _rooms: + _rooms[room_id]["muted"] = True + + +def unmute_room(room_id: str) -> None: + if room_id in _rooms: + _rooms[room_id]["muted"] = False + + +def is_muted(room_id: str) -> bool: + return _rooms.get(room_id, {}).get("muted", False) +``` + +- [ ] **Run tests — expect PASS (VAD is mocked)** + +```bash +pytest tests/test_ingestion.py -v +``` + +Expected: `3 passed` (the 4th test is a placeholder, it passes trivially) + +- [ ] **Commit** + +```bash +git add app/pipeline/ingestion.py tests/test_ingestion.py +git commit -m "feat: VAD ingestion buffer — Silero VAD, per-room accumulation" +git push origin master +``` + +--- + +## Task 6: Processing pipeline (TDD) + +**Files:** `app/pipeline/processor.py`, `tests/test_processor.py` + +- [ ] **Write `tests/test_processor.py`** + +```python +import pytest +import numpy as np +from datetime import datetime, timezone +from unittest.mock import AsyncMock, MagicMock, patch + + +@pytest.fixture +def proc_settings(settings): + return settings + + +@pytest.mark.asyncio +async def test_process_utterance_known_speaker(mocker, proc_settings, db): + # Setup: insert a room and speaker with embedding + db.execute("INSERT INTO rooms (name) VALUES (?)", ("Kitchen",)) + db.execute("INSERT INTO speakers (name) VALUES (?)", ("Jeremy",)) + db.commit() + room = db.execute("SELECT id FROM rooms WHERE name='Kitchen'").fetchone() + speaker = db.execute("SELECT id FROM speakers WHERE name='Jeremy'").fetchone() + + from app.pipeline.recognition import pack_embedding + ref_emb = np.array([1.0] + [0.0] * 255, dtype=np.float32) + db.execute( + "INSERT INTO voice_embeddings (speaker_id, embedding) VALUES (?,?)", + (speaker["id"], pack_embedding(ref_emb)), + ) + db.commit() + + # Mock Whisper HTTP call + mocker.patch( + "app.pipeline.processor.transcribe_audio", + new=AsyncMock(return_value="hello world"), + ) + # Mock resemblyzer — return a very similar embedding + query_emb = np.array([0.99] + [0.0] * 255, dtype=np.float32) + mocker.patch("app.pipeline.processor.embed_audio", return_value=query_emb) + # Mock clip saving + mocker.patch("app.pipeline.processor.save_clip", return_value="/data/clips/test.wav") + + from app.pipeline.processor import process_utterance + start = datetime(2026, 1, 1, 10, 0, 0, tzinfo=timezone.utc) + end = datetime(2026, 1, 1, 10, 0, 3, tzinfo=timezone.utc) + + utterance_id = await process_utterance( + room_id=room["id"], + audio_bytes=b"\x00" * 100, + start_time=start, + end_time=end, + settings=proc_settings, + db=db, + ) + + assert utterance_id is not None + row = db.execute("SELECT * FROM utterances WHERE id=?", (utterance_id,)).fetchone() + assert row["transcript"] == "hello world" + assert row["match_status"] == "known" + assert row["speaker_id"] == speaker["id"] + + +@pytest.mark.asyncio +async def test_process_utterance_unknown_speaker(mocker, proc_settings, db): + db.execute("INSERT INTO rooms (name) VALUES (?)", ("Office",)) + db.commit() + room = db.execute("SELECT id FROM rooms WHERE name='Office'").fetchone() + + mocker.patch( + "app.pipeline.processor.transcribe_audio", + new=AsyncMock(return_value="test transcript"), + ) + mocker.patch( + "app.pipeline.processor.embed_audio", + return_value=np.zeros(256, dtype=np.float32), + ) + mocker.patch("app.pipeline.processor.save_clip", return_value="/data/clips/test.wav") + + from app.pipeline.processor import process_utterance + start = datetime(2026, 1, 1, 10, 0, 0, tzinfo=timezone.utc) + end = datetime(2026, 1, 1, 10, 0, 2, tzinfo=timezone.utc) + + utterance_id = await process_utterance( + room_id=room["id"], + audio_bytes=b"\x00" * 100, + start_time=start, + end_time=end, + settings=proc_settings, + db=db, + ) + + row = db.execute("SELECT * FROM utterances WHERE id=?", (utterance_id,)).fetchone() + assert row["match_status"] == "unknown" + assert row["speaker_id"] is None +``` + +- [ ] **Run to confirm failure** + +```bash +pytest tests/test_processor.py -v +``` + +Expected: `ModuleNotFoundError: No module named 'app.pipeline.processor'` + +- [ ] **Write `app/pipeline/processor.py`** + +```python +import asyncio +import sqlite3 +import wave +import io +from datetime import datetime, timezone +from pathlib import Path +from typing import Optional + +import httpx +import numpy as np + +from app.shared.config import Settings +from app.pipeline.recognition import embed_audio, find_best_speaker_match, pack_embedding + + +async def transcribe_audio(wav_bytes: bytes, speaches_url: str) -> str: + """POST audio to Speaches/Whisper, return transcript text.""" + async with httpx.AsyncClient(timeout=30.0) as client: + resp = await client.post( + f"{speaches_url}/v1/audio/transcriptions", + files={"file": ("audio.wav", wav_bytes, "audio/wav")}, + data={"model": "Systran/faster-whisper-large-v3-turbo"}, + ) + resp.raise_for_status() + return resp.json()["text"].strip() + + +def save_clip(audio_bytes: bytes, clips_dir: str, utterance_id: int, start_time: datetime) -> str: + """Save raw PCM bytes as a WAV file, return the path.""" + date_str = start_time.strftime("%Y-%m-%d") + clip_dir = Path(clips_dir) / date_str + clip_dir.mkdir(parents=True, exist_ok=True) + path = clip_dir / f"{utterance_id}.wav" + with wave.open(str(path), "wb") as wf: + wf.setnchannels(1) + wf.setsampwidth(2) # int16 + wf.setframerate(16000) + wf.writeframes(audio_bytes) + return str(path) + + +async def process_utterance( + room_id: int, + audio_bytes: bytes, + start_time: datetime, + end_time: datetime, + settings: Settings, + db: sqlite3.Connection, +) -> Optional[int]: + """ + Full pipeline for one speech segment: + 1. Whisper STT + resemblyzer embed (concurrent) + 2. Speaker matching + 3. SQLite write + WAV clip save + Returns the new utterance row id, or None on failure. + """ + try: + transcript, embedding = await asyncio.gather( + transcribe_audio(audio_bytes, settings.speaches_url), + asyncio.to_thread(embed_audio, audio_bytes), + ) + except Exception: + return None + + # Load all stored embeddings for matching + rows = db.execute("SELECT speaker_id, embedding FROM voice_embeddings").fetchall() + stored = [(r["speaker_id"], r["embedding"]) for r in rows] + + match_status, speaker_id, confidence = find_best_speaker_match( + embedding, + stored, + known_threshold=settings.speaker_known_threshold, + ambiguous_threshold=settings.speaker_ambiguous_threshold, + ) + + # Insert utterance row first to get the id + cur = db.execute( + """INSERT INTO utterances + (room_id, speaker_id, transcript, embedding, match_status, match_confidence, + start_time, end_time) + VALUES (?,?,?,?,?,?,?,?)""", + ( + room_id, + speaker_id, + transcript, + pack_embedding(embedding), + match_status, + confidence, + start_time.isoformat(), + end_time.isoformat(), + ), + ) + db.commit() + utterance_id = cur.lastrowid + + # Save WAV clip and update row with path + clip_path = save_clip(audio_bytes, settings.clips_dir, utterance_id, start_time) + db.execute( + "UPDATE utterances SET audio_clip_path=? WHERE id=?", + (clip_path, utterance_id), + ) + db.commit() + + # If known match, store embedding as an additional reference for this speaker + if match_status == "known" and speaker_id is not None: + db.execute( + "INSERT INTO voice_embeddings (speaker_id, embedding) VALUES (?,?)", + (speaker_id, pack_embedding(embedding)), + ) + db.commit() + + return utterance_id +``` + +- [ ] **Run tests — expect PASS** + +```bash +pytest tests/test_processor.py -v +``` + +Expected: `2 passed` + +- [ ] **Commit** + +```bash +git add app/pipeline/processor.py tests/test_processor.py +git commit -m "feat: processing pipeline — Whisper + resemblyzer + speaker match + SQLite write" +git push origin master +``` + +--- + +## Task 7: Pipeline FastAPI app (WebSocket + internal HTTP) + +**Files:** `app/pipeline/main.py` + +*(No TDD — integration test requires running server; covered in Task 18 deployment verification)* + +- [ ] **Write `app/pipeline/main.py`** + +```python +import asyncio +import sqlite3 +from contextlib import asynccontextmanager +from datetime import datetime, timezone + +import uvicorn +from fastapi import FastAPI, WebSocket, WebSocketDisconnect, HTTPException +from fastapi.responses import JSONResponse + +from app.shared.config import get_settings, Settings +from app.shared.database import get_db, init_schema +from app.pipeline.ingestion import get_or_create_room, mute_room, unmute_room, is_muted +from app.pipeline.processor import process_utterance + +_settings: Settings | None = None + + +def settings() -> Settings: + global _settings + if _settings is None: + _settings = get_settings() + return _settings + + +@asynccontextmanager +async def lifespan(app: FastAPI): + s = settings() + init_schema(s) + yield + + +app = FastAPI(lifespan=lifespan) + + +@app.websocket("/ws/{room_name}") +async def audio_ws(websocket: WebSocket, room_name: str): + await websocket.accept() + s = settings() + db = get_db(s) + + # Upsert room in DB + db.execute( + "INSERT INTO rooms (name, is_active, last_seen) VALUES (?,1,CURRENT_TIMESTAMP) " + "ON CONFLICT(name) DO UPDATE SET is_active=1, last_seen=CURRENT_TIMESTAMP", + (room_name,), + ) + db.commit() + room = db.execute("SELECT id FROM rooms WHERE name=?", (room_name,)).fetchone() + room_id = room["id"] + + room_state = get_or_create_room(room_name, s.vad_silence_ms, s.vad_min_speech_ms) + + try: + while True: + data = await websocket.receive() + if "bytes" in data: + chunk = data["bytes"] + if not is_muted(room_name): + db.execute( + "UPDATE rooms SET last_seen=CURRENT_TIMESTAMP WHERE id=?", + (room_id,), + ) + now = datetime.now(timezone.utc) + result = room_state["buffer"].process_chunk(chunk, now) + if result is not None: + audio_bytes, start_time = result + end_time = datetime.now(timezone.utc) + asyncio.create_task( + process_utterance(room_id, audio_bytes, start_time, end_time, s, db) + ) + elif "text" in data: + import json + msg = json.loads(data["text"]) + if msg.get("type") == "mute": + mute_room(room_name) + elif msg.get("type") == "unmute": + unmute_room(room_name) + except WebSocketDisconnect: + db.execute( + "UPDATE rooms SET is_active=0 WHERE id=?", (room_id,) + ) + db.commit() + + +@app.post("/internal/rooms/{room_name}/mute") +async def internal_mute(room_name: str): + mute_room(room_name) + return {"status": "muted", "room": room_name} + + +@app.post("/internal/rooms/{room_name}/unmute") +async def internal_unmute(room_name: str): + unmute_room(room_name) + return {"status": "unmuted", "room": room_name} + + +@app.get("/health") +async def health(): + return {"status": "ok"} + + +if __name__ == "__main__": + uvicorn.run("app.pipeline.main:app", host="0.0.0.0", port=8300, reload=False) +``` + +- [ ] **Commit** + +```bash +git add app/pipeline/main.py +git commit -m "feat: pipeline FastAPI app — WebSocket audio ingestion + internal mute API" +git push origin master +``` + +--- + +## Task 8: API routes — utterances (TDD) + +**Files:** `app/api/routes/utterances.py`, `tests/routes/conftest.py`, `tests/routes/test_utterances.py` + +- [ ] **Write `tests/routes/conftest.py`** + +```python +import pytest +from fastapi.testclient import TestClient +from app.shared.database import init_schema, get_db +from app.shared.config import Settings + + +@pytest.fixture +def api_client(settings): + from app.api.main import create_app + app = create_app(settings) + return TestClient(app) + + +@pytest.fixture +def seeded_db(db): + """DB with a room and speaker pre-inserted.""" + db.execute("INSERT INTO rooms (name) VALUES (?)", ("Kitchen",)) + db.execute("INSERT INTO speakers (name) VALUES (?)", ("Jeremy",)) + db.commit() + return db +``` + +- [ ] **Write `tests/routes/test_utterances.py`** + +```python +from datetime import datetime + + +def test_list_utterances_empty(api_client): + resp = api_client.get("/api/utterances") + assert resp.status_code == 200 + assert resp.json() == [] + + +def test_list_utterances_with_data(api_client, seeded_db): + room = seeded_db.execute("SELECT id FROM rooms WHERE name='Kitchen'").fetchone() + seeded_db.execute( + "INSERT INTO utterances (room_id, transcript, match_status, start_time, end_time) " + "VALUES (?,?,?,?,?)", + (room["id"], "hello world", "unknown", "2026-01-01 10:00:00", "2026-01-01 10:00:03"), + ) + seeded_db.commit() + resp = api_client.get("/api/utterances") + assert resp.status_code == 200 + data = resp.json() + assert len(data) == 1 + assert data[0]["transcript"] == "hello world" + assert data[0]["room_name"] == "Kitchen" + + +def test_filter_utterances_by_date(api_client, seeded_db): + room = seeded_db.execute("SELECT id FROM rooms WHERE name='Kitchen'").fetchone() + seeded_db.execute( + "INSERT INTO utterances (room_id, transcript, match_status, start_time, end_time) VALUES (?,?,?,?,?)", + (room["id"], "today speech", "unknown", "2026-05-27 10:00:00", "2026-05-27 10:00:03"), + ) + seeded_db.execute( + "INSERT INTO utterances (room_id, transcript, match_status, start_time, end_time) VALUES (?,?,?,?,?)", + (room["id"], "yesterday speech", "unknown", "2026-05-26 10:00:00", "2026-05-26 10:00:03"), + ) + seeded_db.commit() + resp = api_client.get("/api/utterances?date=2026-05-27") + data = resp.json() + assert len(data) == 1 + assert data[0]["transcript"] == "today speech" + + +def test_tag_utterance_existing_speaker(api_client, seeded_db): + room = seeded_db.execute("SELECT id FROM rooms WHERE name='Kitchen'").fetchone() + speaker = seeded_db.execute("SELECT id FROM speakers WHERE name='Jeremy'").fetchone() + seeded_db.execute( + "INSERT INTO utterances (room_id, transcript, match_status, start_time, end_time, embedding) VALUES (?,?,?,?,?,?)", + (room["id"], "tag me", "unknown", "2026-01-01 10:00:00", "2026-01-01 10:00:03", + b"\x00" * 1024), + ) + seeded_db.commit() + utt = seeded_db.execute("SELECT id FROM utterances").fetchone() + resp = api_client.post(f"/api/utterances/{utt['id']}/tag", json={"speaker_id": speaker["id"]}) + assert resp.status_code == 200 + updated = seeded_db.execute("SELECT * FROM utterances WHERE id=?", (utt["id"],)).fetchone() + assert updated["speaker_id"] == speaker["id"] + assert updated["match_status"] == "known" + assert updated["match_confidence"] == 1.0 + + +def test_tag_utterance_create_new_speaker(api_client, seeded_db): + room = seeded_db.execute("SELECT id FROM rooms WHERE name='Kitchen'").fetchone() + seeded_db.execute( + "INSERT INTO utterances (room_id, transcript, match_status, start_time, end_time) VALUES (?,?,?,?,?)", + (room["id"], "new speaker", "unknown", "2026-01-01 10:00:00", "2026-01-01 10:00:03"), + ) + seeded_db.commit() + utt = seeded_db.execute("SELECT id FROM utterances WHERE transcript='new speaker'").fetchone() + resp = api_client.post( + f"/api/utterances/{utt['id']}/tag", + json={"name": "Alice", "create_new": True}, + ) + assert resp.status_code == 200 + alice = seeded_db.execute("SELECT id FROM speakers WHERE name='Alice'").fetchone() + assert alice is not None + + +def test_dismiss_utterance(api_client, seeded_db): + room = seeded_db.execute("SELECT id FROM rooms WHERE name='Kitchen'").fetchone() + seeded_db.execute( + "INSERT INTO utterances (room_id, transcript, match_status, start_time, end_time) VALUES (?,?,?,?,?)", + (room["id"], "dismiss me", "unreviewed", "2026-01-01 10:00:00", "2026-01-01 10:00:03"), + ) + seeded_db.commit() + utt = seeded_db.execute("SELECT id FROM utterances WHERE transcript='dismiss me'").fetchone() + resp = api_client.post(f"/api/utterances/{utt['id']}/dismiss") + assert resp.status_code == 200 + updated = seeded_db.execute("SELECT match_status FROM utterances WHERE id=?", (utt["id"],)).fetchone() + assert updated["match_status"] == "unknown" +``` + +- [ ] **Run to confirm failure** + +```bash +pytest tests/routes/test_utterances.py -v +``` + +Expected: `ModuleNotFoundError: No module named 'app.api.main'` + +- [ ] **Write `app/api/routes/utterances.py`** + +```python +import sqlite3 +from typing import Optional +from fastapi import APIRouter, Depends, HTTPException +from fastapi.responses import FileResponse +from pydantic import BaseModel + +from app.shared.config import Settings +from app.pipeline.recognition import pack_embedding, unpack_embedding + +router = APIRouter(prefix="/api/utterances") + + +class TagRequest(BaseModel): + speaker_id: Optional[int] = None + name: Optional[str] = None + create_new: bool = False + + +def _row_to_dict(row: sqlite3.Row) -> dict: + return dict(row) + + +def get_utterances( + db: sqlite3.Connection, + date: Optional[str] = None, + room_id: Optional[int] = None, + speaker_id: Optional[int] = None, + match_status: Optional[str] = None, + limit: int = 200, +) -> list[dict]: + query = """ + SELECT u.*, r.name as room_name, s.name as speaker_name + FROM utterances u + JOIN rooms r ON u.room_id = r.id + LEFT JOIN speakers s ON u.speaker_id = s.id + WHERE 1=1 + """ + params: list = [] + if date: + query += " AND date(u.start_time) = ?" + params.append(date) + if room_id: + query += " AND u.room_id = ?" + params.append(room_id) + if speaker_id: + query += " AND u.speaker_id = ?" + params.append(speaker_id) + if match_status: + query += " AND u.match_status = ?" + params.append(match_status) + query += " ORDER BY u.start_time DESC LIMIT ?" + params.append(limit) + rows = db.execute(query, params).fetchall() + return [_row_to_dict(r) for r in rows] + + +def make_router(settings: Settings, db_factory): + + @router.get("") + def list_utterances( + date: Optional[str] = None, + room_id: Optional[int] = None, + speaker_id: Optional[int] = None, + match_status: Optional[str] = None, + ): + db = db_factory() + return get_utterances(db, date, room_id, speaker_id, match_status) + + @router.get("/{utterance_id}/clip") + def get_clip(utterance_id: int): + db = db_factory() + row = db.execute( + "SELECT audio_clip_path FROM utterances WHERE id=?", (utterance_id,) + ).fetchone() + if not row or not row["audio_clip_path"]: + raise HTTPException(404, "Clip not found or expired") + return FileResponse(row["audio_clip_path"], media_type="audio/wav") + + @router.post("/{utterance_id}/tag") + def tag_utterance(utterance_id: int, body: TagRequest): + db = db_factory() + row = db.execute("SELECT * FROM utterances WHERE id=?", (utterance_id,)).fetchone() + if not row: + raise HTTPException(404, "Utterance not found") + + speaker_id = body.speaker_id + if body.create_new and body.name: + cur = db.execute("INSERT INTO speakers (name) VALUES (?)", (body.name,)) + db.commit() + speaker_id = cur.lastrowid + + if speaker_id is None: + raise HTTPException(400, "Provide speaker_id or name+create_new") + + db.execute( + "UPDATE utterances SET speaker_id=?, match_status='known', match_confidence=1.0 WHERE id=?", + (speaker_id, utterance_id), + ) + db.commit() + + # Store utterance embedding as a reference for this speaker + if row["embedding"]: + db.execute( + "INSERT INTO voice_embeddings (speaker_id, embedding) VALUES (?,?)", + (speaker_id, row["embedding"]), + ) + db.commit() + + return {"ok": True, "speaker_id": speaker_id} + + @router.post("/{utterance_id}/dismiss") + def dismiss_utterance(utterance_id: int): + db = db_factory() + db.execute( + "UPDATE utterances SET match_status='unknown' WHERE id=?", (utterance_id,) + ) + db.commit() + return {"ok": True} + + return router +``` + +- [ ] **Write minimal `app/api/main.py` (enough for tests to import)** + +```python +import sqlite3 +from fastapi import FastAPI +from fastapi.staticfiles import StaticFiles +from pathlib import Path + +from app.shared.config import Settings +from app.shared.database import get_db, init_schema +from app.api.routes import utterances, speakers, rooms, search, rag + + +def create_app(settings: Settings) -> FastAPI: + init_schema(settings) + app = FastAPI(title="LinkedStorm") + + def db_factory() -> sqlite3.Connection: + return get_db(settings) + + app.include_router(utterances.make_router(settings, db_factory)) + app.include_router(speakers.make_router(settings, db_factory)) + app.include_router(rooms.make_router(settings, db_factory)) + app.include_router(search.make_router(settings, db_factory)) + app.include_router(rag.make_router(settings, db_factory)) + + static_dir = Path(__file__).parent / "static" + if static_dir.exists(): + app.mount("/", StaticFiles(directory=str(static_dir), html=True), name="static") + + return app +``` + +Note: `speakers`, `rooms`, `search`, `rag` routers are stub files — write them now: + +- [ ] **Write stub `app/api/routes/speakers.py`** + +```python +from fastapi import APIRouter +router = APIRouter(prefix="/api/speakers") +def make_router(settings, db_factory): + return router +``` + +- [ ] **Write stub `app/api/routes/rooms.py`** + +```python +from fastapi import APIRouter +router = APIRouter(prefix="/api/rooms") +def make_router(settings, db_factory): + return router +``` + +- [ ] **Write stub `app/api/routes/search.py`** + +```python +from fastapi import APIRouter +router = APIRouter(prefix="/api/search") +def make_router(settings, db_factory): + return router +``` + +- [ ] **Write stub `app/api/routes/rag.py`** + +```python +from fastapi import APIRouter +router = APIRouter(prefix="/api") +def make_router(settings, db_factory): + return router +``` + +- [ ] **Run tests — expect PASS** + +```bash +pytest tests/routes/test_utterances.py -v +``` + +Expected: `6 passed` + +- [ ] **Commit** + +```bash +git add app/api/ tests/routes/ +git commit -m "feat: utterances API routes — list, clip, tag, dismiss" +git push origin master +``` + +--- + +## Task 9: API routes — speakers, rooms, search, RAG (TDD) + +**Files:** `app/api/routes/speakers.py`, `rooms.py`, `search.py`, `rag.py` (replace stubs), plus tests + +- [ ] **Write `tests/routes/test_speakers.py`** + +```python +def test_list_speakers_empty(api_client): + resp = api_client.get("/api/speakers") + assert resp.status_code == 200 + assert resp.json() == [] + + +def test_create_speaker(api_client): + resp = api_client.post("/api/speakers", json={"name": "Alice"}) + assert resp.status_code == 200 + data = resp.json() + assert data["name"] == "Alice" + assert "id" in data + + +def test_list_speakers_includes_utterance_count(api_client, seeded_db): + room = seeded_db.execute("SELECT id FROM rooms WHERE name='Kitchen'").fetchone() + speaker = seeded_db.execute("SELECT id FROM speakers WHERE name='Jeremy'").fetchone() + seeded_db.execute( + "INSERT INTO utterances (room_id, speaker_id, transcript, match_status, start_time, end_time) VALUES (?,?,?,?,?,?)", + (room["id"], speaker["id"], "hi", "known", "2026-01-01 10:00:00", "2026-01-01 10:00:01"), + ) + seeded_db.commit() + resp = api_client.get("/api/speakers") + data = resp.json() + jeremy = next(s for s in data if s["name"] == "Jeremy") + assert jeremy["utterance_count"] == 1 +``` + +- [ ] **Write `tests/routes/test_rooms.py`** + +```python +def test_list_rooms(api_client, seeded_db): + resp = api_client.get("/api/rooms") + assert resp.status_code == 200 + data = resp.json() + assert any(r["name"] == "Kitchen" for r in data) + + +def test_mute_room_calls_pipeline(api_client, seeded_db, mocker): + mock_post = mocker.patch("app.api.routes.rooms.httpx.post") + mock_post.return_value.status_code = 200 + room = seeded_db.execute("SELECT id FROM rooms WHERE name='Kitchen'").fetchone() + resp = api_client.post(f"/api/rooms/{room['id']}/mute") + assert resp.status_code == 200 + mock_post.assert_called_once() +``` + +- [ ] **Write `tests/routes/test_search.py`** + +```python +def test_search_returns_matches(api_client, seeded_db): + room = seeded_db.execute("SELECT id FROM rooms WHERE name='Kitchen'").fetchone() + seeded_db.execute( + "INSERT INTO utterances (room_id, transcript, match_status, start_time, end_time) VALUES (?,?,?,?,?)", + (room["id"], "the grocery list is ready", "unknown", "2026-01-01 10:00:00", "2026-01-01 10:00:03"), + ) + seeded_db.commit() + resp = api_client.get("/api/search?q=grocery") + assert resp.status_code == 200 + data = resp.json() + assert len(data) == 1 + assert "grocery" in data[0]["transcript"] + + +def test_search_no_results(api_client): + resp = api_client.get("/api/search?q=xyznotfound") + assert resp.status_code == 200 + assert resp.json() == [] +``` + +- [ ] **Write `tests/routes/test_rag.py`** + +```python +def test_ask_returns_response(api_client, seeded_db, mocker): + mock_post = mocker.patch("app.api.routes.rag.httpx.post") + mock_post.return_value.json.return_value = { + "message": {"content": "You talked about groceries."} + } + mock_post.return_value.status_code = 200 + room = seeded_db.execute("SELECT id FROM rooms WHERE name='Kitchen'").fetchone() + seeded_db.execute( + "INSERT INTO utterances (room_id, transcript, match_status, start_time, end_time) VALUES (?,?,?,?,?)", + (room["id"], "we need milk from the store", "unknown", "2026-01-01 10:00:00", "2026-01-01 10:00:03"), + ) + seeded_db.commit() + resp = api_client.post("/api/ask", json={"question": "What did we need from the store?"}) + assert resp.status_code == 200 + data = resp.json() + assert "answer" in data + + +def test_get_summary_not_found(api_client): + resp = api_client.get("/api/summary/2026-01-01") + assert resp.status_code == 404 +``` + +- [ ] **Run to confirm failures** + +```bash +pytest tests/routes/ -v +``` + +Expected: New test files fail. `test_utterances.py` still passes. + +- [ ] **Replace stub `app/api/routes/speakers.py`** + +```python +import sqlite3 +from fastapi import APIRouter, HTTPException +from pydantic import BaseModel + +router = APIRouter(prefix="/api/speakers") + + +class CreateSpeakerRequest(BaseModel): + name: str + + +def make_router(settings, db_factory): + + @router.get("") + def list_speakers(): + db: sqlite3.Connection = db_factory() + rows = db.execute(""" + SELECT s.id, s.name, s.created_at, + COUNT(u.id) as utterance_count + FROM speakers s + LEFT JOIN utterances u ON u.speaker_id = s.id + GROUP BY s.id + ORDER BY s.name + """).fetchall() + return [dict(r) for r in rows] + + @router.post("") + def create_speaker(body: CreateSpeakerRequest): + db: sqlite3.Connection = db_factory() + cur = db.execute("INSERT INTO speakers (name) VALUES (?)", (body.name,)) + db.commit() + row = db.execute("SELECT * FROM speakers WHERE id=?", (cur.lastrowid,)).fetchone() + return dict(row) + + return router +``` + +- [ ] **Replace stub `app/api/routes/rooms.py`** + +```python +import sqlite3 +import httpx +from fastapi import APIRouter, HTTPException +from app.shared.config import Settings + +router = APIRouter(prefix="/api/rooms") + + +def make_router(settings: Settings, db_factory): + + @router.get("") + def list_rooms(): + db: sqlite3.Connection = db_factory() + rows = db.execute( + "SELECT id, name, device_label, is_active, last_seen, created_at FROM rooms ORDER BY name" + ).fetchall() + return [dict(r) for r in rows] + + @router.post("/{room_id}/mute") + def mute_room(room_id: int): + db: sqlite3.Connection = db_factory() + row = db.execute("SELECT name FROM rooms WHERE id=?", (room_id,)).fetchone() + if not row: + raise HTTPException(404, "Room not found") + httpx.post(f"{settings.pipeline_internal_url}/internal/rooms/{row['name']}/mute") + return {"ok": True, "room": row["name"]} + + @router.post("/{room_id}/unmute") + def unmute_room(room_id: int): + db: sqlite3.Connection = db_factory() + row = db.execute("SELECT name FROM rooms WHERE id=?", (room_id,)).fetchone() + if not row: + raise HTTPException(404, "Room not found") + httpx.post(f"{settings.pipeline_internal_url}/internal/rooms/{row['name']}/unmute") + return {"ok": True, "room": row["name"]} + + return router +``` + +- [ ] **Replace stub `app/api/routes/search.py`** + +```python +import sqlite3 +from typing import Optional +from fastapi import APIRouter, HTTPException + +router = APIRouter(prefix="/api") + + +def make_router(settings, db_factory): + + @router.get("/search") + def search(q: str, date: Optional[str] = None): + if not q.strip(): + return [] + db: sqlite3.Connection = db_factory() + fts_query = q.strip() + rows = db.execute(""" + SELECT u.*, r.name as room_name, s.name as speaker_name + FROM utterances_fts fts + JOIN utterances u ON u.id = fts.rowid + JOIN rooms r ON u.room_id = r.id + LEFT JOIN speakers s ON u.speaker_id = s.id + WHERE utterances_fts MATCH ? + ORDER BY u.start_time DESC + LIMIT 100 + """, (fts_query,)).fetchall() + return [dict(r) for r in rows] + + return router +``` + +- [ ] **Replace stub `app/api/routes/rag.py`** + +```python +import sqlite3 +import httpx +from fastapi import APIRouter, HTTPException +from pydantic import BaseModel +from app.shared.config import Settings + +router = APIRouter(prefix="/api") + + +class AskRequest(BaseModel): + question: str + date: str | None = None + + +def make_router(settings: Settings, db_factory): + + @router.post("/ask") + def ask(body: AskRequest): + db: sqlite3.Connection = db_factory() + # Fetch recent transcripts as context (last 200 utterances for the date or today) + date_filter = body.date or "date('now')" + rows = db.execute(""" + SELECT u.start_time, u.transcript, r.name as room_name, s.name as speaker_name + FROM utterances u + JOIN rooms r ON u.room_id = r.id + LEFT JOIN speakers s ON u.speaker_id = s.id + WHERE date(u.start_time) = date(?) + ORDER BY u.start_time ASC + LIMIT 200 + """, (body.date or "now",)).fetchall() + + context_lines = [ + f"[{r['room_name']} {r['start_time']}] {r['speaker_name'] or 'Unknown'}: {r['transcript']}" + for r in rows + ] + context = "\n".join(context_lines) if context_lines else "(no transcripts for this date)" + + prompt = ( + f"You are a helpful assistant with access to household conversation transcripts.\n\n" + f"Transcripts:\n{context}\n\n" + f"Question: {body.question}\n\n" + f"Answer based only on the transcripts above. Cite the time and room when relevant." + ) + + resp = httpx.post( + f"{settings.ollama_url}/api/chat", + json={ + "model": settings.ollama_model, + "messages": [{"role": "user", "content": prompt}], + "stream": False, + }, + timeout=60.0, + ) + resp.raise_for_status() + answer = resp.json()["message"]["content"] + return {"answer": answer, "context_count": len(rows)} + + @router.get("/summary/{date}") + def get_summary(date: str): + db: sqlite3.Connection = db_factory() + row = db.execute( + "SELECT * FROM daily_summaries WHERE date=?", (date,) + ).fetchone() + if not row: + raise HTTPException(404, f"No summary for {date}") + return dict(row) + + return router +``` + +- [ ] **Run all route tests** + +```bash +pytest tests/routes/ -v +``` + +Expected: `~14 passed` + +- [ ] **Commit** + +```bash +git add app/api/routes/ tests/routes/ +git commit -m "feat: API routes — speakers, rooms, search, RAG" +git push origin master +``` + +--- + +## Task 10: Cron jobs (TDD) + +**Files:** `app/cron/daily_summary.py`, `app/cron/cluster_unknowns.py`, `app/cron/purge_clips.py`, `app/cron/main.py`, `tests/test_cron.py` + +- [ ] **Write `tests/test_cron.py`** + +```python +import pytest +import os +from datetime import datetime, timezone +from pathlib import Path +from unittest.mock import patch, MagicMock + + +def _seed_utterances(db, room_id, count=3): + for i in range(count): + db.execute( + "INSERT INTO utterances (room_id, transcript, match_status, start_time, end_time) VALUES (?,?,?,?,?)", + (room_id, f"utterance {i}", "unknown", + "2026-05-27 10:00:00", "2026-05-27 10:00:03"), + ) + db.commit() + + +def test_daily_summary_inserts_row(mocker, settings, db): + db.execute("INSERT INTO rooms (name) VALUES (?)", ("Kitchen",)) + db.commit() + room = db.execute("SELECT id FROM rooms WHERE name='Kitchen'").fetchone() + _seed_utterances(db, room["id"]) + + mock_post = mocker.patch("app.cron.daily_summary.httpx.post") + mock_post.return_value.json.return_value = {"message": {"content": "Busy day."}} + mock_post.return_value.raise_for_status = MagicMock() + + from app.cron.daily_summary import generate_daily_summary + generate_daily_summary("2026-05-27", settings, db) + + row = db.execute("SELECT * FROM daily_summaries WHERE date='2026-05-27'").fetchone() + assert row is not None + assert row["summary_text"] == "Busy day." + + +def test_daily_summary_skips_if_no_utterances(mocker, settings, db): + mock_post = mocker.patch("app.cron.daily_summary.httpx.post") + from app.cron.daily_summary import generate_daily_summary + generate_daily_summary("2026-05-27", settings, db) + mock_post.assert_not_called() + + +def test_purge_clips_deletes_old_files(settings, db, tmp_path): + from app.cron.purge_clips import purge_old_clips + import shutil + + clips_dir = tmp_path / "clips" + old_dir = clips_dir / "2026-04-01" + old_dir.mkdir(parents=True) + old_file = old_dir / "1.wav" + old_file.write_bytes(b"\x00" * 100) + + new_dir = clips_dir / "2026-05-27" + new_dir.mkdir(parents=True) + new_file = new_dir / "2.wav" + new_file.write_bytes(b"\x00" * 100) + + settings_override = type(settings)( + db_path=settings.db_path, + clips_dir=str(clips_dir), + clip_retention_days=30, + speaches_url=settings.speaches_url, + ollama_url=settings.ollama_url, + livekit_api_key=settings.livekit_api_key, + livekit_api_secret=settings.livekit_api_secret, + ) + + purge_old_clips("2026-05-28", settings_override, db) + assert not old_file.exists() + assert new_file.exists() +``` + +- [ ] **Run to confirm failures** + +```bash +pytest tests/test_cron.py -v +``` + +Expected: `ModuleNotFoundError: No module named 'app.cron.daily_summary'` + +- [ ] **Write `app/cron/daily_summary.py`** + +```python +import sqlite3 +import httpx +from app.shared.config import Settings + + +def generate_daily_summary(date: str, settings: Settings, db: sqlite3.Connection) -> None: + rows = db.execute(""" + SELECT u.start_time, u.transcript, r.name as room_name, s.name as speaker_name + FROM utterances u + JOIN rooms r ON u.room_id = r.id + LEFT JOIN speakers s ON u.speaker_id = s.id + WHERE date(u.start_time) = ? + ORDER BY u.start_time ASC + """, (date,)).fetchall() + + if not rows: + return + + lines = [ + f"[{r['room_name']} {r['start_time']}] {r['speaker_name'] or 'Unknown'}: {r['transcript']}" + for r in rows + ] + context = "\n".join(lines) + + prompt = ( + f"Summarize the following household conversations from {date} in 2-4 sentences. " + f"Focus on key decisions, activities, and topics discussed. Do not transcribe verbatim.\n\n" + f"{context}" + ) + + resp = httpx.post( + f"{settings.ollama_url}/api/chat", + json={ + "model": settings.ollama_model, + "messages": [{"role": "user", "content": prompt}], + "stream": False, + }, + timeout=120.0, + ) + resp.raise_for_status() + summary = resp.json()["message"]["content"] + + db.execute( + "INSERT INTO daily_summaries (date, summary_text) VALUES (?,?) " + "ON CONFLICT(date) DO UPDATE SET summary_text=?, generated_at=CURRENT_TIMESTAMP", + (date, summary, summary), + ) + db.commit() +``` + +- [ ] **Write `app/cron/cluster_unknowns.py`** + +```python +import sqlite3 +import numpy as np +from app.shared.config import Settings +from app.pipeline.recognition import unpack_embedding, cosine_similarity + + +def cluster_unknown_speakers(settings: Settings, db: sqlite3.Connection) -> None: + """ + Groups unknown/ambiguous utterances from the last 7 days by voice similarity. + Assigns speaker_id from an existing 'Unknown #N' speaker or creates a new one. + Threshold: 0.70 cosine similarity. + """ + CLUSTER_THRESHOLD = 0.70 + + rows = db.execute(""" + SELECT id, embedding FROM utterances + WHERE match_status IN ('unknown', 'ambiguous') + AND embedding IS NOT NULL + AND date(start_time) >= date('now', '-7 days') + ORDER BY start_time ASC + """).fetchall() + + if not rows: + return + + # Build clusters greedily + clusters: list[list[int]] = [] # list of utterance id lists + cluster_centroids: list[np.ndarray] = [] + + for row in rows: + emb = unpack_embedding(row["embedding"]) + best_cluster = None + best_sim = 0.0 + for i, centroid in enumerate(cluster_centroids): + sim = cosine_similarity(emb, centroid) + if sim > best_sim and sim >= CLUSTER_THRESHOLD: + best_sim = sim + best_cluster = i + if best_cluster is not None: + clusters[best_cluster].append(row["id"]) + # Update centroid as mean + n = len(clusters[best_cluster]) + cluster_centroids[best_cluster] = ( + (cluster_centroids[best_cluster] * (n - 1) + emb) / n + ) + else: + clusters.append([row["id"]]) + cluster_centroids.append(emb) + + # Assign Unknown #N speakers to clusters with > 1 utterance + for i, cluster in enumerate(clusters): + if len(cluster) < 2: + continue + name = f"Unknown #{i + 1}" + existing = db.execute("SELECT id FROM speakers WHERE name=?", (name,)).fetchone() + if existing: + speaker_id = existing["id"] + else: + cur = db.execute("INSERT INTO speakers (name) VALUES (?)", (name,)) + db.commit() + speaker_id = cur.lastrowid + for utt_id in cluster: + db.execute( + "UPDATE utterances SET speaker_id=?, match_status='ambiguous' WHERE id=?", + (speaker_id, utt_id), + ) + db.commit() +``` + +- [ ] **Write `app/cron/purge_clips.py`** + +```python +import sqlite3 +from datetime import datetime, timedelta +from pathlib import Path +from app.shared.config import Settings + + +def purge_old_clips(today: str, settings: Settings, db: sqlite3.Connection) -> None: + cutoff = datetime.strptime(today, "%Y-%m-%d") - timedelta(days=settings.clip_retention_days) + clips_root = Path(settings.clips_dir) + deleted = 0 + for date_dir in clips_root.iterdir(): + if not date_dir.is_dir(): + continue + try: + dir_date = datetime.strptime(date_dir.name, "%Y-%m-%d") + except ValueError: + continue + if dir_date < cutoff: + for wav in date_dir.glob("*.wav"): + wav.unlink() + deleted += 1 + try: + date_dir.rmdir() + except OSError: + pass + # Null out audio_clip_path for purged clips + cutoff_str = cutoff.strftime("%Y-%m-%d") + db.execute( + "UPDATE utterances SET audio_clip_path=NULL WHERE date(start_time) < ?", + (cutoff_str,), + ) + db.commit() +``` + +- [ ] **Write `app/cron/main.py`** + +```python +import time +from datetime import datetime, timezone + +from app.shared.config import get_settings +from app.shared.database import get_db, init_schema +from app.cron.daily_summary import generate_daily_summary +from app.cron.cluster_unknowns import cluster_unknown_speakers +from app.cron.purge_clips import purge_old_clips + + +def run_nightly(settings, db): + today = datetime.now(timezone.utc).strftime("%Y-%m-%d") + yesterday = (datetime.now(timezone.utc).date()).strftime("%Y-%m-%d") + generate_daily_summary(yesterday, settings, db) + cluster_unknown_speakers(settings, db) + purge_old_clips(today, settings, db) + + +def main(): + settings = get_settings() + init_schema(settings) + db = get_db(settings) + + while True: + now = datetime.now(timezone.utc) + # Sleep until midnight UTC + next_midnight = now.replace(hour=0, minute=0, second=0, microsecond=0) + if next_midnight <= now: + from datetime import timedelta + next_midnight += timedelta(days=1) + sleep_secs = (next_midnight - now).total_seconds() + time.sleep(sleep_secs) + run_nightly(settings, db) + + +if __name__ == "__main__": + main() +``` + +- [ ] **Run tests — expect PASS** + +```bash +pytest tests/test_cron.py -v +``` + +Expected: `3 passed` + +- [ ] **Commit** + +```bash +git add app/cron/ tests/test_cron.py +git commit -m "feat: cron jobs — daily summary, speaker clustering, clip purge" +git push origin master +``` + +--- + +## Task 11: Capture page + +**Files:** `app/api/static/capture.html` + +*(No unit tests — browser JS; verified manually in Task 18)* + +- [ ] **Create `app/api/static/capture.html`** + +```html + + + + + +LinkedStorm Capture + + + +
+

⚡ LinkedStorm

+ + +
+ + + +``` + +- [ ] **Commit** + +```bash +git add app/api/static/capture.html +git commit -m "feat: browser capture page — MediaRecorder WebSocket streaming + mute" +git push origin master +``` + +--- + +## Task 12: Dashboard SPA + +**Files:** `app/api/static/index.html`, `app/api/static/app.js` + +- [ ] **Create `app/api/static/index.html`** + +```html + + + + + +LinkedStorm + + + + +
+
+ +
+
+
+
+ + + +``` + +- [ ] **Create `app/api/static/app.js`** + +```javascript +// --- Utilities --- +const COLORS = ['#60a5fa','#a78bfa','#f472b6','#34d399','#fb923c','#38bdf8','#e879f9','#a3e635']; +const speakerColor = (id) => COLORS[(id || 0) % COLORS.length]; +const speakerInitial = (name) => (name || '?')[0].toUpperCase(); +const fmtTime = (iso) => new Date(iso).toLocaleTimeString([], {hour:'2-digit',minute:'2-digit',second:'2-digit'}); +const fmtDate = (iso) => new Date(iso).toLocaleDateString([], {weekday:'long',month:'long',day:'numeric',year:'numeric'}); +const today = () => new Date().toISOString().slice(0,10); + +let _currentDate = today(); + +function showPanel(name, el) { + document.querySelectorAll('.panel').forEach(p => p.classList.remove('active')); + document.querySelectorAll('.nav-item').forEach(n => n.classList.remove('active')); + document.getElementById('panel-' + name).classList.add('active'); + if (el) el.classList.add('active'); + if (name === 'log') loadLog(); + if (name === 'search') renderSearch(); + if (name === 'speakers') loadSpeakers(); + if (name === 'rag') renderRag(); + if (name === 'rooms') loadRooms(); +} + +// --- Daily Log --- +async function loadLog(date) { + if (date) _currentDate = date; + const panel = document.getElementById('panel-log'); + const utterances = await fetch(`/api/utterances?date=${_currentDate}`).then(r => r.json()); + + const nav = `
+ + ${fmtDate(_currentDate + 'T12:00:00')} + + +
`; + + if (!utterances.length) { + panel.innerHTML = nav + '
No utterances recorded for this day.
'; + return; + } + + // Group by hour + const byHour = {}; + for (const u of utterances.reverse()) { + const h = new Date(u.start_time).getHours(); + const label = new Date(u.start_time).toLocaleTimeString([], {hour:'2-digit',minute:'2-digit'}); + const key = label.slice(0,5); + if (!byHour[key]) byHour[key] = []; + byHour[key].push(u); + } + + let html = nav; + for (const [timeKey, utts] of Object.entries(byHour)) { + html += `
${timeKey}
`; + for (const u of utts) { + const color = speakerColor(u.speaker_id); + const name = u.speaker_name || 'Unknown'; + const isUnknown = !u.speaker_id; + const badge = u.match_status === 'unknown' ? 'unknown' + : u.match_status === 'ambiguous' ? 'ambiguous' : ''; + const actions = isUnknown ? `
+ + + +
` : ''; + html += `
+
${speakerInitial(name)}
+
+
+ ${name}${badge} + ${u.room_name} + ${fmtTime(u.start_time)} +
+
${escHtml(u.transcript)}
+ ${actions} +
+
`; + } + html += '
'; + } + panel.innerHTML = html; +} + +function shiftDate(days) { + const d = new Date(_currentDate + 'T12:00:00'); + d.setDate(d.getDate() + days); + return d.toISOString().slice(0,10); +} + +async function playClip(id) { + const a = document.createElement('audio'); + a.src = `/api/utterances/${id}/clip`; + a.controls = true; + a.autoplay = true; + document.body.appendChild(a); + a.onended = () => a.remove(); +} + +async function tagSpeaker(utteranceId) { + const speakers = await fetch('/api/speakers').then(r => r.json()); + const name = prompt('Speaker name (existing or new):' + speakers.map(s => '\n• ' + s.name).join('')); + if (!name) return; + const existing = speakers.find(s => s.name.toLowerCase() === name.toLowerCase()); + const body = existing + ? { speaker_id: existing.id } + : { name: name.trim(), create_new: true }; + await fetch(`/api/utterances/${utteranceId}/tag`, { method: 'POST', headers: {'Content-Type':'application/json'}, body: JSON.stringify(body) }); + loadLog(); +} + +async function dismiss(id) { + await fetch(`/api/utterances/${id}/dismiss`, { method: 'POST' }); + loadLog(); +} + +// --- Search --- +function renderSearch() { + const panel = document.getElementById('panel-search'); + panel.innerHTML = `

Search Transcripts

+
+ + +
+
`; +} + +async function doSearch() { + const q = document.getElementById('searchQ').value.trim(); + if (!q) return; + const results = await fetch(`/api/search?q=${encodeURIComponent(q)}`).then(r => r.json()); + const container = document.getElementById('searchResults'); + if (!results.length) { container.innerHTML = '
No results.
'; return; } + const highlight = (text) => text.replace(new RegExp(`(${q})`, 'gi'), '$1'); + container.innerHTML = results.map(r => `
+
${r.speaker_name || 'Unknown'} · ${r.room_name} · ${fmtDate(r.start_time)} ${fmtTime(r.start_time)}
+
${highlight(escHtml(r.transcript))}
+
`).join(''); +} + +// --- Speakers --- +async function loadSpeakers() { + const panel = document.getElementById('panel-speakers'); + const [speakers, unknowns] = await Promise.all([ + fetch('/api/speakers').then(r => r.json()), + fetch('/api/utterances?match_status=unreviewed').then(r => r.json()), + ]); + let html = ''; + if (unknowns.length) { + html += `
⚠ ${unknowns.length} utterances need speaker tags + +
`; + } + html += `
` + speakers.map(s => { + const color = speakerColor(s.id); + return `
+
${speakerInitial(s.name)}
+
${escHtml(s.name)}
+
${s.utterance_count} utterances
+
`; + }).join('') + `
+
+ Add speaker
+
`; + panel.innerHTML = `

Speakers

` + html; +} + +async function createSpeaker() { + const name = prompt('Speaker name:'); + if (!name) return; + await fetch('/api/speakers', { method: 'POST', headers: {'Content-Type':'application/json'}, body: JSON.stringify({name}) }); + loadSpeakers(); +} + +// --- Ask AI --- +const _ragHistory = []; +function renderRag() { + const panel = document.getElementById('panel-rag'); + panel.innerHTML = `

Ask AI

+
+
+ + +
`; + renderRagMessages(); +} + +function renderRagMessages() { + const el = document.getElementById('ragMessages'); + if (!el) return; + if (!_ragHistory.length) { el.innerHTML = '
Ask a question about any conversation.
'; return; } + el.innerHTML = _ragHistory.map(m => `
+
${m.role === 'user' ? 'You' : 'LinkedStorm AI'}
+
${escHtml(m.content)}${m.sources ? `
Sources: ${m.sources}
` : ''}
+
`).join(''); + el.scrollTop = el.scrollHeight; +} + +async function askRag() { + const q = document.getElementById('ragQ').value.trim(); + if (!q) return; + document.getElementById('ragQ').value = ''; + _ragHistory.push({ role: 'user', content: q }); + renderRagMessages(); + const resp = await fetch('/api/ask', { + method: 'POST', headers: {'Content-Type':'application/json'}, + body: JSON.stringify({ question: q, date: _currentDate }), + }).then(r => r.json()); + _ragHistory.push({ role: 'ai', content: resp.answer, sources: `${resp.context_count} transcript segments` }); + renderRagMessages(); +} + +// --- Rooms --- +async function loadRooms() { + const panel = document.getElementById('panel-rooms'); + const rooms = await fetch('/api/rooms').then(r => r.json()); + let html = `

Capture Rooms

`; + for (const r of rooms) { + const cls = r.is_active ? (r.muted ? 'muted' : 'active') : ''; + const dotColor = r.is_active ? (r.muted ? '#ef4444' : '#4ade80') : '#475569'; + const statusText = r.is_active ? (r.muted ? 'Muted' : 'Streaming') : 'Disconnected'; + html += `
+
${escHtml(r.name)}
+
+
+ ${statusText} +
+ ${r.last_seen ? `
Last: ${fmtTime(r.last_seen)}
` : ''} + + + Open capture page ↗ + +
`; + } + html += `
`; + panel.innerHTML = html; +} + +async function toggleMute(roomId) { + const rooms = await fetch('/api/rooms').then(r => r.json()); + const room = rooms.find(r => r.id === roomId); + const action = room?.muted ? 'unmute' : 'mute'; + await fetch(`/api/rooms/${roomId}/${action}`, { method: 'POST' }); + loadRooms(); +} + +function escHtml(s) { + return String(s).replace(/&/g,'&').replace(//g,'>'); +} + +// Initial load +loadLog(); +setInterval(loadLog, 30000); // Refresh every 30s +``` + +- [ ] **Commit** + +```bash +git add app/api/static/ +git commit -m "feat: dashboard SPA — 5-tab log/search/speakers/RAG/rooms UI" +git push origin master +``` + +--- + +## Task 13: Dockerfiles + +**Files:** `pipeline/Dockerfile`, `api/Dockerfile`, `cron/Dockerfile` + +- [ ] **Write `pipeline/Dockerfile`** + +```dockerfile +FROM python:3.12-slim + +RUN apt-get update && apt-get install -y --no-install-recommends \ + ffmpeg libsndfile1 && rm -rf /var/lib/apt/lists/* + +WORKDIR /app + +COPY pyproject.toml . +RUN pip install --no-cache-dir torch --index-url https://download.pytorch.org/whl/cpu \ + && pip install --no-cache-dir -e "." \ + && pip uninstall -y torch && pip install --no-cache-dir torch --index-url https://download.pytorch.org/whl/cpu + +COPY app/ app/ + +# Pre-download Silero VAD and resemblyzer models into the image layer +RUN python -c "import torch; torch.hub.load('snakers4/silero-vad','silero_vad',trust_repo=True)" \ + && python -c "from resemblyzer import VoiceEncoder; VoiceEncoder()" + +ENV PYTHONUNBUFFERED=1 +CMD ["python", "-m", "app.pipeline.main"] +``` + +- [ ] **Write `api/Dockerfile`** + +```dockerfile +FROM python:3.12-slim + +WORKDIR /app +COPY pyproject.toml . +RUN pip install --no-cache-dir fastapi uvicorn[standard] pydantic-settings httpx livekit-api + +COPY app/ app/ + +ENV PYTHONUNBUFFERED=1 +CMD ["uvicorn", "app.api.main:app", "--host", "0.0.0.0", "--port", "8310", "--factory"] +``` + +Update `app/api/main.py` to add an `app` factory callable at module level: + +```python +# At the bottom of app/api/main.py, add: +import os +_app_instance = None +def app_factory(): + global _app_instance + if _app_instance is None: + from app.shared.config import get_settings + _app_instance = create_app(get_settings()) + return _app_instance + +# Uvicorn --factory needs this to be importable as app.api.main:app +# Use: uvicorn app.api.main:app_factory --factory +``` + +Change CMD in api/Dockerfile: +```dockerfile +CMD ["uvicorn", "app.api.main:app_factory", "--factory", "--host", "0.0.0.0", "--port", "8310"] +``` + +- [ ] **Write `cron/Dockerfile`** + +```dockerfile +FROM python:3.12-slim + +RUN apt-get update && apt-get install -y --no-install-recommends \ + libsndfile1 && rm -rf /var/lib/apt/lists/* + +WORKDIR /app +COPY pyproject.toml . +RUN pip install --no-cache-dir torch --index-url https://download.pytorch.org/whl/cpu \ + && pip install --no-cache-dir -e "." +COPY app/ app/ + +# Pre-download resemblyzer model +RUN python -c "from resemblyzer import VoiceEncoder; VoiceEncoder()" + +ENV PYTHONUNBUFFERED=1 +CMD ["python", "-m", "app.cron.main"] +``` + +- [ ] **Commit** + +```bash +git add pipeline/ api/ cron/ +git commit -m "feat: Dockerfiles for pipeline, api, and cron containers" +git push origin master +``` + +--- + +## Task 14: Docker Compose + Traefik + deploy + +**Files:** +- `compose/hf-pdocker-01/linkedstorm/docker-compose.yml` (in docker repo) +- `compose/hf-pdocker-01/linkedstorm/.env.example` (in docker repo) +- `stacks/traefik/dynamic/linkedstorm.yml` (in docker repo) + +- [ ] **Create `compose/hf-pdocker-01/linkedstorm/docker-compose.yml`** (in the docker repo at `/mnt/c/Users/Jeremy/projects/docker/`) + +```yaml +services: + linkedstorm-pipeline: + build: + context: /opt/linkedstorm + dockerfile: pipeline/Dockerfile + container_name: linkedstorm-pipeline + restart: unless-stopped + ports: + - "8300:8300" + volumes: + - /mnt/docker-data/configs/linkedstorm:/data + environment: + - DB_PATH=/data/linkedstorm.db + - CLIPS_DIR=/data/clips + - SPEACHES_URL=${SPEACHES_URL} + - OLLAMA_URL=${OLLAMA_URL} + - LIVEKIT_API_KEY=${LIVEKIT_API_KEY} + - LIVEKIT_API_SECRET=${LIVEKIT_API_SECRET} + networks: + - linkedstorm-net + labels: + - com.centurylinklabs.watchtower.enable=false + + linkedstorm-api: + build: + context: /opt/linkedstorm + dockerfile: api/Dockerfile + container_name: linkedstorm-api + restart: unless-stopped + ports: + - "8310:8310" + volumes: + - /mnt/docker-data/configs/linkedstorm:/data + environment: + - DB_PATH=/data/linkedstorm.db + - CLIPS_DIR=/data/clips + - PIPELINE_INTERNAL_URL=http://linkedstorm-pipeline:8300 + - OLLAMA_URL=${OLLAMA_URL} + - OLLAMA_MODEL=${OLLAMA_MODEL:-qwen2.5:14b} + - LIVEKIT_API_KEY=${LIVEKIT_API_KEY} + - LIVEKIT_API_SECRET=${LIVEKIT_API_SECRET} + networks: + - linkedstorm-net + depends_on: + - linkedstorm-pipeline + labels: + - com.centurylinklabs.watchtower.enable=false + + linkedstorm-cron: + build: + context: /opt/linkedstorm + dockerfile: cron/Dockerfile + container_name: linkedstorm-cron + restart: unless-stopped + volumes: + - /mnt/docker-data/configs/linkedstorm:/data + environment: + - DB_PATH=/data/linkedstorm.db + - CLIPS_DIR=/data/clips + - OLLAMA_URL=${OLLAMA_URL} + - OLLAMA_MODEL=${OLLAMA_MODEL:-qwen2.5:14b} + - CLIP_RETENTION_DAYS=${CLIP_RETENTION_DAYS:-30} + networks: + - linkedstorm-net + labels: + - com.centurylinklabs.watchtower.enable=false + +networks: + linkedstorm-net: + driver: bridge +``` + +- [ ] **Create `compose/hf-pdocker-01/linkedstorm/.env.example`** + +```bash +SPEACHES_URL=http://192.168.86.150:8000 +OLLAMA_URL=http://192.168.86.150:11434 +OLLAMA_MODEL=qwen2.5:14b +LIVEKIT_API_KEY=your-livekit-api-key +LIVEKIT_API_SECRET=your-livekit-api-secret +CLIP_RETENTION_DAYS=30 +``` + +- [ ] **Create `stacks/traefik/dynamic/linkedstorm.yml`** + +```yaml +http: + routers: + linkedstorm: + rule: "Host(`linkedstorm.hobbs.farm`)" + entryPoints: + - websecure + service: linkedstorm + tls: + certResolver: letsencrypt + + services: + linkedstorm: + loadBalancer: + servers: + - url: "http://192.168.86.192:8310" +``` + +- [ ] **Commit to docker repo** + +```bash +cd /mnt/c/Users/Jeremy/projects/docker +git add compose/hf-pdocker-01/linkedstorm/ stacks/traefik/dynamic/linkedstorm.yml +git commit -m "feat: linkedstorm docker-compose and Traefik route" +git push origin master +``` + +--- + +## Task 15: Deploy and verify on hf-pdocker-01 + +- [ ] **SSH to hf-pdocker-01** + +```bash +ssh glyptic@192.168.86.192 +``` + +- [ ] **Clone linkedstorm repo and create data directory** + +```bash +sudo git clone http://192.168.86.100:3003/jeremy/linkedstorm /opt/linkedstorm +sudo mkdir -p /mnt/docker-data/configs/linkedstorm/clips +sudo chown -R 1000:1000 /opt/linkedstorm +``` + +- [ ] **Copy compose files from NAS** + +```bash +# Compose files are already synced to NAS via gitea-backup +cd /mnt/docker-data/compose/hf-pdocker-01/linkedstorm +# OR pull from docker repo: +sudo git clone http://192.168.86.100:3003/jeremy/hobbs-iac /opt/hobbs-iac +# then copy compose dir to /mnt/docker-data/compose/hf-pdocker-01/linkedstorm/ +``` + +- [ ] **Create `.env` from example and fill in real credentials** + +```bash +cd /mnt/docker-data/compose/hf-pdocker-01/linkedstorm +cp .env.example .env +# Edit .env: add real LIVEKIT_API_KEY and LIVEKIT_API_SECRET from Vaultwarden +nano .env +``` + +- [ ] **Build and start all 3 containers** + +```bash +cd /mnt/docker-data/compose/hf-pdocker-01/linkedstorm +docker compose up -d --build +``` + +Expected: 3 containers start: `linkedstorm-pipeline`, `linkedstorm-api`, `linkedstorm-cron` + +- [ ] **Verify 1 — all containers healthy** + +```bash +docker ps --format "table {{.Names}}\t{{.Status}}" | grep linkedstorm +``` + +Expected: +``` +linkedstorm-pipeline Up X minutes +linkedstorm-api Up X minutes +linkedstorm-cron Up X minutes +``` + +- [ ] **Verify 2 — health endpoint responds** + +```bash +curl -s http://localhost:8310/health # Note: add /health to api/main.py +curl -s http://localhost:8300/health +``` + +Expected: `{"status":"ok"}` from both + +Note: Add health endpoint to `app/api/main.py` before deploying: +```python +@app.get("/health") +async def health(): + return {"status": "ok"} +``` + +- [ ] **Verify 3 — open capture page and speak** + +On any device on the LAN, open: `http://192.168.86.192:8310/capture?room=Kitchen` + +Allow microphone access, speak a sentence. Check dashboard at `http://192.168.86.192:8310`: + +```bash +# Or check directly: +curl -s "http://localhost:8310/api/utterances" | python3 -m json.tool | head -40 +``` + +Expected: At least one utterance with a transcript appears within a few seconds. + +- [ ] **Verify 4 — speaker labeling** + +In the dashboard Speakers tab, click "Review now", tag the unknown utterance as your name. Speak again. Confirm the next utterance resolves to your name automatically. + +- [ ] **Verify 5 — search** + +Search for a word you spoke. Confirm it appears with keyword highlighting. + +- [ ] **Verify 6 — Ask AI** + +Ask "What did we talk about today?" in the Ask AI tab. Confirm Ollama returns a coherent response. + +- [ ] **Verify 7 — mute** + +Click mute on the capture page. Speak. Confirm no new utterances appear in the log. Unmute and speak again — utterance appears. + +- [ ] **Verify 8 — Traefik route** + +```bash +curl -s https://linkedstorm.hobbs.farm/health +``` + +Expected: `{"status":"ok"}` (may take a minute for Let's Encrypt cert) + +- [ ] **Verify 9 — manual cron test** + +```bash +docker exec linkedstorm-cron python -c " +from app.shared.config import get_settings +from app.shared.database import get_db +from app.cron.daily_summary import generate_daily_summary +import datetime +s = get_settings() +db = get_db(s) +generate_daily_summary(datetime.date.today().isoformat(), s, db) +print('Done') +" +``` + +Expected: `Done` (if Ollama reachable, a row appears in daily_summaries) + +- [ ] **Verify 10 — add Prometheus probe** (in docker repo) + +```bash +cd /mnt/c/Users/Jeremy/projects/docker +``` + +Edit `compose/hf-pdocker-01/monitoring/prometheus.yml` — add to the `http-pdocker01` job's `static_configs` targets: + +```yaml +- http://192.168.86.192:8310/health +``` + +Then hot-reload Prometheus: + +```bash +ssh glyptic@192.168.86.192 "curl -s -X POST http://localhost:9095/-/reload" +``` + +- [ ] **Final commit to docker repo** + +```bash +cd /mnt/c/Users/Jeremy/projects/docker +git add compose/hf-pdocker-01/monitoring/ +git commit -m "monitoring: add Prometheus probe for linkedstorm-api" +git push origin master +``` + +--- + +## Self-Review Notes + +**Spec coverage check:** +- ✅ 3 containers: pipeline (Task 7), api (Tasks 8-9, 12-13), cron (Task 10) +- ✅ Browser capture page with WebSocket + mute (Task 11) +- ✅ Silero VAD (Task 5) +- ✅ Whisper STT (Task 6, processor.py) +- ✅ resemblyzer voice embedding + cosine similarity (Task 4) +- ✅ Three-tier matching (Task 4, recognition.py) +- ✅ SQLite schema + FTS5 (Task 3) +- ✅ Speaker labeling workflow — tag endpoint (Task 8, utterances.py) +- ✅ Daily log timeline (Task 12, app.js) +- ✅ Full-text search (Task 9, search.py + Task 12) +- ✅ RAG via Ollama (Task 9, rag.py + Task 12) +- ✅ Room status + mute relay (Task 9, rooms.py + Task 12) +- ✅ Nightly daily summary (Task 10) +- ✅ Unknown speaker clustering (Task 10) +- ✅ 30-day clip purge (Task 10) +- ✅ Docker Compose (Task 14) +- ✅ Traefik route (Task 14) +- ✅ Prometheus probe (Task 15) +- ⚠️ LiveKit publishing from pipeline — simplified: capture page connects directly to LiveKit via JS SDK for live monitoring (future work, out of scope for v1). Replay uses stored WAV clips via /api/utterances/{id}/clip. + +**Type consistency:** `find_best_speaker_match` returns `tuple[str, int | None, float | None]` — used consistently in recognition.py (Task 4) and processor.py (Task 6). `pack_embedding`/`unpack_embedding` used in recognition.py, processor.py, and utterances.py — same import path `app.pipeline.recognition` throughout. diff --git a/docs/superpowers/specs/2026-03-11-rename-bart-design.md b/docs/superpowers/specs/2026-03-11-rename-bart-design.md new file mode 100644 index 0000000..ab636d1 --- /dev/null +++ b/docs/superpowers/specs/2026-03-11-rename-bart-design.md @@ -0,0 +1,89 @@ +# Rename bart → hf-pdocker-03 Design + +**Date:** 2026-03-11 +**Server:** 192.168.86.167 (currently "bart", becoming "hf-pdocker-03") + +## Overview + +Rename the server "bart" to "hf-pdocker-03" across the entire infrastructure — hostname, repository files, NAS paths, Docker Swarm node, monitoring, and documentation. The IP address 192.168.86.167 does not change. + +## Scope + +### Server-Level + +- Change hostname from `bart` to `hf-pdocker-03` via `hostnamectl` +- Update `/etc/hosts` on the server itself +- Docker Swarm: drain → remove → change hostname → re-join with new name +- Swarm global services (Telegraf, Node-exporter) auto-reschedule on new node name + +### NAS Path Rename + +- `/mnt/docker-data/compose/bart/` → `/mnt/docker-data/compose/hf-pdocker-03/` +- **Onyx exception:** `compose/hf-pdocker-03/onyx/docker-compose.yml` has hardcoded volume mounts referencing `/mnt/docker-data/compose/bart/onyx`. These must be updated in the compose file AND the Onyx containers redeployed after the NAS rename. +- All other running containers are unaffected (they don't reference the compose file path at runtime) + +### Repository Files + +**Directory rename:** +- `compose/bart/` → `compose/hf-pdocker-03/` (14 files across 8 subdirectories) + - Active services: `media/`, `openvas/`, `watchtower/`, `onyx/` + - Stale failover dirs (preserved for reference): `smarthome/`, `matrix/`, `guacamole/`, `ai-agents/` + +**File inventory (all files containing "bart"):** + +| File | Type | Action | +|------|------|--------| +| `CLAUDE.md` | Config | Must update — service distribution, storage, internal services | +| `stacks/monitoring/prometheus.yml` | Config | Must update — hostname label, job name `http-bart` → `http-hf-pdocker-03` | +| `stacks/monitoring/alerts.yml` | Config | Must update — PromQL `hostname="bart"` matchers. Review: elevated 90% CPU threshold was for Frigate, which is now on hf-pdocker-01, not this node. Consider reverting to standard 80% threshold. | +| `stacks/monitoring/matrix-receiver-config.yml` | Config | Must update — "bart" in comment. Note: file appears stale (still references old IPs). Update comment only. | +| `stacks/frigate/docker-compose.yml` | Stale Swarm def | Update for consistency — has `node.hostname == bart` placement constraint. This is an unused Swarm stack (Frigate runs as standalone Compose on hf-pdocker-01). | +| `compose/bart/onyx/docker-compose.yml` | Active compose | **CRITICAL** — has hardcoded volume mount `/mnt/docker-data/compose/bart/onyx:/mnt/gitea_patch:ro`. Must update path AND redeploy Onyx containers. | +| `.claude/agents/gitea-backup.md` | Agent def | Must update | +| `.claude/agents/alert-manager.md` | Agent def | Must update | +| `.claude/agents/documentation-keeper.md` | Agent def | Must update | +| `DEPLOYMENT_SUMMARY.md` | Doc | Must update | +| `docs/plans/2026-03-08-hf-pdocker-01-phase2-migration.md` | Historical | Update for consistency | +| `docs/plans/2026-03-08-hf-pdocker-01-rebuild.md` | Historical | Update for consistency | +| `docs/plans/2026-03-07-frigate-notify-v2-plan.md` | Historical | Update for consistency | +| `docs/plans/2026-03-05-farm-manager.md` | Historical | Update for consistency | +| `docs/plans/2026-03-05-farm-manager-design.md` | Historical | Update for consistency | + +**Memory/auto-memory files:** +- `MEMORY.md` — ~30 references to "bart" +- `memory/migration-details.md` — bart references + +### Documentation + +- Wiki pages — infrastructure overview, service access, changelog + +## Order of Operations + +1. **Repo file updates** — rename `compose/bart/` directory, update all file contents (including Onyx compose volume path). No service impact yet. +2. **SCP updated Onyx compose file** to NAS at the current path (`/mnt/docker-data/compose/bart/onyx/docker-compose.yml`) with the NEW volume path (`/mnt/docker-data/compose/hf-pdocker-03/onyx`). This prepares for the rename. +3. **NAS path rename** — `mv /mnt/docker-data/compose/bart /mnt/docker-data/compose/hf-pdocker-03` (via SSH). Atomic on NFS. +4. **Redeploy Onyx** — `cd /mnt/docker-data/compose/hf-pdocker-03/onyx && docker compose up -d` to pick up the new volume mount path. +5. **Server hostname change** — `sudo hostnamectl set-hostname hf-pdocker-03` + update `/etc/hosts` on 192.168.86.167. +6. **Docker Swarm re-join** — on hf-pdocker-01: drain old bart node, remove it. On 192.168.86.167: leave swarm, re-join with new hostname. Global services auto-schedule. ~30s gap. +7. **Prometheus + alerts** — SCP updated prometheus.yml and alerts.yml, hot-reload. +8. **Update /etc/hosts on other nodes** — if hf-pdocker-01 or hf-pdocker-02 have "bart" in /etc/hosts, update to hf-pdocker-03. +9. **MEMORY.md + Wiki** — update all documentation. +10. **Commit + push to Gitea**. + +## Risk Assessment + +- **Low risk:** Running Compose services (Jellyfin, Sonarr, Radarr, etc.) are unaffected — containers keep running. +- **Onyx restart required:** Onyx containers need redeployment after NAS rename to pick up new volume mount path. Brief downtime for Onyx (~30s). +- **Low risk:** Swarm global services have ~30s gap during node re-join. +- **No IP changes:** All service-to-service communication uses IPs, not hostnames. Nothing breaks at the network level. +- **NAS path rename is atomic:** `mv` on NFS is atomic for same-filesystem renames. + +## Verification + +After completion: +- `hostname` on 192.168.86.167 returns `hf-pdocker-03` +- `docker node ls` on hf-pdocker-01 shows `hf-pdocker-03` instead of `bart` +- All containers on 192.168.86.167 still running (`docker ps`) +- Onyx containers running with new volume mount path +- Prometheus targets for hf-pdocker-03 are UP +- `grep -r bart` in repo returns zero matches (excluding git history) diff --git a/stacks/traefik/dynamic/gitea.yml b/stacks/traefik/dynamic/gitea.yml index d2d5c4f..e070532 100644 --- a/stacks/traefik/dynamic/gitea.yml +++ b/stacks/traefik/dynamic/gitea.yml @@ -4,6 +4,8 @@ http: rule: "Host(`git.hobbs.farm`)" entryPoints: - websecure + middlewares: + - security-headers service: gitea tls: certResolver: letsencrypt diff --git a/stacks/traefik/dynamic/openwebui.yml b/stacks/traefik/dynamic/openwebui.yml deleted file mode 100644 index 9053383..0000000 --- a/stacks/traefik/dynamic/openwebui.yml +++ /dev/null @@ -1,26 +0,0 @@ -http: - routers: - openwebui: - rule: "Host(`open.hobbs.farm`)" - entryPoints: - - websecure - service: openwebui - tls: - certResolver: letsencrypt - openwebui-http: - rule: "Host(`open.hobbs.farm`)" - entryPoints: - - web - middlewares: - - openwebui-https-redirect - service: openwebui - middlewares: - openwebui-https-redirect: - redirectScheme: - scheme: https - permanent: true - services: - openwebui: - loadBalancer: - servers: - - url: "http://192.168.86.192:3010" diff --git a/stacks/traefik/dynamic/security-headers.yml b/stacks/traefik/dynamic/security-headers.yml new file mode 100644 index 0000000..0000216 --- /dev/null +++ b/stacks/traefik/dynamic/security-headers.yml @@ -0,0 +1,60 @@ +http: + middlewares: + security-headers: + headers: + # HSTS - force browsers to use HTTPS for 1 year, include subdomains + stsSeconds: 31536000 + stsIncludeSubdomains: true + stsPreload: true + # Prevent MIME-type sniffing + contentTypeNosniff: true + # Block rendering in iframes from other origins + frameDeny: true + # Referrer policy - send origin only on cross-origin requests + referrerPolicy: "strict-origin-when-cross-origin" + # Permissions policy - restrict browser features + permissionsPolicy: "camera=(), microphone=(), geolocation=(), payment=()" + # Prevent browsers from caching sensitive responses + browserXssFilter: true + # Remove server identity header + customResponseHeaders: + X-Powered-By: "" + Server: "" + + # Variant that allows microphone (for VoiceCoach) + security-headers-allow-mic: + headers: + stsSeconds: 31536000 + stsIncludeSubdomains: true + stsPreload: true + contentTypeNosniff: true + frameDeny: true + referrerPolicy: "strict-origin-when-cross-origin" + permissionsPolicy: "camera=(), microphone=(self), geolocation=(), payment=()" + browserXssFilter: true + customResponseHeaders: + X-Powered-By: "" + Server: "" + + # BasicAuth for sensitive services (Guacamole) + solo-basicauth: + basicAuth: + users: + - "glyptic:$2y$05$6AcmRIjMX6YrxY3fACeDhui4ghBGZtjmiPHbzWDb.qZ.8Fdip4GM." + realm: "Restricted" + + # Variant that allows framing (for Guacamole, Home Assistant embeds) + security-headers-allow-frames: + headers: + stsSeconds: 31536000 + stsIncludeSubdomains: true + stsPreload: true + contentTypeNosniff: true + frameDeny: false + customFrameOptionsValue: "SAMEORIGIN" + referrerPolicy: "strict-origin-when-cross-origin" + permissionsPolicy: "camera=(), microphone=(), geolocation=(), payment=()" + browserXssFilter: true + customResponseHeaders: + X-Powered-By: "" + Server: "" diff --git a/stacks/traefik/dynamic/tls-options.yml b/stacks/traefik/dynamic/tls-options.yml new file mode 100644 index 0000000..6df92f9 --- /dev/null +++ b/stacks/traefik/dynamic/tls-options.yml @@ -0,0 +1,18 @@ +tls: + options: + default: + minVersion: VersionTLS12 + cipherSuites: + # TLS 1.3 ciphers are always enabled and not configurable + # These are the strong TLS 1.2 ciphers + - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 + - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 + - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 + - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 + - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305 + - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 + curvePreferences: + - X25519 + - CurveP256 + - CurveP384 + sniStrict: true diff --git a/stacks/traefik/dynamic/vaultwarden.yml b/stacks/traefik/dynamic/vaultwarden.yml index f99021a..16725db 100644 --- a/stacks/traefik/dynamic/vaultwarden.yml +++ b/stacks/traefik/dynamic/vaultwarden.yml @@ -4,6 +4,8 @@ http: rule: "Host(`vault.hobbs.farm`)" entryPoints: - websecure + middlewares: + - security-headers service: vault tls: certResolver: letsencrypt diff --git a/vllm-hermes-deployment-brief.md b/vllm-hermes-deployment-brief.md new file mode 100644 index 0000000..fd321a6 --- /dev/null +++ b/vllm-hermes-deployment-brief.md @@ -0,0 +1,215 @@ +# Deployment Brief: vLLM serving Hermes 4.3 for OpenClaw/Pluto + +**Audience:** Claude Code, executing autonomously on Jeremy's homelab. +**Goal:** Stand up a vLLM OpenAI-compatible inference server hosting NousResearch Hermes 4.3, with native Hermes tool-call parsing, and wire it into the existing OpenClaw gateway as Pluto's reasoning model. + +--- + +## 0. Why this exists (context you need) + +We tried serving Hermes 4.3 36B (Q4_K_M GGUF) through Ollama on the Windows box. The model loads and runs at ~98% GPU after forcing `num_gpu 64`, but **structured tool calling fails** — Ollama returns empty `content` and no `tool_calls` array, because Ollama's parser doesn't extract Hermes's `` XML format into the OpenAI-structured field on a self-imported GGUF. This is a known, recurring Ollama sharp edge. + +vLLM solves this directly: the `--tool-call-parser hermes` flag parses the XML `` format natively into the OpenAI-compatible response schema, no post-processing. That is the entire reason for this migration. + +**Do not** spend effort trying to fix the Ollama path. The decision to move to vLLM is final. + +--- + +## 1. Hard constraints — read before doing anything + +1. **The GPU is in the Windows desktop, NOT the Linux swarm.** + - Host: `DESKTOP-5AHPRKR` — ASUS Z790 (TUF GAMING Z790-PLUS WIFI), i7-13700KF, 32GB RAM, **RTX 3090 (24GB VRAM)**. + - The Docker Swarm nodes (`hf-pdocker-01`, `hf-pdocker-02`, `bart`) are CPU-only and **cannot** run a 36B model usefully. Do not attempt to schedule vLLM onto the swarm. + - vLLM is **Linux/CUDA only**. On Windows this means running inside **WSL2** (Ubuntu) with CUDA passthrough, using the official vLLM Docker image via Docker Desktop's WSL2 backend, OR a native WSL2 Python install. Docker image is preferred for reproducibility. + +2. **24GB VRAM is the binding limit.** Hermes 4.3 36B in FP16 is ~72GB — impossible here. Options, in order of preference: + - **(A)** A pre-quantized **AWQ or GPTQ 4-bit** build of Hermes 4.3 36B (~18-20GB weights). Tight but workable with `--gpu-memory-utilization 0.95`, `--max-model-len 8192`, and `--enforce-eager` (skip CUDA graph capture to save VRAM). **First choice if a quant exists.** + - **(B)** If no 36B AWQ/GPTQ exists, fall back to **Hermes 4.3 in a smaller parameter size** if Nous published one (e.g. a 14B), served in FP16 or AWQ — fits comfortably, full context, fast. **Strongly preferred over fighting the 36B on 24GB.** + - **(C)** Last resort: serve the **GGUF Q4** through vLLM (vLLM supports GGUF). Weaker than AWQ in vLLM and partially defeats the point, but the hermes parser still works. Only if A and B are unavailable. + +3. **WSL2 will fight you for RAM.** Jeremy's box already had `VmmemWSL` consuming ~13GB. There may now be a `C:\Users\Jeremy\.wslconfig` capping it. Check it. vLLM's model loading stages through system RAM, so ensure WSL2's memory cap is high enough (≥24GB) for the load, OR raise it temporarily. Coordinate: Docker Desktop restarts WSL2 automatically, so don't rely on `wsl --shutdown` to free RAM permanently. + +4. **Stop the Ollama server before testing** to avoid port/VRAM contention. Ollama holds VRAM while a model is resident. `ollama stop hermes` and confirm `ollama ps` is empty, or quit the Ollama app entirely. vLLM and Ollama cannot both hold the 3090 at once. + +--- + +## 2. Pre-flight verification (do these first, report results) + +Run inside WSL2 (Ubuntu): + +```bash +# Confirm CUDA reaches WSL2 and sees the 3090 +nvidia-smi +# Expect: "NVIDIA GeForce RTX 3090", 24576 MiB total, driver present. +# If this fails, CUDA-on-WSL2 isn't set up — STOP and report. + +# Confirm Docker can see the GPU +docker run --rm --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi +# Expect same output from inside a container. +``` + +If `nvidia-smi` works on the Windows side but not in WSL2, the fix is installing the WSL2 CUDA toolkit (NOT a full driver — the Windows driver provides the WSL2 GPU stub). Report before proceeding so Jeremy can confirm. + +Also check current RAM headroom and the WSL config: + +```bash +cat /mnt/c/Users/Jeremy/.wslconfig 2>/dev/null || echo "no .wslconfig" +free -h +``` + +--- + +## 3. Model selection (decide, then download) + +**Step 3a — Find the best-fitting quant.** Search HuggingFace for, in priority order: +1. `Hermes-4.3-36B` AWQ (search orgs: NousResearch, and community quantizers like `casperhansen`, `solidrust`, `bartowski` for AWQ/GPTQ variants) +2. A smaller Hermes 4.3 (14B-class) if one exists — this is genuinely the easier, faster path on 24GB +3. GGUF Q4 as fallback (we already have `hf.co/NousResearch/Hermes-4.3-36B-GGUF:Q4_K_M`) + +**Report what you find and which option (A/B/C from §1.2) you're proceeding with before downloading 20GB.** If a 36B AWQ exists, confirm its repo name. If the only realistic fit is a smaller Hermes 4.3, say so — Jeremy explicitly prefers a model that runs 100% on-GPU with full context over grinding a 36B into a too-small card. + +**Step 3b — Download to a persistent WSL2 path** (not the container, so it survives restarts): + +```bash +mkdir -p ~/models +pip install -U "huggingface_hub[cli]" +huggingface-cli download --local-dir ~/models/hermes-4.3 +``` + +--- + +## 4. Launch vLLM (Docker, OpenAI-compatible) + +Create `~/vllm-hermes/docker-compose.yml`. Template for **Option A (36B AWQ)** — adjust `--quantization`, `--model`, and `--max-model-len` per the resolved model: + +```yaml +services: + vllm-hermes: + image: vllm/vllm-openai:latest + container_name: vllm-hermes + runtime: nvidia + deploy: + resources: + reservations: + devices: + - driver: nvidia + count: all + capabilities: [gpu] + ports: + - "8000:8000" + volumes: + - ~/models/hermes-4.3:/models/hermes-4.3 + ipc: host + command: > + --model /models/hermes-4.3 + --served-model-name hermes + --quantization awq + --gpu-memory-utilization 0.95 + --max-model-len 8192 + --enforce-eager + --enable-auto-tool-choice + --tool-call-parser hermes + --host 0.0.0.0 + --port 8000 + restart: unless-stopped +``` + +Key flag rationale: +- `--enable-auto-tool-choice --tool-call-parser hermes` — **the whole point.** Native Hermes XML tool-call parsing into OpenAI schema. +- `--quantization awq` — set to `gptq` for GPTQ, omit entirely for GGUF (use `--quantization gguf` and point `--model` at the .gguf file) or for an unquantized smaller model. +- `--enforce-eager` — disables CUDA graph capture, saving ~1-3GB VRAM. **Critical on the tight 24GB 36B path.** Remove it if you have headroom (smaller model) for a latency improvement. +- `--max-model-len 8192` — keep KV cache small to fit. Raise cautiously if VRAM allows; watch for OOM on startup. +- `--gpu-memory-utilization 0.95` — aggressive, appropriate for a dedicated inference box. Drop to 0.90 if the desktop also drives a display off the 3090 (it does — there's a monitor at 1680x1050 on it; budget ~0.5-1GB for the desktop, so 0.90 may be safer). + +Launch and watch logs: + +```bash +cd ~/vllm-hermes +docker compose up -d +docker compose logs -f vllm-hermes +# Wait for: "Application startup complete" and "Uvicorn running on http://0.0.0.0:8000" +``` + +**If it OOMs on startup:** lower `--max-model-len` to 4096, confirm `--enforce-eager` is present, drop `--gpu-memory-utilization` to 0.90. If it still won't fit, that's the signal Option A is not viable on this card — fall back to Option B (smaller Hermes 4.3) and report. + +--- + +## 5. Verification — the test that actually matters + +This is the exact test that FAILED on Ollama. It must pass on vLLM. + +```bash +curl -s http://localhost:8000/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "hermes", + "messages": [{"role": "user", "content": "What is 8347 multiplied by 291? Use the calculator tool."}], + "tools": [{ + "type": "function", + "function": { + "name": "calculator", + "description": "Multiply two numbers", + "parameters": { + "type": "object", + "properties": {"a": {"type": "number"}, "b": {"type": "number"}}, + "required": ["a", "b"] + } + } + }], + "tool_choice": "auto" + }' | python3 -m json.tool +``` + +**PASS criteria:** the response `choices[0].message` has a populated `tool_calls` array containing `function.name == "calculator"` and `function.arguments` parsing to `{"a": 8347, "b": 291}` (or equivalent), with `content` empty/null. That is structured tool calling working. + +**FAIL:** if `tool_calls` is null/absent and the JSON is dumped in `content` as a string, the parser flag didn't take — verify `--tool-call-parser hermes` is in the running command (`docker compose logs` shows the args at boot) and that the model's chat template actually emits `` tags. Report the raw response. + +Also sanity-check throughput: + +```bash +# Plain generation latency +curl -s http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" \ + -d '{"model":"hermes","messages":[{"role":"user","content":"Say hello in one sentence."}]}' \ + | python3 -m json.tool +nvidia-smi # confirm VRAM occupancy and that it's the vLLM process +``` + +--- + +## 6. Wire into OpenClaw (Pluto's brain) + +OpenClaw is model-agnostic and speaks the OpenAI API. Point it at the vLLM endpoint. The vLLM server is on the Windows box at `http://:8000/v1` (or `http://localhost:8000/v1` if OpenClaw runs on the same Windows box; confirm where Pluto's gateway actually runs — it may be on the swarm). + +In OpenClaw's model/provider config: +- **Base URL:** `http://:8000/v1` +- **Model name:** `hermes` (matches `--served-model-name`) +- **API key:** any non-empty string (vLLM ignores it by default; use `EMPTY` or `sk-noauth`) + +**Networking notes:** +- vLLM in the WSL2 container binds `0.0.0.0:8000`, but WSL2 networking means the port must be reachable from the Windows host and the LAN. WSL2 usually auto-forwards localhost; for LAN access you may need a `netsh interface portproxy` rule on Windows forwarding the host's LAN IP:8000 to the WSL2 IP:8000, plus a Windows Firewall inbound rule for port 8000 **scoped to the LAN subnet only**. Set this up and verify from another machine with `curl http://:8000/v1/models`. +- Do **not** expose port 8000 to the internet. If remote access is wanted later, that goes behind Traefik with forward-auth on the swarm, tunneled to the Windows box — a separate task, not part of this brief. + +After wiring, restart the OpenClaw gateway and run a Pluto interaction that should trigger a tool (whatever real tools Pluto has registered) to confirm end-to-end tool execution, not just the synthetic calculator test. + +--- + +## 7. Operational notes / gotchas + +- **Display + inference share the 3090.** The desktop's monitor runs off the same card. Under VRAM pressure this can cause display glitches or vLLM OOM. The `0.90` utilization setting accounts for this; if Jeremy ever runs the box headless (RDP only, which the earlier session showed — there was a Remote Display Adapter), more VRAM frees up and utilization can go to 0.95+. +- **vLLM holds VRAM for its lifetime.** It does not unload like Ollama. That's expected and desirable for an always-on Pluto brain, but it means the 3090 is committed — no gaming / other GPU work while it runs. +- **Auto-start on boot:** `restart: unless-stopped` handles container restarts, but WSL2 + Docker Desktop must themselves start on Windows login. For true always-on, enable Docker Desktop "Start on login" and consider WSL2 boot via Task Scheduler, OR (better long-term) move this whole stack to a dedicated headless Linux box with the 3090 if one becomes available. Flag this to Jeremy as a follow-up if always-on reliability matters for VoiceCoach. +- **Context length is capped at 8192 here** for VRAM reasons. Hermes 4.3 supports far more (trained to 512K), but you can't afford the KV cache on 24GB. If Pluto workflows need longer context, that's a hardware conversation (bigger card / second GPU), not a config tweak. +- **Version pinning:** `vllm/vllm-openai:latest` is convenient but pin to a specific tag for production stability once verified working, so an image update doesn't silently break the tool parser. + +--- + +## 8. Report back to Jeremy with + +1. `nvidia-smi` output from inside WSL2 (GPU visible y/n). +2. Which model option (A/B/C) you proceeded with and the exact repo ID. +3. The full JSON from the §5 calculator tool-call test (the pass/fail evidence). +4. Measured tokens/sec and VRAM occupancy from `nvidia-smi`. +5. Whether OpenClaw end-to-end tool execution worked with the real Pluto toolset. +6. Any networking steps needed for LAN access (portproxy/firewall rules added). + +Do not mark this complete until the §5 tool-call test passes with a structured `tool_calls` array. That single result is the definition of done for the inference layer.