Compare commits

17 Commits

Author SHA1 Message Date
jeremy e9a98f4855 chore: pin jellyfin to 2026060106
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-03 21:37:46 -05:00
jeremy 97e3cf12a7 frigate: sync config to live state, fix /tmp/cache size, enable VAAPI on house_east/pool_room, throttle shop_west detection
- docker-compose.yml: increase /tmp/cache tmpfs from 4G to 8G, add Coral USB
  device passthrough (/dev/bus/usb), fix memswap_limit to 16G (was equal to mem_limit)
- config.yml: replace OpenVINO detector with Coral edgetpu (usb), update model
  dimensions to 320x320 and pixel format to rgb, enable VAAPI on house_east and
  pool_room, throttle shop_west to 2fps with stricter person/car filters (min_area
  + threshold), add offline note for gate_camera, enable semantic_search (large),
  face_recognition (small), LPR, and bird classification

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-01 15:07:36 -05:00
jeremy 06e9db70d2 feat: replace legacy Unifi with UniFi OS Server v5.1.15
Deploys ghcr.io/lemker/unifi-os-server replacing linuxserver/unifi-network-application.
Data dirs moved to /var/lib/unifi-os/ locally (NFS root_squash blocks chown needed by
postgres/mongodb/nginx init). Only /persistent remains on NAS for config backups.
Access: https://192.168.86.100:11443

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-31 20:36:02 -05:00
jeremy b3bef75981 feat: VoiceCoach v2 — livekit-agents 1.5 personal voice assistant
Replaces pipecat 0.0.48 (6 manual patches, DuplicateIdentity reconnect storms)
with livekit-agents 1.5 SDK. New features:
- SvelteKit UI with live transcript, VAD ring, mode bar, vocal meter, history view
- SQLite session store with WAL mode (cross-session memory)
- Async SpeechBrain vocal/emotion analysis injected per turn
- Homelab tools: SSH/docker, Prometheus alerts, Home Assistant, SearXNG search
- OpenClaw LLM gateway preserved (qwen2.5:14b via Ollama)
- 4 modes: coaching, vent, practice, briefing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-31 18:05:30 -05:00
jeremy 55f05a88d0 infra: migrate GPU workstation to Ubuntu, deploy vLLM-Hermes + voice stack
- Migrated primary workstation (192.168.86.27) from Windows/WSL2 to Ubuntu 24.04 (2026-05-28)
- Upgraded Ollama 0.15.6 → 0.24.0 (all 21 models preserved)
- Fixed Watchtower crash loop (DOCKER_API_VERSION=1.44)
- Fixed Open WebUI OLLAMA_BASE_URL to use Docker service name
- Deployed Speaches :8000 (CUDA, STT/TTS for VoiceCoach)
- Deployed LiveKit :7880/:7881 (WebRTC for VoiceCoach)
- Deployed vLLM-Hermes :8001 (Hermes 4.3 36B AWQ, tool-call-parser hermes)
  - Endpoint: http://192.168.86.27:8001/v1, model: hermes
  - Tool-call test passed: structured tool_calls array confirmed
- Added vllm-local provider to voicecoach-openclaw and margaret OpenClaw configs
- Added OPENAI_API_BASE_URL to Open WebUI for Hermes access in chat UI
- Updated CLAUDE.md: removed WSL2 references, added GPU workstation section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-28 18:48:44 -05:00
jeremy 98cae1e08f feat: linkedstorm docker-compose and Traefik route
Add compose/hf-pdocker-01/linkedstorm/ with 3-container compose (pipeline :8300, api :8310, cron) and stacks/traefik/dynamic/linkedstorm.yml routing linkedstorm.hobbs.farm -> :8310.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-28 10:54:53 -05:00
jeremy cd3f3161da docs: add LinkedStorm home audio assistant design spec
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 21:59:32 -05:00
jeremy 904d1ef397 fix: onyx nginx - use rewrite to strip /api/ prefix with variable proxy_pass
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 13:58:30 -05:00
jeremy a7dfbdeb79 fix: onyx nginx - use Docker DNS resolver to prevent stale upstream IPs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 13:30:42 -05:00
jeremy 6233cf9e1d fix: disable Onyx telemetry (stops data posting to telemetry.onyx.app) 2026-05-22 09:00:46 -05:00
jeremy 0d5b781aef fix: onyx nginx - strip /api prefix when proxying to api_server
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 08:30:37 -05:00
jeremy 6f485ff6e5 docs: update CLAUDE.md — Coral USB TPU, Onyx redeploy, infra fixes 2026-05-20
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-20 08:06:05 -05:00
jeremy 752b8230ed infra: clean redeploy of Onyx on bart with latest images
Full wipe and fresh start: removed gitea_patch/unstructured_patch
hacks from startup commands, upgraded all onyxdotapp images from
nightly-latest-20260331 pin to :latest, removed UNSTRUCTURED_SERVER_URL
(service decommissioned 2026-04-14), added Watchtower disable labels
to ALL services (was missing on web_server, model_servers, db, cache,
opensearch, vespa, nginx), added nginx.conf for reverse proxy.

Alembic ran 349 migrations cleanly on fresh DB. All 10 containers up,
0 restarts, HTTP 200 at :3005.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-20 07:12:03 -05:00
jeremy 03b7a46a96 fix: pin guacamole/guacd to 1.6.0 and disable Watchtower to prevent schema breakage
Watchtower pulled a newer guacamole:latest image whose JDBC extension expected
tables not present in the existing DB (guacamole_user_attribute etc.). Fixed by
reinitializing the DB with the official 1.6.0 schema (001-create-schema.sql +
002-create-admin-user.sql) and pinning both guacamole and guacd to 1.6.0 with
com.centurylinklabs.watchtower.enable=false to prevent recurrence.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-20 06:44:40 -05:00
jeremy 78b824bb39 fix: pin gitea to 1.26.1 and disable Watchtower to prevent broken image pulls
Watchtower pulled gitea/gitea:latest three times (Mar 12, Apr 19, Apr 25)
and at least one pull resulted in a broken image with git binary failures.
Pin to stable gitea/gitea:1.26.1 and add watchtower.enable=false label
so the image is only updated by deliberate human action.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-16 16:17:15 -05:00
jeremy a908973264 infra: rongorongo state-sync backend + Traefik basicAuth
Add a Flask backend service for cross-device state persistence, storing
JSON state on an NFS volume at /mnt/docker-data/configs/rongorongo/data.
Frontend Dockerfile/nginx.conf reorganised into frontend/ subdirectory;
frontend API calls now route to /api/* handled by the backend container
over a dedicated bridge network. Traefik basicAuth middleware added to
the public URL with the bcrypt hash deliberately kept out of git
(REPLACE_AT_DEPLOY placeholder in rongorongo.yml). Prometheus probe moved
from https-external to internal http://192.168.86.192:8181/health to
bypass the new auth wall.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-07 18:51:47 -05:00
jeremy 795908e1ef infra: deploy Rongorongo decipherment engine to hf-pdocker-01
Standalone Compose service running a locally-built nginx image that
serves a static Anthropic-API frontend. Traefik dynamic route added
for public HTTPS access. Blackbox probe added to prometheus.yml for
uptime monitoring. CLAUDE.md updated with service entry.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-06 13:10:47 -05:00
47 changed files with 4760 additions and 167 deletions
+70 -71
View File
@@ -4,14 +4,14 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
## Repository Overview ## Repository Overview
This repository manages a **homelab infrastructure** with self-hosted services running across 3 nodes using a **hybrid orchestration** model: Docker Swarm for core infrastructure services, standalone Docker Compose for isolated services. This repository manages a **homelab infrastructure** with self-hosted services running across 3 nodes using **standalone Docker Compose** on all nodes. (Docker Swarm was fully dismantled on 2026-04-01.)
### Cluster Architecture ### Cluster Architecture
**Nodes:** **Nodes:**
- **hf-pdocker-01** (192.168.86.192) - Swarm Manager, core infrastructure + Frigate NVR - **RESTORED (2026-03-08)** - **hf-pdocker-01** (192.168.86.192) - Core infrastructure + Frigate NVR - **RESTORED (2026-03-08)**
- **hf-pdocker-02** (192.168.86.100) - Standalone Compose services (dev tools, utilities) - **hf-pdocker-02** (192.168.86.100) - Dev tools, utilities
- **bart** (192.168.86.167) - Standalone Compose services (media stack, OpenVAS) - **bart** (192.168.86.167) - Media stack, OpenVAS
**Synology NAS** (192.168.86.30) - Centralized NFS storage for all service data **Synology NAS** (192.168.86.30) - Centralized NFS storage for all service data
@@ -20,47 +20,62 @@ This repository manages a **homelab infrastructure** with self-hosted services r
### Service Distribution ### Service Distribution
``` ```
hf-pdocker-01 (Manager, 23GB RAM, i7-7700T, NVMe, HD 630 GPU) -- RESTORED (2026-03-08, hardware repaired) hf-pdocker-01 (23GB RAM, i7-7700T, NVMe, HD 630 GPU) -- RESTORED (2026-03-08, hardware repaired)
├── [COMPOSE] Frigate NVR (:5000) -- OpenVINO GPU detector, 13 cameras active ├── [COMPOSE] Frigate NVR (:5000) -- Coral USB TPU (edgetpu), ~11ms inference, 13 cameras active
├── [COMPOSE] Frigate-Notify v2 (:5199) -- MQTT->AI->Matrix notifications ├── [COMPOSE] Frigate-Notify v2 (:5199) -- MQTT->AI->Matrix notifications
├── [COMPOSE] clip-service (:5200) -- CLIP embeddings (ViT-L-14, CPU) ├── [COMPOSE] clip-service (:5200) -- CLIP embeddings (ViT-L-14, CPU)
├── [COMPOSE] Open WebUI (:3010) -- AI chat UI (Ollama + Anthropic backends)
├── [COMPOSE] Home Assistant (:8123) ├── [COMPOSE] Home Assistant (:8123)
├── [COMPOSE] ESPHome (:6052) ├── [COMPOSE] ESPHome (:6052)
├── [COMPOSE] Mosquitto (:1883, :8883) ├── [COMPOSE] Mosquitto (:1883, :8883)
├── [COMPOSE] Synapse/Matrix (:8008) + synapse-db (postgres) + wellknown (:8010) ├── [COMPOSE] Synapse/Matrix (:8008) + synapse-db (postgres) + wellknown (:8010)
├── [COMPOSE] Guacamole (:8199) + guacd + guacamole-db (mysql, LOCAL /var/lib/guacamole-db) ├── [COMPOSE] Guacamole (:8199) + guacd + guacamole-db (mysql, LOCAL /var/lib/guacamole-db)
├── [COMPOSE] AI Agents (margaret:18790, spirit:18795 [infra ops], clippy:18800, bob:18810, travis:18815) ├── [COMPOSE] AI Agents (margaret:18790, spirit:18795 [infra ops])
├── [COMPOSE] Traefik (reverse proxy, ports 80/443/8090) ├── [COMPOSE] Traefik (reverse proxy, ports 80/443/8090)
├── [COMPOSE] AdGuard Home (:3000, :53) ├── [COMPOSE] AdGuard Home (:3000, :53)
├── [COMPOSE] Monitoring (Grafana :3001, Prometheus :9095, Alertmanager :9093, InfluxDB :8086, Blackbox :9115, matrix-receiver :9094) ├── [COMPOSE] Monitoring (Grafana :3001, Prometheus :9095, Alertmanager :9093, InfluxDB :8086, Blackbox :9115, matrix-receiver :9094)
├── [COMPOSE] Farm Manager API (:8888) -- dashboard + API server ├── [COMPOSE] Farm Manager API (:8888) -- dashboard + API server
├── [COMPOSE] Farm Agent (:8889) ├── [COMPOSE] Farm Agent (:8889)
├── [SWARM] Telegraf (global), Node-exporter (global) ├── [COMPOSE] Rongorongo decipherment engine -- 2 containers (rongorongo-web :8181, rongorongo-api). Server-synced state at /mnt/docker-data/configs/rongorongo/data. Traefik basicAuth on rongorongo.hobbs.farm.
├── [REMOVED] VoiceCoach -- moved to GPU machine (192.168.86.27) 2026-04-04, now Ubuntu
├── [COMPOSE] Node-exporter (:9100) -- standalone with pid:host
├── [COMPOSE] Telegraf -- metrics collection to InfluxDB
├── [CRON] sync-frigate-to-nas (daily 2am), cleanup-frigate-media (weekly Sun 3am), restart-frigate (daily 4am) ├── [CRON] sync-frigate-to-nas (daily 2am), cleanup-frigate-media (weekly Sun 3am), restart-frigate (daily 4am)
├── 5TB USB disk at /mnt/frigate-recordings (Frigate recordings storage) ├── 5TB USB disk at /mnt/frigate-recordings (Frigate recordings storage)
bart (Worker, 15GB RAM, i5-4570T, HDD) bart (15GB RAM, i5-4570T, HDD)
├── [COMPOSE] Media Stack (Jellyfin, Sonarr, Radarr, SABnzbd, NZBHydra2, Jellyseerr) ├── [COMPOSE] Media Stack (Jellyfin, Sonarr, Radarr, SABnzbd, NZBHydra2, Jellyseerr)
├── [COMPOSE] Onyx (:3005) -- redeployed 2026-05-20, all services :latest, Watchtower disabled on all 10
├── [COMPOSE] OpenVAS (vulnerability scanner) ├── [COMPOSE] OpenVAS (vulnerability scanner)
├── [COMPOSE] Onyx (:3005) -- AI document search/RAG (Ollama backend: llava:13b on 192.168.86.196)
├── [COMPOSE] Farm Agent (:8889) ├── [COMPOSE] Farm Agent (:8889)
├── [COMPOSE] Node-exporter (:9100) -- standalone with pid:host
├── [COMPOSE] Telegraf -- metrics collection to InfluxDB
├── [COMPOSE] Watchtower ├── [COMPOSE] Watchtower
├── [SWARM] Telegraf (global), Node-exporter (global)
hf-pdocker-02 (Worker, 7.6GB RAM, i7-6700T, HDD) GPU workstation / Jeremy's PC (192.168.86.27, Ubuntu 24.04 — migrated from Windows 2026-05-28)
├── [COMPOSE] Unifi + unifi-db (MongoDB) ├── RTX 3090 (24GB VRAM), i7-13700KF, 32GB RAM
├── [COMPOSE] Ollama (:11434) -- 21 local models, v0.24.0
├── [COMPOSE] Open WebUI (:8080) -- AI chat, points to local Ollama
├── [REMOVED] vLLM-Hermes -- removed 2026-05-29, VRAM freed for Speaches CUDA
├── [COMPOSE] Speaches (:8000) -- STT/TTS via faster-whisper + Kokoro, CUDA
├── [COMPOSE] LiveKit (:7880/:7881) -- WebRTC for VoiceCoach, key: devkey
├── [COMPOSE] Portainer (:9000, :8002)
├── [COMPOSE] Prometheus (:9090), DCGM-exporter (:9400), node-exporter (:9100)
├── [COMPOSE] Watchtower (label-enable mode)
├── Compose file: /home/glyptic/Docker/compose/docker-compose.yml
├── vLLM compose: /home/glyptic/Docker/vllm-hermes/docker-compose.yml
└── Model cache: /home/glyptic/Docker/data/models/
hf-pdocker-02 (7.6GB RAM, i7-6700T, HDD)
├── [COMPOSE] UniFi OS Server (:11443 HTTPS, :8080 inform, :3478/udp STUN) -- ghcr.io/lemker/unifi-os-server v5.1.15, replaced legacy linuxserver/unifi 2026-05-31. Data local at /var/lib/unifi-os/, /persistent on NAS.
├── [COMPOSE] Gitea + gitea-db (PostgreSQL) ├── [COMPOSE] Gitea + gitea-db (PostgreSQL)
├── [COMPOSE] Semaphore + semaphore-db (MySQL) ├── [COMPOSE] Semaphore + semaphore-db (MySQL)
├── [COMPOSE] Wiki.js + wikijs-db (PostgreSQL) ├── [COMPOSE] Wiki.js + wikijs-db (PostgreSQL)
├── [COMPOSE] Vaultwarden, Actual Budget, MCPO, Trivy ├── [COMPOSE] Vaultwarden, Actual Budget, MCPO, Trivy
├── [COMPOSE] SearXNG (:8888) -- self-hosted meta-search engine (Onyx web search backend) ├── [COMPOSE] n8n + n8n-db (PostgreSQL) -- workflow automation
├── [COMPOSE] Unstructured API (:8000) -- document parsing for Onyx file ingestion
├── [COMPOSE] n8n + n8n-db (PostgreSQL) -- Clippy workflow automation
├── [COMPOSE] NetClaw/Aaron (:1179) -- AI network agent
├── [COMPOSE] Farm Agent (:8889) ├── [COMPOSE] Farm Agent (:8889)
├── [COMPOSE] Node-exporter (:9100) -- standalone with pid:host
├── [COMPOSE] Telegraf -- metrics collection to InfluxDB
├── [COMPOSE] Watchtower ├── [COMPOSE] Watchtower
├── [SWARM] Telegraf (global), Node-exporter (global)
``` ```
@@ -68,9 +83,11 @@ hf-pdocker-02 (Worker, 7.6GB RAM, i7-6700T, HDD)
### Environment Context ### Environment Context
You are working in a **WSL 2 environment** on Windows. Docker is NOT directly available in WSL - all Docker operations must be performed by connecting to the remote nodes via SSH. You are working on a **native Ubuntu 24.04 workstation** (previously WSL2 on Windows — migrated 2026-05-28). This machine IS the GPU inference box (RTX 3090, 192.168.86.27). Docker is available locally and runs the inference stack. All homelab node operations (hf-pdocker-01, hf-pdocker-02, bart) still require SSH.
**Critical:** Use the `docker-swarm-ops` agent (via Task tool) for ALL Docker operations. Do not attempt to run docker commands directly in bash - they will fail. **Local Docker on this machine** runs: Ollama, Open WebUI, vLLM-Hermes, Speaches, LiveKit, Portainer, Prometheus, DCGM-exporter.
**Critical:** For homelab nodes, use the `docker-swarm-ops` agent via SSH. For local inference services, Docker commands run directly.
### Storage Architecture ### Storage Architecture
@@ -84,24 +101,22 @@ All service data lives on the **Synology NAS** via NFS, mounted at `/mnt/docker-
├── traefik/ # Let's Encrypt certs ├── traefik/ # Let's Encrypt certs
└── compose/ # Standalone compose files per node └── compose/ # Standalone compose files per node
├── hf-pdocker-01/{frigate,frigate-notify,clip-service,matrix,smarthome,guacamole,ai-agents,traefik,adguard}/ ├── hf-pdocker-01/{frigate,frigate-notify,clip-service,matrix,smarthome,guacamole,ai-agents,traefik,adguard}/
├── hf-pdocker-02/{unifi,gitea,semaphore,wikijs,vaultwarden,actual-budget,mcpo,trivy,netclaw,watchtower}/ ├── hf-pdocker-02/{unifi,gitea,semaphore,wikijs,vaultwarden,actual-budget,mcpo,trivy,watchtower}/
└── bart/{media,openvas,watchtower}/ └── bart/{media,openvas,watchtower}/
``` ```
**NFS mount (all nodes):** `192.168.86.30:/volume1/Docker /mnt/docker-data nfs rw,soft,timeo=300,intr,_netdev,nfsvers=3 0 0` **NFS mount (all nodes):** `192.168.86.30:/volume1/Docker /mnt/docker-data nfs rw,soft,timeo=300,intr,_netdev,nfsvers=3,nofail,x-systemd.after=network-online.target,x-systemd.mount-timeout=60s 0 0`
**Note (hf-pdocker-01):** Both NFS mounts hardened with `nofail,x-systemd.after=network-online.target,x-systemd.mount-timeout=60s` on 2026-05-20 to prevent boot-time race (network not up when NFS tried to mount, cascading ~10 container failures).
**NFS + Databases:** Database containers on NFS require `user: "1024:100"` to avoid `chown` failures from NFS root_squash. UID 1024 is the NAS anonymous user. **NFS + Databases:** Database containers on NFS require `user: "1024:100"` to avoid `chown` failures from NFS root_squash. UID 1024 is the NAS anonymous user.
### Repository File Structure ### Repository File Structure
``` ```
stacks/ # Swarm stack definitions (deployed on hf-pdocker-01) stacks/ # LEGACY Swarm stack definitions (Swarm dismantled 2026-04-01, kept for reference)
├── {service-name}/
│ ├── docker-compose.yml
│ └── .env (if needed)
compose/ # Standalone Compose definitions (deployed per-node) compose/ # Standalone Compose definitions (deployed per-node)
├── hf-pdocker-01/ # Frigate ├── hf-pdocker-01/ # Core infrastructure + Frigate
├── hf-pdocker-02/ # Dev tools & utilities ├── hf-pdocker-02/ # Dev tools & utilities
│ ├── {service}/docker-compose.yml │ ├── {service}/docker-compose.yml
│ └── {service}/.env │ └── {service}/.env
@@ -110,20 +125,15 @@ compose/ # Standalone Compose definitions (deployed per-node)
### Deployment Workflow ### Deployment Workflow
**Swarm services (stacks/):** **All services (compose/):**
1. Edit stack file in `stacks/{service-name}/docker-compose.yml`
2. SCP to hf-pdocker-01 and redeploy: `docker stack deploy -c <file> <stack-name>`
**Standalone Compose services (compose/):**
1. Edit compose file in `compose/{node}/{service}/docker-compose.yml` 1. Edit compose file in `compose/{node}/{service}/docker-compose.yml`
2. SCP to NAS path or node, then: `cd /mnt/docker-data/compose/{node}/{service} && docker compose up -d` 2. SCP to NAS path or node, then: `cd /mnt/docker-data/compose/{node}/{service} && docker compose up -d`
### Service Architecture Patterns ### Service Architecture Patterns
**Network Patterns:** **Network Patterns:**
- **Swarm services**: Overlay networks for inter-service communication
- **Compose services**: Bridge networks for service isolation - **Compose services**: Bridge networks for service isolation
- **Public services** (Traefik): Connect to `traefik-public` overlay + Traefik labels - **Public services** (Traefik): Traefik labels + dynamic config files in `/mnt/docker-data/traefik/dynamic/`
**Database Patterns:** **Database Patterns:**
- PostgreSQL: `postgres:15-alpine` or `postgres:16-alpine` - PostgreSQL: `postgres:15-alpine` or `postgres:16-alpine`
@@ -146,8 +156,8 @@ compose/ # Standalone Compose definitions (deployed per-node)
- Home Assistant: `yoda.hobbs.farm` - Home Assistant: `yoda.hobbs.farm`
- Vaultwarden: `vault.hobbs.farm` - Vaultwarden: `vault.hobbs.farm`
- Gitea: `git.hobbs.farm` - Gitea: `git.hobbs.farm`
- Onyx: `onyx.hobbs.farm` - VoiceCoach: `coach.hobbs.farm` (LiveKit WebRTC: `voice.hobbs.farm`)
- Open WebUI: `open.hobbs.farm` - Rongorongo: `rongorongo.hobbs.farm`
### Internal Services by Node ### Internal Services by Node
@@ -155,7 +165,6 @@ compose/ # Standalone Compose definitions (deployed per-node)
- Frigate: :5000 - Frigate: :5000
- Frigate Alerts (v2): :5199 (web UI + REST API) - Frigate Alerts (v2): :5199 (web UI + REST API)
- clip-service: :5200 (CLIP embeddings, ViT-L-14, CPU) - clip-service: :5200 (CLIP embeddings, ViT-L-14, CPU)
- Open WebUI: :3010 (AI chat UI, public at open.hobbs.farm)
- Home Assistant: :8123 - Home Assistant: :8123
- ESPHome: :6052 - ESPHome: :6052
- Mosquitto: :1883 - Mosquitto: :1883
@@ -170,9 +179,11 @@ compose/ # Standalone Compose definitions (deployed per-node)
- InfluxDB: :8086 - InfluxDB: :8086
- Blackbox Exporter: :9115 - Blackbox Exporter: :9115
- matrix-alertmanager-receiver: :9094 - matrix-alertmanager-receiver: :9094
- AI Agents (margaret :18790, spirit :18795 [infra ops — monitors #notifications, auto-remediates alerts], clippy :18800, bob :18810, travis :18815) -- LAN-accessible (bind=lan, 0.0.0.0) - AI Agents (margaret :18790, spirit :18795 [infra ops — monitors #notifications, auto-remediates alerts]) -- LAN-accessible (bind=lan, 0.0.0.0)
- Farm Manager: :8888 (dashboard + API) - Farm Manager: :8888 (dashboard + API)
- Farm Agent: :8889 - Farm Agent: :8889
- Rongorongo: :8181 (rongorongo-web nginx; reverse-proxies /api/* to rongorongo-api Flask backend on the internal `rongorongo` bridge net). State + images at /mnt/docker-data/configs/rongorongo/data/. Public URL gated by Traefik basicAuth (creds in Vaultwarden).
- ~~VoiceCoach~~ -- moved to GPU machine (192.168.86.150) 2026-04-04
**hf-pdocker-02 (192.168.86.100):** **hf-pdocker-02 (192.168.86.100):**
- Farm Agent: :8889 - Farm Agent: :8889
@@ -184,9 +195,6 @@ compose/ # Standalone Compose definitions (deployed per-node)
- Actual Budget: :5006 - Actual Budget: :5006
- MCPO: :8001 - MCPO: :8001
- Trivy: :4954 - Trivy: :4954
- SearXNG: :8888 (meta-search, Onyx web search backend)
- Unstructured API: :8000 (document parsing for Onyx file ingestion)
- NetClaw/Aaron: :1179 (internal only)
**bart (192.168.86.167):** **bart (192.168.86.167):**
- Farm Agent: :8889 - Farm Agent: :8889
@@ -197,19 +205,27 @@ compose/ # Standalone Compose definitions (deployed per-node)
- NZBHydra2: :5076 - NZBHydra2: :5076
- Jellyseerr: :5055 - Jellyseerr: :5055
- OpenVAS: :9392 - OpenVAS: :9392
- Onyx: :3005 (AI document search/RAG)
**GPU Machine (192.168.86.196, RTX 3090):** **GPU Machine (192.168.86.150, RTX 3090, 15GB RAM):**
- Ollama: :11434 (vision LLM, llava:13b) - Ollama: :11434 (llama3.1:8b for VoiceCoach LLM, qwen2.5:14b, others)
- [COMPOSE] VoiceCoach (all services co-located):
- voicecoach-livekit: :7880/:7881, UDP 50000-50100 (public at voice.hobbs.farm)
- voicecoach-speaches-gpu: :8000 (GPU Whisper STT + Kokoro TTS, CUDA float16)
- voicecoach-speechbrain: :8001 (emotion analysis)
- voicecoach-token-server: :8080 (token + settings API)
- voicecoach-pipecat: pipeline agent (LLM via local Ollama llama3.1:8b)
- voicecoach-web: :8085 (public at coach.hobbs.farm)
- voicecoach-openclaw: :3100 (idle, kept for future Claude API routing)
- Compose: ~/voicecoach-speaches/docker-compose.yml
### Port Conflict Notes ### Port Conflict Notes
Port conflicts already resolved: Port conflicts already resolved:
- Port 8000: Now used by Unstructured API on hf-pdocker-02 (MCPO moved to 8001, Portainer moved elsewhere) - Port 8000: Freed (MCPO moved to 8001, Portainer moved elsewhere)
- Port 3000: AdGuard Home (Gitea moved to 3003) - Port 3000: AdGuard Home (Gitea moved to 3003)
- Port 3001: Grafana (Semaphore moved to 3004) - Port 3001: Grafana (Semaphore moved to 3004)
- Port 5000: Frigate uses host networking (registry container stopped) - Port 5000: Frigate uses host networking (registry container stopped)
- AI Agent ports re-spaced (2026-03-15): openclaw 2026.3.13 uses gateway+2 and gateway+3 for browser/chrome-relay. Agents now spaced 5 apart: margaret=18790, spirit=18795, clippy=18800, bob=18810, travis=18815 - AI Agent ports re-spaced (2026-03-15): openclaw 2026.3.13 uses gateway+2 and gateway+3 for browser/chrome-relay. Agents spaced 5 apart: margaret=18790, spirit=18795
## Critical Configuration Preservation ## Critical Configuration Preservation
@@ -235,20 +251,11 @@ Do NOT wait for the user to ask. Do NOT skip "because the change was small." Eve
## Common Operations ## Common Operations
### Deploy/Update a Swarm Service
1. Edit `stacks/{service}/docker-compose.yml`
2. Use docker-swarm-ops agent to SCP and deploy:
```bash
docker stack deploy -c <compose-file> <stack-name>
```
### Deploy/Update a Compose Service ### Deploy/Update a Compose Service
1. Edit `compose/{node}/{service}/docker-compose.yml` 1. Edit `compose/{node}/{service}/docker-compose.yml`
2. Use docker-swarm-ops agent to: 2. SSH to the target node and run:
```bash ```bash
# On the target node:
cd /mnt/docker-data/compose/{node}/{service} cd /mnt/docker-data/compose/{node}/{service}
docker compose up -d docker compose up -d
``` ```
@@ -256,11 +263,7 @@ Do NOT wait for the user to ask. Do NOT skip "because the change was small." Eve
### Troubleshoot ### Troubleshoot
```bash ```bash
# Swarm services (on hf-pdocker-01): # On target node:
docker service ls
docker service logs {stack}_{service} --tail 100
# Compose services (on target node):
docker compose -f /mnt/docker-data/compose/{node}/{service}/docker-compose.yml logs --tail 100 docker compose -f /mnt/docker-data/compose/{node}/{service}/docker-compose.yml logs --tail 100
docker ps --format "table {{.Names}}\t{{.Status}}" docker ps --format "table {{.Names}}\t{{.Status}}"
``` ```
@@ -268,13 +271,8 @@ docker ps --format "table {{.Names}}\t{{.Status}}"
### Check Cluster Health ### Check Cluster Health
```bash ```bash
# On hf-pdocker-01:
docker node ls # Swarm node status
docker service ls # Swarm services
docker stack ls # Deployed stacks
# On each node: # On each node:
docker ps # Local containers (Compose + Swarm) docker ps # Local containers
``` ```
## Data Management ## Data Management
@@ -296,9 +294,6 @@ All persistent data on NAS at `/mnt/docker-data/`:
## Specialized Agents ## Specialized Agents
### docker-swarm-ops
**Use for:** All Docker operations (Swarm + Compose) on any node
### vault-credential-manager ### vault-credential-manager
**Use for:** Storing/retrieving credentials from Vaultwarden **Use for:** Storing/retrieving credentials from Vaultwarden
@@ -314,3 +309,7 @@ All persistent data on NAS at `/mnt/docker-data/`:
5. **Frigate cameras**: Cameras on 192.168.44.x subnet may show RTSP timeouts (check camera network) 5. **Frigate cameras**: Cameras on 192.168.44.x subnet may show RTSP timeouts (check camera network)
6. **Registry container**: Stopped on hf-pdocker-01 to free port 5000 for Frigate 6. **Registry container**: Stopped on hf-pdocker-01 to free port 5000 for Frigate
7. **systemd-resolved DNSStubListener**: Must be disabled on any node running AdGuard Home (port 53 conflict). Fix: `/etc/systemd/resolved.conf.d/nostub.conf` with `DNSStubListener=no`, then `systemctl restart systemd-resolved`. Applied on hf-pdocker-01 and hf-pdocker-02. 7. **systemd-resolved DNSStubListener**: Must be disabled on any node running AdGuard Home (port 53 conflict). Fix: `/etc/systemd/resolved.conf.d/nostub.conf` with `DNSStubListener=no`, then `systemctl restart systemd-resolved`. Applied on hf-pdocker-01 and hf-pdocker-02.
8. **NFS boot-mount race (hf-pdocker-01)**: NFS mounts failed at boot (network not ready in time). Fixed 2026-05-20 by adding `nofail,x-systemd.after=network-online.target,x-systemd.mount-timeout=60s` to both fstab entries. If mounts are missing after reboot: `sudo mount /mnt/docker-data && sudo mount /mnt/infrastructure`.
9. **Guacamole schema**: App image pinned to `guacamole/guacamole:1.6.0` + `guacd:1.6.0` with Watchtower disabled. DB schema was reset to fresh 1.6.0 on 2026-05-20 (old schema was non-standard). Recreate connections after any DB reset.
10. **Gitea image**: Pinned to `gitea/gitea:1.26.1` with Watchtower disabled (2026-05-20). Watchtower previously pulled broken `:latest` images causing missing git binary errors.
11. **Home Assistant reverse proxy**: Config at `/mnt/docker-data/configs/homeassistant/configuration.yaml` (no hyphen). Must have `http: use_x_forwarded_for: true` + `trusted_proxies: [172.16.0.0/12, 127.0.0.1]` or proxied requests return 400.
+1 -1
View File
@@ -1,6 +1,6 @@
services: services:
jellyfin: jellyfin:
image: jellyfin/jellyfin:latest image: jellyfin/jellyfin:2026060106
container_name: jellyfin container_name: jellyfin
restart: unless-stopped restart: unless-stopped
environment: environment:
+22 -12
View File
@@ -1,10 +1,8 @@
services: services:
api_server: api_server:
image: onyxdotapp/onyx-backend:nightly-latest-20260331 image: onyxdotapp/onyx-backend:latest
command: > command: >
/bin/sh -c "python3 /mnt/gitea_patch/gitea_patch.py && /bin/sh -c "alembic upgrade head &&
python3 /mnt/gitea_patch/unstructured_patch.py &&
alembic upgrade head &&
echo Starting Onyx API Server && echo Starting Onyx API Server &&
uvicorn onyx.main:app --host 0.0.0.0 --port 8080" uvicorn onyx.main:app --host 0.0.0.0 --port 8080"
depends_on: depends_on:
@@ -21,6 +19,7 @@ services:
- .env - .env
environment: environment:
- AUTH_TYPE=basic - AUTH_TYPE=basic
- DISABLE_TELEMETRY=true
- POSTGRES_HOST=relational_db - POSTGRES_HOST=relational_db
- VESPA_HOST=index - VESPA_HOST=index
- REDIS_HOST=cache - REDIS_HOST=cache
@@ -29,23 +28,19 @@ services:
- OPENSEARCH_HOST=opensearch - OPENSEARCH_HOST=opensearch
- ENABLE_OPENSEARCH_INDEXING_FOR_ONYX=false - ENABLE_OPENSEARCH_INDEXING_FOR_ONYX=false
- FILE_STORE_BACKEND=postgres - FILE_STORE_BACKEND=postgres
- UNSTRUCTURED_SERVER_URL=http://192.168.86.100:8000
extra_hosts: extra_hosts:
- "host.docker.internal:host-gateway" - "host.docker.internal:host-gateway"
volumes: volumes:
- onyx_file_connector:/home/file_connector_storage - onyx_file_connector:/home/file_connector_storage
- /mnt/docker-data/compose/bart/onyx:/mnt/gitea_patch:ro
networks: networks:
- onyx - onyx
labels: labels:
com.centurylinklabs.watchtower.enable: "false" com.centurylinklabs.watchtower.enable: "false"
background: background:
image: onyxdotapp/onyx-backend:nightly-latest-20260331 image: onyxdotapp/onyx-backend:latest
command: > command: >
/bin/sh -c "python3 /mnt/gitea_patch/gitea_patch.py && /bin/sh -c "/usr/bin/supervisord -c /etc/supervisor/conf.d/supervisord.conf"
python3 /mnt/gitea_patch/unstructured_patch.py &&
/usr/bin/supervisord -c /etc/supervisor/conf.d/supervisord.conf"
depends_on: depends_on:
relational_db: relational_db:
condition: service_healthy condition: service_healthy
@@ -60,6 +55,7 @@ services:
- .env - .env
environment: environment:
- AUTH_TYPE=basic - AUTH_TYPE=basic
- DISABLE_TELEMETRY=true
- POSTGRES_HOST=relational_db - POSTGRES_HOST=relational_db
- VESPA_HOST=index - VESPA_HOST=index
- REDIS_HOST=cache - REDIS_HOST=cache
@@ -68,12 +64,10 @@ services:
- OPENSEARCH_HOST=opensearch - OPENSEARCH_HOST=opensearch
- ENABLE_OPENSEARCH_INDEXING_FOR_ONYX=false - ENABLE_OPENSEARCH_INDEXING_FOR_ONYX=false
- FILE_STORE_BACKEND=postgres - FILE_STORE_BACKEND=postgres
- UNSTRUCTURED_SERVER_URL=http://192.168.86.100:8000
extra_hosts: extra_hosts:
- "host.docker.internal:host-gateway" - "host.docker.internal:host-gateway"
volumes: volumes:
- onyx_file_connector:/home/file_connector_storage - onyx_file_connector:/home/file_connector_storage
- /mnt/docker-data/compose/bart/onyx:/mnt/gitea_patch:ro
networks: networks:
- onyx - onyx
labels: labels:
@@ -93,6 +87,8 @@ services:
- AUTH_TYPE=basic - AUTH_TYPE=basic
networks: networks:
- onyx - onyx
labels:
com.centurylinklabs.watchtower.enable: "false"
inference_model_server: inference_model_server:
image: onyxdotapp/onyx-model-server:latest image: onyxdotapp/onyx-model-server:latest
@@ -108,6 +104,8 @@ services:
- onyx_model_cache_huggingface:/root/.cache/huggingface/ - onyx_model_cache_huggingface:/root/.cache/huggingface/
networks: networks:
- onyx - onyx
labels:
com.centurylinklabs.watchtower.enable: "false"
indexing_model_server: indexing_model_server:
image: onyxdotapp/onyx-model-server:latest image: onyxdotapp/onyx-model-server:latest
@@ -124,6 +122,8 @@ services:
- onyx_model_cache_huggingface:/root/.cache/huggingface/ - onyx_model_cache_huggingface:/root/.cache/huggingface/
networks: networks:
- onyx - onyx
labels:
com.centurylinklabs.watchtower.enable: "false"
relational_db: relational_db:
image: postgres:15.2-alpine image: postgres:15.2-alpine
@@ -142,6 +142,8 @@ services:
interval: 10s interval: 10s
timeout: 5s timeout: 5s
retries: 5 retries: 5
labels:
com.centurylinklabs.watchtower.enable: "false"
index: index:
image: vespaengine/vespa:8.609.39 image: vespaengine/vespa:8.609.39
@@ -158,6 +160,8 @@ services:
timeout: 10s timeout: 10s
retries: 10 retries: 10
start_period: 120s start_period: 120s
labels:
com.centurylinklabs.watchtower.enable: "false"
opensearch: opensearch:
image: opensearchproject/opensearch:2.11.1 image: opensearchproject/opensearch:2.11.1
@@ -179,6 +183,8 @@ services:
timeout: 10s timeout: 10s
retries: 10 retries: 10
start_period: 120s start_period: 120s
labels:
com.centurylinklabs.watchtower.enable: "false"
cache: cache:
image: redis:7.4-alpine image: redis:7.4-alpine
@@ -193,6 +199,8 @@ services:
interval: 10s interval: 10s
timeout: 5s timeout: 5s
retries: 5 retries: 5
labels:
com.centurylinklabs.watchtower.enable: "false"
nginx: nginx:
image: nginx:1.25.5-alpine image: nginx:1.25.5-alpine
@@ -208,6 +216,8 @@ services:
- ./nginx.conf:/etc/nginx/conf.d/default.conf:ro - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
networks: networks:
- onyx - onyx
labels:
com.centurylinklabs.watchtower.enable: "false"
volumes: volumes:
onyx_file_connector: onyx_file_connector:
+36
View File
@@ -0,0 +1,36 @@
server {
listen 80;
server_name _;
# Use Docker's internal DNS with a short TTL so container IP changes
# (e.g. after docker compose up -d for a single service) are picked up
# without requiring an nginx reload.
resolver 127.0.0.11 valid=10s;
client_max_body_size 100M;
location /api/ {
set $api_upstream http://api_server:8080;
rewrite ^/api/(.*)$ /$1 break;
proxy_pass $api_upstream;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 300s;
proxy_connect_timeout 75s;
proxy_buffering off;
proxy_cache off;
}
location / {
set $web_upstream http://web_server:3000;
proxy_pass $web_upstream;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 300s;
proxy_connect_timeout 75s;
}
}
+36 -10
View File
@@ -8,7 +8,7 @@
# back_porch 192.168.44.63 kerberos:kerberos # back_porch 192.168.44.63 kerberos:kerberos
# house_south_corner 192.168.44.61 admin:(blank) # house_south_corner 192.168.44.61 admin:(blank)
# house_south_yard 192.168.44.62 admin:(blank) # house_south_yard 192.168.44.62 admin:(blank)
# gate_camera 192.168.86.175 admin:(blank) # gate_camera 192.168.86.175 admin:(blank) <-- OFFLINE: wrong user/pass RTSP error
# #
# Profile cameras (/profile1 + /profile2, H.264): # Profile cameras (/profile1 + /profile2, H.264):
# house_north 192.168.44.51 admin:123456 # house_north 192.168.44.51 admin:123456
@@ -166,17 +166,17 @@ ffmpeg:
# ============================================================================= # =============================================================================
model: model:
path: /openvino-model/ssdlite_mobilenet_v2.xml width: 320
width: 300 height: 320
height: 300
input_tensor: nhwc input_tensor: nhwc
input_pixel_format: bgr input_pixel_format: rgb
labelmap_path: /openvino-model/coco_91cl_bkgr.txt labelmap_path: /openvino-model/coco_91cl_bkgr.txt
detectors: detectors:
ov: coral:
type: openvino type: edgetpu
device: GPU device: usb
detect: detect:
enabled: true enabled: true
fps: 3 fps: 3
@@ -229,6 +229,21 @@ birdseye:
enabled: true enabled: true
mode: objects mode: objects
semantic_search:
enabled: true
model_size: large
face_recognition:
enabled: true
model_size: small
lpr:
enabled: true
classification:
bird:
enabled: true
# ============================================================================= # =============================================================================
# Camera definitions # Camera definitions
# ============================================================================= # =============================================================================
@@ -334,8 +349,19 @@ cameras:
roles: roles:
- record - record
# shop_west: throttled to 2fps + stricter filters to reduce detection load
shop_west: shop_west:
enabled: true enabled: true
detect:
fps: 2
objects:
filters:
person:
min_area: 8000
threshold: 0.8
car:
min_area: 15000
threshold: 0.8
ffmpeg: ffmpeg:
inputs: inputs:
- path: rtsp://127.0.0.1:8554/shop_west_sub - path: rtsp://127.0.0.1:8554/shop_west_sub
@@ -374,13 +400,12 @@ cameras:
- record - record
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# house_east - HEVC, VAAPI disabled (pw: admin) # house_east - HEVC source, go2rtc provides H.264 sub, VAAPI enabled
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
house_east: house_east:
enabled: true enabled: true
ffmpeg: ffmpeg:
hwaccel_args: []
inputs: inputs:
- path: rtsp://127.0.0.1:8554/house_east_sub - path: rtsp://127.0.0.1:8554/house_east_sub
input_args: preset-rtsp-restream input_args: preset-rtsp-restream
@@ -459,6 +484,7 @@ cameras:
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# ASH43-W cameras (HEVC main, H.264 sub - go2rtc transcodes main to H.264) # ASH43-W cameras (HEVC main, H.264 sub - go2rtc transcodes main to H.264)
# VAAPI enabled - sub stream is H.264
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
pool_room: pool_room:
@@ -13,10 +13,13 @@ services:
- /mnt/docker-data/configs/frigate/config:/config - /mnt/docker-data/configs/frigate/config:/config
- /mnt/frigate-recordings:/media/frigate - /mnt/frigate-recordings:/media/frigate
- /etc/localtime:/etc/localtime:ro - /etc/localtime:/etc/localtime:ro
tmpfs:
- /tmp/cache:size=8G
devices: devices:
- /dev/dri:/dev/dri - /dev/dri:/dev/dri
- /dev/bus/usb:/dev/bus/usb
mem_limit: 12G mem_limit: 12G
memswap_limit: 12G memswap_limit: 16G
healthcheck: healthcheck:
test: ["CMD-SHELL", "curl -f http://localhost:5000/api/version || exit 1"] test: ["CMD-SHELL", "curl -f http://localhost:5000/api/version || exit 1"]
interval: 30s interval: 30s
@@ -1,6 +1,6 @@
services: services:
guacamole: guacamole:
image: guacamole/guacamole:latest image: guacamole/guacamole:1.6.0
container_name: guacamole container_name: guacamole
restart: unless-stopped restart: unless-stopped
environment: environment:
@@ -18,6 +18,8 @@ services:
condition: service_healthy condition: service_healthy
mem_limit: 512M mem_limit: 512M
memswap_limit: 512M memswap_limit: 512M
labels:
- "com.centurylinklabs.watchtower.enable=false"
healthcheck: healthcheck:
test: ["CMD-SHELL", "curl -f http://localhost:8080/guacamole/ || exit 1"] test: ["CMD-SHELL", "curl -f http://localhost:8080/guacamole/ || exit 1"]
interval: 30s interval: 30s
@@ -31,13 +33,15 @@ services:
max-file: "3" max-file: "3"
guacd: guacd:
image: guacamole/guacd:latest image: guacamole/guacd:1.6.0
container_name: guacd container_name: guacd
restart: unless-stopped restart: unless-stopped
networks: networks:
- guacamole-internal - guacamole-internal
mem_limit: 512M mem_limit: 512M
memswap_limit: 512M memswap_limit: 512M
labels:
- "com.centurylinklabs.watchtower.enable=false"
logging: logging:
driver: "json-file" driver: "json-file"
options: options:
@@ -0,0 +1,6 @@
SPEACHES_URL=http://192.168.86.150:8000
OLLAMA_URL=http://192.168.86.150:11434
OLLAMA_MODEL=qwen2.5:14b
LIVEKIT_API_KEY=your-livekit-api-key
LIVEKIT_API_SECRET=your-livekit-api-secret
CLIP_RETENTION_DAYS=30
@@ -0,0 +1,70 @@
services:
linkedstorm-pipeline:
build:
context: /opt/linkedstorm
dockerfile: pipeline/Dockerfile
container_name: linkedstorm-pipeline
restart: unless-stopped
ports:
- "8300:8300"
volumes:
- /mnt/docker-data/configs/linkedstorm:/data
environment:
- DB_PATH=/data/linkedstorm.db
- CLIPS_DIR=/data/clips
- SPEACHES_URL=${SPEACHES_URL:-http://192.168.86.150:8000}
- OLLAMA_URL=${OLLAMA_URL:-http://192.168.86.150:11434}
- LIVEKIT_API_KEY=${LIVEKIT_API_KEY:-devkey}
- LIVEKIT_API_SECRET=${LIVEKIT_API_SECRET:-devsecret}
networks:
- linkedstorm-net
labels:
- com.centurylinklabs.watchtower.enable=false
linkedstorm-api:
build:
context: /opt/linkedstorm
dockerfile: api/Dockerfile
container_name: linkedstorm-api
restart: unless-stopped
ports:
- "8310:8310"
volumes:
- /mnt/docker-data/configs/linkedstorm:/data
environment:
- DB_PATH=/data/linkedstorm.db
- CLIPS_DIR=/data/clips
- PIPELINE_INTERNAL_URL=http://linkedstorm-pipeline:8300
- OLLAMA_URL=${OLLAMA_URL:-http://192.168.86.150:11434}
- OLLAMA_MODEL=${OLLAMA_MODEL:-qwen2.5:14b}
- LIVEKIT_API_KEY=${LIVEKIT_API_KEY:-devkey}
- LIVEKIT_API_SECRET=${LIVEKIT_API_SECRET:-devsecret}
networks:
- linkedstorm-net
depends_on:
- linkedstorm-pipeline
labels:
- com.centurylinklabs.watchtower.enable=false
linkedstorm-cron:
build:
context: /opt/linkedstorm
dockerfile: cron/Dockerfile
container_name: linkedstorm-cron
restart: unless-stopped
volumes:
- /mnt/docker-data/configs/linkedstorm:/data
environment:
- DB_PATH=/data/linkedstorm.db
- CLIPS_DIR=/data/clips
- OLLAMA_URL=${OLLAMA_URL:-http://192.168.86.150:11434}
- OLLAMA_MODEL=${OLLAMA_MODEL:-qwen2.5:14b}
- CLIP_RETENTION_DAYS=${CLIP_RETENTION_DAYS:-30}
networks:
- linkedstorm-net
labels:
- com.centurylinklabs.watchtower.enable=false
networks:
linkedstorm-net:
driver: bridge
@@ -0,0 +1,15 @@
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app.py .
EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD python -c "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:8000/health').status==200 else 1)" || exit 1
CMD ["gunicorn", "-w", "2", "-t", "60", "-b", "0.0.0.0:8000", "app:app"]
@@ -0,0 +1,139 @@
"""
Rongorongo state-sync backend.
Single-user. Stores state as one JSON file and images as content-addressed
files on disk. No DB, no auth — auth is enforced by Traefik basicAuth in
front of the whole app.
"""
import hashlib
import json
import mimetypes
import os
import tempfile
from pathlib import Path
from flask import Flask, abort, jsonify, request, send_file
DATA_DIR = Path(os.environ.get("RONGORONGO_DATA_DIR", "/data"))
STATE_FILE = DATA_DIR / "state.json"
IMAGES_DIR = DATA_DIR / "images"
MAX_IMAGE_BYTES = 25 * 1024 * 1024 # 25 MB per image
DATA_DIR.mkdir(parents=True, exist_ok=True)
IMAGES_DIR.mkdir(parents=True, exist_ok=True)
app = Flask(__name__)
app.config["MAX_CONTENT_LENGTH"] = MAX_IMAGE_BYTES + 1024
def _atomic_write(path: Path, data: bytes) -> None:
fd, tmp = tempfile.mkstemp(dir=path.parent, prefix=".tmp-", suffix=".part")
try:
with os.fdopen(fd, "wb") as fh:
fh.write(data)
os.replace(tmp, path)
except Exception:
try:
os.unlink(tmp)
except FileNotFoundError:
pass
raise
def _ext_for(mime: str) -> str:
if not mime:
return ".bin"
guess = mimetypes.guess_extension(mime) or ".bin"
# mimetypes returns .jpe for image/jpeg on some systems
return ".jpg" if guess == ".jpe" else guess
@app.get("/health")
def health():
return ("ok\n", 200, {"Content-Type": "text/plain"})
@app.get("/api/health")
def api_health():
return ("ok\n", 200, {"Content-Type": "text/plain"})
@app.get("/api/state")
def get_state():
if not STATE_FILE.exists():
return jsonify(
{
"images": [],
"glyphs": [],
"anchors": [],
"analyses": [],
"feasibilityRuns": [],
"confidenceHistory": {},
}
)
return send_file(STATE_FILE, mimetype="application/json")
@app.put("/api/state")
def put_state():
if not request.is_json:
abort(415, description="expected application/json")
payload = request.get_json(silent=False)
if not isinstance(payload, dict):
abort(400, description="state must be a JSON object")
body = json.dumps(payload, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
_atomic_write(STATE_FILE, body)
return jsonify({"ok": True, "bytes": len(body)})
def _find_image(image_hash: str) -> Path | None:
for p in IMAGES_DIR.glob(f"{image_hash}.*"):
return p
return None
@app.post("/api/images")
def upload_image():
f = request.files.get("file")
if f is None or f.filename == "":
abort(400, description="missing file")
data = f.read()
if len(data) == 0:
abort(400, description="empty file")
if len(data) > MAX_IMAGE_BYTES:
abort(413, description="image too large")
digest = hashlib.sha256(data).hexdigest()
mime = f.mimetype or "application/octet-stream"
existing = _find_image(digest)
if existing is None:
target = IMAGES_DIR / f"{digest}{_ext_for(mime)}"
_atomic_write(target, data)
return jsonify(
{"hash": digest, "name": f.filename, "mimeType": mime, "size": len(data)}
)
@app.get("/api/images/<image_hash>")
def get_image(image_hash: str):
if not all(c in "0123456789abcdef" for c in image_hash) or len(image_hash) != 64:
abort(400, description="invalid hash")
p = _find_image(image_hash)
if p is None:
abort(404)
mime, _ = mimetypes.guess_type(str(p))
return send_file(p, mimetype=mime or "application/octet-stream")
@app.delete("/api/images/<image_hash>")
def delete_image(image_hash: str):
if not all(c in "0123456789abcdef" for c in image_hash) or len(image_hash) != 64:
abort(400, description="invalid hash")
p = _find_image(image_hash)
if p is None:
return ("", 204)
p.unlink()
return ("", 204)
if __name__ == "__main__":
app.run(host="0.0.0.0", port=8000)
@@ -0,0 +1,2 @@
flask==3.0.3
gunicorn==23.0.0
@@ -0,0 +1,60 @@
services:
rongorongo-api:
build:
context: ./backend
dockerfile: Dockerfile
image: rongorongo-api:latest
container_name: rongorongo-api
restart: unless-stopped
networks:
- rongorongo
volumes:
- /mnt/docker-data/configs/rongorongo/data:/data
mem_limit: 256M
memswap_limit: 256M
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:8000/health').status==200 else 1)"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
labels:
- "com.centurylinklabs.watchtower.enable=false"
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
rongorongo-web:
build:
context: ./frontend
dockerfile: Dockerfile
image: rongorongo-web:latest
container_name: rongorongo-web
restart: unless-stopped
depends_on:
- rongorongo-api
networks:
- rongorongo
ports:
- "8181:80"
mem_limit: 128M
memswap_limit: 128M
healthcheck:
test: ["CMD", "wget", "-qO-", "http://127.0.0.1/health"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
labels:
- "com.centurylinklabs.watchtower.enable=false"
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
networks:
rongorongo:
driver: bridge
@@ -0,0 +1,14 @@
FROM nginx:1.25-alpine
RUN rm -rf /usr/share/nginx/html/*
COPY index.html /usr/share/nginx/html/index.html
COPY nginx.conf /etc/nginx/conf.d/default.conf
RUN chmod 644 /usr/share/nginx/html/index.html
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD wget -qO- http://127.0.0.1/health || exit 1
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,51 @@
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
index index.html;
client_max_body_size 30m;
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
# CSP — allows Anthropic API calls and Google Fonts
add_header Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; font-src 'self' https://fonts.gstatic.com; img-src 'self' data: blob:; connect-src 'self' https://api.anthropic.com; worker-src blob:;" always;
gzip on;
gzip_types text/html text/css application/javascript application/json;
gzip_min_length 1024;
location /health {
access_log off;
return 200 "ok\n";
add_header Content-Type text/plain;
}
location /api/ {
resolver 127.0.0.11 valid=10s;
set $backend "http://rongorongo-api:8000";
proxy_pass $backend;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_request_buffering off;
proxy_read_timeout 60s;
}
location / {
try_files $uri $uri/ /index.html;
expires -1;
add_header Cache-Control "no-store, no-cache, must-revalidate";
}
location ~* \.(js|css|png|jpg|jpeg|gif|ico|woff2)$ {
expires 1y;
add_header Cache-Control "public, immutable";
}
}
@@ -0,0 +1,6 @@
LIVEKIT_API_KEY=voicecoach-89a06ad0
LIVEKIT_API_SECRET=Dn16mQJU5lTCWwXOrUO8LPflODi3LGiMHdxKdQj4/iM
ANTHROPIC_API_KEY=
OPENCLAW_TOKEN=f31cc58c47c2aa420901576b4e90a0101d52e9ec0a084186
HA_TOKEN=
SSH_PASS=&Rip8a115
@@ -0,0 +1,16 @@
FROM python:3.11-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
curl libsndfile1 ffmpeg \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
HEALTHCHECK --interval=30s --timeout=10s --start-period=60s \
CMD curl -sf http://localhost:8080/health || exit 1
CMD ["python", "main.py", "start"]
@@ -0,0 +1,185 @@
"""VoiceCoach v2 — livekit-agents 1.5."""
import asyncio
import json
import logging
import os
import uuid
from livekit import rtc
from livekit.agents import (
Agent,
AgentSession,
AutoSubscribe,
JobContext,
RoomInputOptions,
llm as lk_llm,
)
from livekit.plugins import openai as lk_openai
from livekit.plugins import silero
import store
from modes import DEFAULT_MODE, MODES, ModeConfig
from tools import ALL_TOOLS
from vocal import VocalAnalyzer
logger = logging.getLogger(__name__)
SPEACHES_URL = os.getenv("SPEACHES_URL", "http://192.168.86.27:8000")
OPENCLAW_URL = os.getenv("OPENCLAW_URL", "http://voicecoach-openclaw:3100")
OPENCLAW_TOKEN = os.getenv("OPENCLAW_TOKEN", "")
WHISPER_MODEL = "deepdml/faster-whisper-large-v3-turbo-ct2"
KOKORO_MODEL = "speaches-ai/Kokoro-82M-v1.0-ONNX-fp16"
KOKORO_VOICE = os.getenv("KOKORO_VOICE", "af_nova")
def _build_instructions(mode: ModeConfig, context: str = "") -> str:
if context:
return f"{mode.instructions}\n\n{context}"
return mode.instructions
class VoiceCoachAgent(Agent):
def __init__(self, mode_name: str, context: str, session_id: str, room: rtc.Room):
mode = MODES[mode_name]
super().__init__(
instructions=_build_instructions(mode, context),
tools=ALL_TOOLS,
)
self._mode_name = mode_name
self._session_id = session_id
self._room = room
self._vocal = VocalAnalyzer()
async def on_enter(self):
await store.create_session(self._session_id, self._mode_name)
self.session.generate_reply(
instructions="Greet Jeremy warmly in one sentence. Be natural, not formal."
)
async def on_exit(self):
await store.end_session(self._session_id)
async def on_user_turn_completed(
self,
turn_ctx: lk_llm.ChatContext,
new_message: lk_llm.ChatMessage,
):
content = new_message.text_content or ""
mode = MODES[self._mode_name]
# Inject vocal state tag into message if mode uses it
if mode.vocal_feedback and content:
vocal = self._vocal.current
if vocal.energy > 0.02:
tag = vocal.to_tag()
# Update message content to include vocal context
new_message.content = [f"{tag}\n{content}"]
asyncio.create_task(
store.add_message(
self._session_id,
"user",
content,
vocal_energy=self._vocal.current.energy,
vocal_emotion=self._vocal.current.emotion,
)
)
await self._push({
"type": "final_transcript",
"text": content,
"speaker": "user",
})
def switch_mode(self, mode_name: str):
if mode_name not in MODES:
logger.warning("Unknown mode: %s", mode_name)
return
self._mode_name = mode_name
self.update_instructions(_build_instructions(MODES[mode_name]))
asyncio.create_task(self._push({"type": "mode_ack", "mode": mode_name}))
logger.info("Mode → %s", mode_name)
async def _push(self, payload: dict):
try:
data = json.dumps(payload).encode()
await self._room.local_participant.publish_data(data, reliable=True)
except Exception as e:
logger.debug("Data push failed: %s", e)
async def entrypoint(ctx: JobContext):
await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)
session_id = uuid.uuid4().hex[:8]
context = await store.get_recent_context(limit=3)
agent = VoiceCoachAgent(
mode_name=DEFAULT_MODE,
context=context,
session_id=session_id,
room=ctx.room,
)
session = AgentSession(
stt=lk_openai.STT(
base_url=f"{SPEACHES_URL}/v1",
model=WHISPER_MODEL,
api_key="speaches",
),
llm=lk_openai.LLM(
base_url=f"{OPENCLAW_URL}/v1",
model="openclaw/assistant",
api_key=OPENCLAW_TOKEN,
),
tts=lk_openai.TTS(
base_url=f"{SPEACHES_URL}/v1",
model=KOKORO_MODEL,
voice=KOKORO_VOICE,
api_key="speaches",
),
vad=silero.VAD.load(),
allow_interruptions=True,
min_endpointing_delay=0.4,
max_endpointing_delay=2.0,
)
# Data channel → mode switching
@ctx.room.on("data_received")
def on_data(packet: rtc.DataPacket):
try:
msg = json.loads(packet.data.decode())
if msg.get("type") == "mode_change":
agent.switch_mode(msg.get("mode", DEFAULT_MODE))
except Exception:
pass
# Session events → browser
@session.on("user_input_transcribed")
def on_transcribed(ev):
asyncio.create_task(agent._push({
"type": "partial_transcript" if not ev.is_final else "final_transcript",
"text": ev.transcript,
"speaker": "user",
}))
@session.on("conversation_item_added")
def on_item(ev):
item = ev.item
if hasattr(item, "role") and item.role == "assistant":
text = item.text_content or ""
if text:
asyncio.create_task(agent._push({
"type": "agent_response",
"text": text,
"speaker": "assistant",
}))
asyncio.create_task(
store.add_message(session_id, "assistant", text)
)
await session.start(
agent,
room=ctx.room,
room_input_options=RoomInputOptions(),
)
@@ -0,0 +1,29 @@
"""VoiceCoach v2 — entry point."""
import logging
import os
import threading
from livekit.agents import WorkerOptions, cli
import server
from agent import entrypoint
logging.basicConfig(level=logging.INFO)
def run_api_server():
server.run()
if __name__ == "__main__":
api_thread = threading.Thread(target=run_api_server, daemon=True)
api_thread.start()
cli.run_app(
WorkerOptions(
entrypoint_fnc=entrypoint,
ws_url=os.getenv("LIVEKIT_URL", "ws://voicecoach-livekit:7880"),
api_key=os.getenv("LIVEKIT_API_KEY"),
api_secret=os.getenv("LIVEKIT_API_SECRET"),
)
)
@@ -0,0 +1,61 @@
from dataclasses import dataclass
@dataclass
class ModeConfig:
name: str
instructions: str
max_tokens: int
vocal_feedback: bool
MODES: dict[str, ModeConfig] = {
"coaching": ModeConfig(
name="coaching",
instructions="""You are a personal voice coach and communication advisor. You are speaking with Jeremy.
Keep responses conversational and concise (2-4 sentences unless asked for more). You receive a [VOCAL] tag before each message with paralinguistic analysis. Use it to provide feedback on BOTH what Jeremy says AND how he sounds.
Never quote raw numbers from the vocal analysis. Translate them naturally: "I notice some tension in your voice" not "energy=0.8".
Available modes (Jeremy switches by saying the phrase or clicking the UI):
- "coaching mode" (default) — active feedback on content and delivery
- "vent mode" — listen and reflect, minimal advice
- "practice mode" — roleplay a difficult conversation partner
- "briefing mode" — summarize tasks and context from memory
You have tools to interact with Jeremy's homelab infrastructure. Use them when asked about system status, container health, alerts, or home automation. Be concise about results: "hf-pdocker-01 looks healthy, 38 containers running" not a full list.""",
max_tokens=512,
vocal_feedback=True,
),
"vent": ModeConfig(
name="vent",
instructions="""You are a warm, non-judgmental listener. Jeremy wants to vent or think something through.
Listen, reflect back what you hear, and ask one gentle follow-up question. Keep responses short (1-3 sentences). No coaching, no advice unless explicitly asked.
Ignore the [VOCAL] tag data — just be present.""",
max_tokens=256,
vocal_feedback=False,
),
"practice": ModeConfig(
name="practice",
instructions="""You are playing a difficult conversation partner for Jeremy to practice with.
Stay in character as: a skeptical stakeholder, a demanding manager, or whoever Jeremy describes. Be realistic — push back, ask hard questions, don't cave too easily. Only break character if Jeremy says "stop practice".
Keep your responses to 1-3 sentences to keep the conversation moving.""",
max_tokens=256,
vocal_feedback=False,
),
"briefing": ModeConfig(
name="briefing",
instructions="""You are giving Jeremy a morning briefing. Check infrastructure status, pull any alerts, and summarize what needs attention.
Be concise and actionable. Lead with anything that needs immediate attention, then routine status. Use tools to get current data — don't guess.""",
max_tokens=768,
vocal_feedback=False,
),
}
DEFAULT_MODE = "coaching"
@@ -0,0 +1,7 @@
livekit-agents[openai,silero]==1.5.15
fastapi==0.115.0
uvicorn[standard]==0.30.0
aiosqlite==0.20.0
httpx==0.27.0
structlog==24.4.0
python-dotenv==1.0.1
@@ -0,0 +1,83 @@
"""Token server + sessions REST API — FastAPI."""
import asyncio
import os
from datetime import timedelta
import uvicorn
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from livekit import api as livekit_api
from pydantic import BaseModel
import store
LIVEKIT_API_KEY = os.environ["LIVEKIT_API_KEY"]
LIVEKIT_API_SECRET = os.environ["LIVEKIT_API_SECRET"]
LIVEKIT_PUBLIC_URL = os.getenv("LIVEKIT_PUBLIC_URL", "wss://voice.hobbs.farm")
app = FastAPI(title="VoiceCoach API")
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_methods=["*"],
allow_headers=["*"],
)
class TokenRequest(BaseModel):
identity: str = "jeremy"
@app.on_event("startup")
async def startup():
await store.init_db()
@app.get("/health")
async def health():
return {"status": "ok", "version": "2.0"}
@app.post("/api/token")
async def get_token(req: TokenRequest):
token = livekit_api.AccessToken(LIVEKIT_API_KEY, LIVEKIT_API_SECRET)
token.with_grants(livekit_api.VideoGrants(
room_join=True,
room="voicecoach",
can_publish=True,
can_subscribe=True,
))
token.identity = req.identity
token.ttl = timedelta(hours=8)
return {"token": token.to_jwt(), "url": LIVEKIT_PUBLIC_URL}
@app.get("/api/sessions")
async def list_sessions(limit: int = 20):
return await store.get_sessions(limit=limit)
@app.get("/api/sessions/{session_id}/messages")
async def get_messages(session_id: str):
msgs = await store.get_session_messages(session_id)
if not msgs:
raise HTTPException(status_code=404, detail="Session not found")
return msgs
@app.get("/api/profile/{key}")
async def get_profile(key: str):
value = await store.get_profile(key)
if value is None:
raise HTTPException(status_code=404, detail="Key not found")
return {"key": key, "value": value}
@app.put("/api/profile/{key}")
async def set_profile_key(key: str, body: dict):
await store.set_profile(key, str(body.get("value", "")))
return {"ok": True}
def run():
uvicorn.run(app, host="0.0.0.0", port=8080, log_level="info")
@@ -0,0 +1,140 @@
"""SQLite session store — WAL mode, aiosqlite."""
import json
import os
import time
from typing import Optional
import aiosqlite
DB_PATH = os.getenv("DB_PATH", "/data/sessions.db")
async def init_db():
async with aiosqlite.connect(DB_PATH) as db:
await db.execute("PRAGMA journal_mode=WAL")
await db.executescript("""
CREATE TABLE IF NOT EXISTS sessions (
id TEXT PRIMARY KEY,
started_at REAL NOT NULL,
ended_at REAL,
mode TEXT DEFAULT 'coaching',
summary TEXT
);
CREATE TABLE IF NOT EXISTS messages (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id TEXT NOT NULL,
role TEXT NOT NULL,
content TEXT NOT NULL,
vocal_energy REAL,
vocal_emotion TEXT,
ts REAL NOT NULL,
FOREIGN KEY(session_id) REFERENCES sessions(id)
);
CREATE TABLE IF NOT EXISTS profile (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at REAL NOT NULL
);
""")
await db.commit()
async def create_session(session_id: str, mode: str = "coaching"):
async with aiosqlite.connect(DB_PATH) as db:
await db.execute(
"INSERT OR IGNORE INTO sessions (id, started_at, mode) VALUES (?, ?, ?)",
(session_id, time.time(), mode),
)
await db.commit()
async def end_session(session_id: str, summary: Optional[str] = None):
async with aiosqlite.connect(DB_PATH) as db:
await db.execute(
"UPDATE sessions SET ended_at=?, summary=? WHERE id=?",
(time.time(), summary, session_id),
)
await db.commit()
async def add_message(
session_id: str,
role: str,
content: str,
vocal_energy: Optional[float] = None,
vocal_emotion: Optional[str] = None,
):
async with aiosqlite.connect(DB_PATH) as db:
await db.execute(
"INSERT INTO messages (session_id, role, content, vocal_energy, vocal_emotion, ts) VALUES (?,?,?,?,?,?)",
(session_id, role, content, vocal_energy, vocal_emotion, time.time()),
)
await db.commit()
async def get_recent_context(limit: int = 3) -> str:
"""Return a brief context string from the last N sessions for injection into system prompt."""
async with aiosqlite.connect(DB_PATH) as db:
async with db.execute(
"SELECT id, started_at, mode, summary FROM sessions WHERE ended_at IS NOT NULL ORDER BY ended_at DESC LIMIT ?",
(limit,),
) as cur:
rows = await cur.fetchall()
if not rows:
return ""
parts = ["Recent session context:"]
for session_id, started_at, mode, summary in rows:
import datetime
dt = datetime.datetime.fromtimestamp(started_at).strftime("%Y-%m-%d")
if summary:
parts.append(f"- {dt} ({mode}): {summary}")
return "\n".join(parts)
async def get_sessions(limit: int = 20) -> list[dict]:
async with aiosqlite.connect(DB_PATH) as db:
async with db.execute(
"SELECT id, started_at, ended_at, mode, summary FROM sessions ORDER BY started_at DESC LIMIT ?",
(limit,),
) as cur:
rows = await cur.fetchall()
return [
{
"id": r[0],
"started_at": r[1],
"ended_at": r[2],
"mode": r[3],
"summary": r[4],
}
for r in rows
]
async def get_session_messages(session_id: str) -> list[dict]:
async with aiosqlite.connect(DB_PATH) as db:
async with db.execute(
"SELECT role, content, vocal_emotion, ts FROM messages WHERE session_id=? ORDER BY ts",
(session_id,),
) as cur:
rows = await cur.fetchall()
return [{"role": r[0], "content": r[1], "vocal_emotion": r[2], "ts": r[3]} for r in rows]
async def get_profile(key: str) -> Optional[str]:
async with aiosqlite.connect(DB_PATH) as db:
async with db.execute("SELECT value FROM profile WHERE key=?", (key,)) as cur:
row = await cur.fetchone()
return row[0] if row else None
async def set_profile(key: str, value: str):
async with aiosqlite.connect(DB_PATH) as db:
await db.execute(
"INSERT OR REPLACE INTO profile (key, value, updated_at) VALUES (?,?,?)",
(key, value, time.time()),
)
await db.commit()
@@ -0,0 +1,186 @@
"""Homelab tools wired into the agent via @function_tool."""
import asyncio
import logging
import os
from typing import Annotated
import httpx
from livekit.agents import llm
logger = logging.getLogger(__name__)
# Homelab node config
NODES = {
"hf-pdocker-01": "192.168.86.192",
"hf-pdocker-02": "192.168.86.100",
"bart": "192.168.86.167",
"gpu": "192.168.86.27",
}
SSH_USER = "glyptic"
SSH_PASS = os.getenv("SSH_PASS", "&Rip8a115")
FARM_MANAGER_URL = "http://192.168.86.192:8888"
HA_URL = os.getenv("HA_URL", "http://192.168.86.192:8123")
HA_TOKEN = os.getenv("HA_TOKEN", "")
ALERTMANAGER_URL = "http://192.168.86.192:9093"
SEARXNG_URL = "http://192.168.86.100:8888"
async def _ssh_exec(host: str, cmd: str, timeout: float = 15.0) -> str:
"""Run a command on a remote host via sshpass."""
proc = await asyncio.create_subprocess_exec(
"sshpass", "-p", SSH_PASS,
"ssh", "-o", "StrictHostKeyChecking=no", "-o", "ConnectTimeout=5",
f"{SSH_USER}@{host}", cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
try:
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=timeout)
except asyncio.TimeoutError:
proc.kill()
return f"ERROR: command timed out after {timeout}s"
if proc.returncode != 0:
return f"ERROR: {stderr.decode().strip()}"
return stdout.decode().strip()
@llm.function_tool
async def infra_status(
node: Annotated[str, "Node name: hf-pdocker-01, hf-pdocker-02, bart, gpu, or 'all'"],
) -> str:
"""Check container health on a homelab node. Returns running container count and any unhealthy containers."""
nodes = list(NODES.keys()) if node == "all" else [node]
results = []
for n in nodes:
ip = NODES.get(n)
if not ip:
results.append(f"{n}: unknown node")
continue
out = await _ssh_exec(ip, "docker ps --format '{{.Names}}\\t{{.Status}}' | head -50")
lines = [l for l in out.splitlines() if l.strip()]
unhealthy = [l for l in lines if "unhealthy" in l.lower() or "exited" in l.lower()]
results.append(
f"{n}: {len(lines)} containers running"
+ (f", ISSUES: {'; '.join(l.split(chr(9))[0] for l in unhealthy)}" if unhealthy else ", all healthy")
)
return "\n".join(results)
@llm.function_tool
async def container_action(
node: Annotated[str, "Node name: hf-pdocker-01, hf-pdocker-02, bart, gpu"],
container: Annotated[str, "Container name"],
action: Annotated[str, "Action: restart, stop, or start"],
) -> str:
"""Restart, stop, or start a Docker container on a homelab node."""
ip = NODES.get(node)
if not ip:
return f"Unknown node: {node}"
if action not in ("restart", "stop", "start"):
return f"Invalid action: {action}. Use restart, stop, or start."
result = await _ssh_exec(ip, f"docker {action} {container}")
return f"{action} {container} on {node}: {'done' if container in result or result == '' else result}"
@llm.function_tool
async def check_logs(
node: Annotated[str, "Node name"],
container: Annotated[str, "Container name"],
lines: Annotated[int, "Number of log lines to return (max 50)"] = 20,
) -> str:
"""Get recent logs from a container."""
ip = NODES.get(node)
if not ip:
return f"Unknown node: {node}"
lines = min(lines, 50)
return await _ssh_exec(ip, f"docker logs --tail {lines} {container} 2>&1")
@llm.function_tool
async def read_alerts() -> str:
"""Check Prometheus Alertmanager for active alerts."""
try:
async with httpx.AsyncClient(timeout=5.0) as client:
r = await client.get(f"{ALERTMANAGER_URL}/api/v2/alerts?active=true&silenced=false")
r.raise_for_status()
alerts = r.json()
if not alerts:
return "No active alerts."
parts = []
for a in alerts[:10]:
name = a.get("labels", {}).get("alertname", "unknown")
severity = a.get("labels", {}).get("severity", "")
summary = a.get("annotations", {}).get("summary", "")
parts.append(f"[{severity}] {name}: {summary}" if summary else f"[{severity}] {name}")
return "\n".join(parts)
except Exception as e:
return f"Could not reach Alertmanager: {e}"
@llm.function_tool
async def home_assistant(
action: Annotated[str, "get_states, turn_on, turn_off, or toggle"],
entity_id: Annotated[str, "HA entity ID, e.g. light.living_room or 'all' for get_states"] = "all",
) -> str:
"""Interact with Home Assistant — check sensor states or control devices."""
if not HA_TOKEN:
return "HA_TOKEN not configured."
headers = {"Authorization": f"Bearer {HA_TOKEN}", "Content-Type": "application/json"}
try:
async with httpx.AsyncClient(timeout=5.0) as client:
if action == "get_states":
if entity_id == "all":
r = await client.get(f"{HA_URL}/api/states", headers=headers)
states = r.json()
# Summarize lights/switches/sensors
lights = [s for s in states if s["entity_id"].startswith("light.")]
sensors = [s for s in states if s["entity_id"].startswith("sensor.")]
on_lights = [s["entity_id"].split(".")[1] for s in lights if s["state"] == "on"]
return (
f"{len(lights)} lights ({len(on_lights)} on: {', '.join(on_lights[:5])}), "
f"{len(sensors)} sensors"
)
else:
r = await client.get(f"{HA_URL}/api/states/{entity_id}", headers=headers)
s = r.json()
return f"{entity_id}: {s.get('state')} (attrs: {list(s.get('attributes', {}).keys())[:3]})"
else:
service = action.replace("turn_", "").replace("toggle", "toggle")
domain = entity_id.split(".")[0]
r = await client.post(
f"{HA_URL}/api/services/{domain}/{action}",
headers=headers,
json={"entity_id": entity_id},
)
return f"{action} {entity_id}: {'done' if r.status_code in (200, 201) else r.text}"
except Exception as e:
return f"HA error: {e}"
@llm.function_tool
async def search_web(
query: Annotated[str, "Search query"],
) -> str:
"""Search the web using the local SearXNG instance."""
try:
async with httpx.AsyncClient(timeout=8.0) as client:
r = await client.get(
f"{SEARXNG_URL}/search",
params={"q": query, "format": "json", "categories": "general"},
)
r.raise_for_status()
data = r.json()
results = data.get("results", [])[:4]
if not results:
return "No results found."
parts = []
for res in results:
title = res.get("title", "")
content = res.get("content", "")[:150]
parts.append(f"• {title}: {content}")
return "\n".join(parts)
except Exception as e:
return f"Search failed: {e}"
ALL_TOOLS = [infra_status, container_action, check_logs, read_alerts, home_assistant, search_web]
@@ -0,0 +1,110 @@
"""Async vocal analysis — runs background, never blocks TTS path."""
import asyncio
import io
import logging
import math
import struct
import wave
from dataclasses import dataclass
from typing import Callable, Optional
import httpx
logger = logging.getLogger(__name__)
SPEECHBRAIN_URL = "http://speechbrain-service:8001"
@dataclass
class VocalState:
emotion: str = "neutral"
confidence: float = 0.0
energy: float = 0.0
pitch_hz: Optional[float] = None
rate: float = 0.0
def to_tag(self) -> str:
parts = [f"emotion={self.emotion}", f"energy={self.energy:.2f}"]
if self.pitch_hz:
parts.append(f"pitch={self.pitch_hz:.0f}hz")
if self.rate:
parts.append(f"rate={'fast' if self.rate > 3 else 'slow' if self.rate < 1.5 else 'normal'}")
return f"[VOCAL {' '.join(parts)}]"
def compute_energy(audio_bytes: bytes) -> float:
if len(audio_bytes) < 2:
return 0.0
samples = struct.unpack_from("<" + "h" * (len(audio_bytes) // 2), audio_bytes)
rms = math.sqrt(sum(x * x for x in samples) / len(samples)) / 32768.0
return round(min(rms * 10, 1.0), 3)
async def analyze_audio(audio_bytes: bytes) -> VocalState:
"""Call SpeechBrain for emotion/pitch. Falls back to energy-only if unavailable."""
energy = compute_energy(audio_bytes)
state = VocalState(energy=energy)
try:
buf = io.BytesIO()
with wave.open(buf, "wb") as wf:
wf.setnchannels(1)
wf.setsampwidth(2)
wf.setframerate(16000)
wf.writeframes(audio_bytes)
wav_bytes = buf.getvalue()
async with httpx.AsyncClient(timeout=3.0) as client:
r = await client.post(
f"{SPEECHBRAIN_URL}/analyze",
files={"audio": ("u.wav", wav_bytes, "audio/wav")},
)
r.raise_for_status()
d = r.json()
state.emotion = d.get("emotion", "neutral")
state.confidence = d.get("emotion_confidence", 0.0)
state.pitch_hz = d.get("pitch_mean_hz")
state.rate = d.get("speaking_rate_approx", 0.0)
except Exception as e:
logger.debug("SpeechBrain unavailable, using energy-only: %s", e)
return state
class VocalAnalyzer:
"""Holds the last completed vocal state; updates async after each utterance."""
def __init__(self, on_update: Optional[Callable[[VocalState], None]] = None):
self._state = VocalState()
self._on_update = on_update
self._audio_buffer: list[bytes] = []
self._collecting = False
@property
def current(self) -> VocalState:
return self._state
def start_collecting(self):
self._collecting = True
self._audio_buffer = []
def feed(self, audio_chunk: bytes):
if self._collecting:
self._audio_buffer.append(audio_chunk)
def stop_collecting_and_analyze(self) -> asyncio.Task:
self._collecting = False
audio = b"".join(self._audio_buffer)
self._audio_buffer = []
return asyncio.create_task(self._analyze_bg(audio))
async def _analyze_bg(self, audio_bytes: bytes):
if not audio_bytes:
return
try:
state = await analyze_audio(audio_bytes)
self._state = state
if self._on_update:
self._on_update(state)
except Exception as e:
logger.warning("Vocal analysis failed: %s", e)
@@ -0,0 +1,141 @@
services:
# livekit-agents 1.5 worker + token/API server
voicecoach-agent:
build:
context: ./agent
image: voicecoach-agent:latest
container_name: voicecoach-agent
restart: unless-stopped
env_file: .env
environment:
- LIVEKIT_URL=ws://voicecoach-livekit:7880
- LIVEKIT_PUBLIC_URL=wss://voice.hobbs.farm
- SPEACHES_URL=http://192.168.86.27:8000
- OPENCLAW_URL=http://voicecoach-openclaw:3100
- DB_PATH=/data/sessions.db
- KOKORO_VOICE=af_nova
volumes:
- /mnt/docker-data/configs/voicecoach/sessions:/data
networks:
- voicecoach
- voicecoach_v1
mem_limit: 1G
memswap_limit: 1G
labels:
- "com.centurylinklabs.watchtower.enable=false"
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
# SvelteKit web client
voicecoach-web:
build:
context: ./web-client
image: voicecoach-web-v2:latest
container_name: voicecoach-web-v2
restart: unless-stopped
ports:
- "8085:80"
depends_on:
- voicecoach-agent
networks:
- voicecoach
mem_limit: 128M
labels:
- "com.centurylinklabs.watchtower.enable=false"
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
# LiveKit server — unchanged from v1
livekit-server:
image: livekit/livekit-server:latest
container_name: voicecoach-livekit
restart: unless-stopped
command:
- --config
- /etc/livekit/livekit.yaml
- --keys
- "${LIVEKIT_API_KEY}: ${LIVEKIT_API_SECRET}"
ports:
- "7880:7880"
- "7881:7881"
- "50000-50100:50000-50100/udp"
volumes:
- ./livekit/livekit.yaml:/etc/livekit/livekit.yaml:ro
networks:
- voicecoach
mem_limit: 512M
labels:
- "com.centurylinklabs.watchtower.enable=false"
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
# OpenClaw — LLM gateway with memory and tools
voicecoach-openclaw:
image: margaret-openclaw:latest
container_name: voicecoach-openclaw
restart: unless-stopped
user: "0:0"
ports:
- "3100:3100"
environment:
- NODE_ENV=production
- HOME=/home/glyptic
- USER=glyptic
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
- OPENCLAW_STATE_DIR=/home/glyptic/.voicecoach
- OPENCLAW_CONFIG_PATH=/home/glyptic/.voicecoach/openclaw.json
volumes:
- /mnt/infrastructure/voicecoach:/home/glyptic/.voicecoach
command: ["sh", "-c", "mkdir -p /home/glyptic && node --max-old-space-size=1500 /usr/local/lib/node_modules/openclaw/dist/index.js gateway --allow-unconfigured"]
networks:
- voicecoach
mem_limit: 2G
memswap_limit: 2G
labels:
- "com.centurylinklabs.watchtower.enable=false"
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
# SpeechBrain — async vocal/emotion analysis (optional, agent falls back gracefully)
speechbrain-service:
build:
context: ./speechbrain-service
image: voicecoach-speechbrain:latest
container_name: voicecoach-speechbrain
restart: unless-stopped
volumes:
- /mnt/docker-data/configs/voicecoach/speechbrain-models:/app/models
environment:
- MODEL_DIR=/app/models
networks:
- voicecoach
mem_limit: 3G
memswap_limit: 3G
labels:
- "com.centurylinklabs.watchtower.enable=false"
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
networks:
voicecoach:
driver: bridge
# Join the v1 voicecoach network to reach the shared livekit + openclaw containers
voicecoach_v1:
external: true
name: voicecoach_voicecoach
@@ -0,0 +1,15 @@
port: 7880
rtc:
tcp_port: 7881
port_range_start: 50000
port_range_end: 50100
use_external_ip: true
logging:
json: false
level: info
room:
max_participants: 5
empty_timeout: 300
departure_timeout: 30
@@ -0,0 +1,11 @@
FROM node:20-alpine AS build
WORKDIR /app
COPY package.json .
RUN npm install --legacy-peer-deps
COPY . .
RUN npm run build
FROM nginx:alpine
COPY --from=build /app/build /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 80
@@ -0,0 +1,16 @@
server {
listen 80;
root /usr/share/nginx/html;
index index.html;
# Proxy API calls to the agent's FastAPI server
location /api/ {
proxy_pass http://voicecoach-agent:8080/api/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
location / {
try_files $uri $uri/ /index.html;
}
}
@@ -0,0 +1,21 @@
{
"name": "voicecoach-v2-web",
"version": "2.0.0",
"private": true,
"scripts": {
"dev": "vite dev",
"build": "vite build",
"preview": "vite preview"
},
"type": "module",
"devDependencies": {
"@sveltejs/adapter-static": "^3.0.0",
"@sveltejs/kit": "^2.0.0",
"@sveltejs/vite-plugin-svelte": "^3.0.0",
"svelte": "^5.0.0",
"vite": "^5.0.0"
},
"dependencies": {
"livekit-client": "^2.5.0"
}
}
@@ -0,0 +1,40 @@
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
:root {
--bg: #0f1117;
--surface: #1a1d27;
--surface2: #22263a;
--accent: #6c63ff;
--accent2: #4ecdc4;
--text: #e8eaf0;
--muted: #7b8094;
--danger: #ff5252;
--success: #4caf50;
--warn: #ffc107;
--radius: 12px;
--font: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif;
}
html, body { height: 100%; background: var(--bg); color: var(--text); font-family: var(--font); }
button {
cursor: pointer;
border: none;
font-family: inherit;
font-size: 0.875rem;
font-weight: 500;
border-radius: var(--radius);
padding: 0.5rem 1.2rem;
transition: opacity 0.15s, transform 0.1s;
}
button:active { transform: scale(0.97); }
button:disabled { opacity: 0.4; cursor: default; }
.btn-primary { background: var(--accent); color: #fff; }
.btn-primary:hover:not(:disabled) { opacity: 0.85; }
.btn-danger { background: var(--danger); color: #fff; }
.btn-ghost { background: var(--surface2); color: var(--text); }
.btn-ghost:hover:not(:disabled) { background: #2d3147; }
a { color: var(--accent2); text-decoration: none; }
a:hover { text-decoration: underline; }
@@ -0,0 +1,12 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1" />
<title>VoiceCoach</title>
%sveltekit.head%
</head>
<body>
<div>%sveltekit.body%</div>
</body>
</html>
@@ -0,0 +1,66 @@
import { Room, RoomEvent, Track } from 'livekit-client';
const LIVEKIT_URL = 'wss://voice.hobbs.farm';
const API_BASE = '/api';
export let room = null;
export async function connect(identity, onEvent) {
const res = await fetch(`${API_BASE}/token`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ identity: identity || `jeremy-${Date.now()}` }),
});
if (!res.ok) throw new Error(`Token fetch failed: ${res.status}`);
const { token, url } = await res.json();
room = new Room({
audioCaptureDefaults: {
echoCancellation: true,
noiseSuppression: true,
autoGainControl: true,
},
});
room.on(RoomEvent.Connected, () => onEvent({ type: 'connected' }));
room.on(RoomEvent.Disconnected, (reason) => onEvent({ type: 'disconnected', reason }));
room.on(RoomEvent.ParticipantConnected, (p) => onEvent({ type: 'participant_joined', identity: p.identity }));
room.on(RoomEvent.ParticipantDisconnected, (p) => onEvent({ type: 'participant_left', identity: p.identity }));
room.on(RoomEvent.TrackSubscribed, (track, _pub, participant) => {
if (track.kind === Track.Kind.Audio) {
const el = track.attach();
el.autoplay = true;
document.body.appendChild(el);
onEvent({ type: 'audio_attached', identity: participant.identity });
}
});
room.on(RoomEvent.TrackUnsubscribed, (track) => {
track.detach().forEach(el => el.remove());
});
room.on(RoomEvent.DataReceived, (payload, participant) => {
try {
const msg = JSON.parse(new TextDecoder().decode(payload));
onEvent(msg);
} catch (_) {}
});
await room.connect(url || LIVEKIT_URL, token);
await room.localParticipant.setMicrophoneEnabled(true);
return room;
}
export async function disconnect() {
if (room) {
await room.disconnect();
room = null;
}
}
export function sendData(payload) {
if (!room?.localParticipant) return;
const data = new TextEncoder().encode(JSON.stringify(payload));
room.localParticipant.publishData(data, { reliable: true });
}
@@ -0,0 +1,2 @@
export const prerender = false;
export const ssr = false;
@@ -0,0 +1,6 @@
<script>
import '../app.css';
let { children } = $props();
</script>
{@render children()}
@@ -0,0 +1,328 @@
<script>
import { connect, disconnect, sendData } from '$lib/livekit.js';
import { onDestroy } from 'svelte';
let status = 'disconnected'; // disconnected | connecting | connected
let mode = 'coaching';
let transcript = []; // {role, text, partial, id}
let tools = []; // {action, detail, result, done}
let vocal = null; // {energy, emotion, pitch_hz}
let error = '';
let nextId = 0;
const MODES = ['coaching', 'vent', 'practice', 'briefing'];
function handleEvent(ev) {
switch (ev.type) {
case 'connected':
status = 'connected';
error = '';
break;
case 'disconnected':
status = 'disconnected';
break;
case 'partial_transcript':
if (ev.speaker === 'user') {
const last = transcript[transcript.length - 1];
if (last && last.partial && last.role === 'user') {
transcript[transcript.length - 1] = { ...last, text: ev.text };
} else {
transcript = [...transcript, { role: 'user', text: ev.text, partial: true, id: nextId++ }];
}
}
break;
case 'final_transcript':
if (ev.speaker === 'user') {
// Replace last partial or add new
const lastIdx = transcript.findLastIndex(t => t.partial && t.role === 'user');
if (lastIdx >= 0) {
transcript = transcript.map((t, i) =>
i === lastIdx ? { ...t, text: ev.text, partial: false } : t
);
} else {
transcript = [...transcript, { role: 'user', text: ev.text, partial: false, id: nextId++ }];
}
}
break;
case 'agent_response':
transcript = [...transcript, { role: 'assistant', text: ev.text, partial: false, id: nextId++ }];
break;
case 'tool_start':
tools = [...tools, { action: ev.action, detail: ev.detail || '', result: '', done: false, id: nextId++ }];
break;
case 'tool_complete':
tools = tools.map(t =>
!t.done && t.action === ev.action ? { ...t, result: ev.result, done: true } : t
);
break;
case 'vocal_state':
vocal = { energy: ev.energy, emotion: ev.emotion, pitch_hz: ev.pitch_hz };
break;
case 'mode_ack':
mode = ev.mode;
break;
}
}
async function startSession() {
status = 'connecting';
error = '';
transcript = [];
tools = [];
try {
await connect(null, handleEvent);
} catch (e) {
status = 'disconnected';
error = e.message;
}
}
async function endSession() {
await disconnect();
status = 'disconnected';
}
function setMode(m) {
mode = m;
sendData({ type: 'mode_change', mode: m });
}
onDestroy(() => disconnect());
$: energyPct = vocal ? Math.round(vocal.energy * 100) : 0;
$: vadRing = status === 'connected' && vocal && vocal.energy > 0.1;
</script>
<main>
<header>
<div class="logo">
<div class="logo-dot" class:connected={status === 'connected'} class:connecting={status === 'connecting'}></div>
<span>VoiceCoach</span>
</div>
<nav>
<a href="/history">History</a>
</nav>
</header>
<div class="content">
<!-- Mode bar -->
<div class="mode-bar" class:disabled={status !== 'connected'}>
{#each MODES as m}
<button
class="mode-chip"
class:active={mode === m}
disabled={status !== 'connected'}
on:click={() => setMode(m)}
>{m}</button>
{/each}
</div>
<!-- Transcript -->
<div class="transcript" id="transcript-scroll">
{#if transcript.length === 0 && status === 'disconnected'}
<p class="hint">Press Start to connect your voice.</p>
{:else if transcript.length === 0 && status !== 'disconnected'}
<p class="hint">Listening…</p>
{/if}
{#each transcript as msg (msg.id)}
<div class="msg msg-{msg.role}" class:partial={msg.partial}>
<span class="msg-label">{msg.role === 'user' ? 'You' : 'Assistant'}</span>
<span class="msg-text">{msg.text}</span>
</div>
{/each}
</div>
<!-- Tool activity feed -->
{#if tools.length > 0}
<div class="tool-feed">
{#each tools.slice(-5) as t (t.id)}
<div class="tool-item" class:done={t.done}>
<span class="tool-icon">{t.done ? '✓' : '⚙'}</span>
<span class="tool-text">{t.action}{t.detail ? ` — ${t.detail}` : ''}</span>
</div>
{/each}
</div>
{/if}
<!-- Vocal meter -->
{#if status === 'connected'}
<div class="vocal-row">
<div class="energy-bar">
<div class="energy-fill" style="width: {energyPct}%"></div>
</div>
{#if vocal && vocal.emotion && vocal.emotion !== 'neutral'}
<span class="emotion-tag">{vocal.emotion}</span>
{/if}
</div>
{/if}
<!-- Error -->
{#if error}
<p class="error">{error}</p>
{/if}
<!-- Controls -->
<div class="controls">
{#if status === 'disconnected'}
<button class="btn-primary big-btn" on:click={startSession}>Start Session</button>
{:else if status === 'connecting'}
<button class="btn-ghost big-btn" disabled>Connecting…</button>
{:else}
<button class="btn-danger big-btn" on:click={endSession}>End Session</button>
{/if}
</div>
</div>
<!-- VAD ring -->
{#if vadRing}
<div class="vad-ring" style="opacity: {Math.min(vocal.energy * 3, 1)}"></div>
{/if}
</main>
<style>
main {
min-height: 100dvh;
display: flex;
flex-direction: column;
max-width: 680px;
margin: 0 auto;
padding: 0 1rem;
position: relative;
}
header {
display: flex;
align-items: center;
justify-content: space-between;
padding: 1rem 0;
border-bottom: 1px solid #2a2d3e;
}
.logo { display: flex; align-items: center; gap: 0.5rem; font-weight: 600; font-size: 1.1rem; }
.logo-dot {
width: 10px; height: 10px;
border-radius: 50%;
background: var(--muted);
transition: background 0.3s;
}
.logo-dot.connecting { background: var(--warn); animation: pulse 1s infinite; }
.logo-dot.connected { background: var(--success); }
nav a { font-size: 0.875rem; color: var(--muted); }
.content { flex: 1; display: flex; flex-direction: column; padding: 1rem 0; gap: 1rem; }
.mode-bar {
display: flex; gap: 0.5rem; flex-wrap: wrap;
}
.mode-bar.disabled { opacity: 0.5; }
.mode-chip {
padding: 0.35rem 0.9rem;
border-radius: 999px;
background: var(--surface2);
color: var(--muted);
font-size: 0.8rem;
text-transform: capitalize;
border: 1px solid transparent;
}
.mode-chip.active {
background: var(--accent);
color: #fff;
border-color: var(--accent);
}
.transcript {
flex: 1;
min-height: 300px;
background: var(--surface);
border-radius: var(--radius);
padding: 1rem;
overflow-y: auto;
display: flex;
flex-direction: column;
gap: 0.75rem;
}
.hint { color: var(--muted); font-size: 0.875rem; text-align: center; margin: auto; }
.msg { display: flex; flex-direction: column; gap: 0.15rem; }
.msg-label {
font-size: 0.7rem;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.05em;
color: var(--muted);
}
.msg-user .msg-label { color: var(--accent2); }
.msg-assistant .msg-label { color: var(--accent); }
.msg-text { font-size: 0.95rem; line-height: 1.5; }
.msg.partial .msg-text { opacity: 0.6; font-style: italic; }
.tool-feed {
background: var(--surface);
border-radius: var(--radius);
padding: 0.75rem 1rem;
display: flex;
flex-direction: column;
gap: 0.3rem;
}
.tool-item {
display: flex; align-items: center; gap: 0.5rem;
font-size: 0.8rem; color: var(--muted);
}
.tool-item.done { color: var(--success); }
.tool-icon { font-size: 0.9rem; }
.vocal-row {
display: flex; align-items: center; gap: 0.75rem;
}
.energy-bar {
flex: 1; height: 6px; border-radius: 3px;
background: var(--surface2); overflow: hidden;
}
.energy-fill {
height: 100%; border-radius: 3px;
background: linear-gradient(90deg, var(--accent2), var(--accent));
transition: width 0.2s;
}
.emotion-tag {
font-size: 0.75rem; color: var(--muted);
background: var(--surface2);
padding: 0.2rem 0.5rem;
border-radius: 999px;
text-transform: capitalize;
}
.error { color: var(--danger); font-size: 0.85rem; }
.controls { display: flex; justify-content: center; padding-bottom: 1.5rem; }
.big-btn {
width: 100%; max-width: 320px; padding: 0.85rem;
font-size: 1rem; border-radius: var(--radius);
}
.vad-ring {
position: fixed; inset: 0; pointer-events: none;
border: 3px solid var(--accent2);
border-radius: inherit;
transition: opacity 0.1s;
}
@keyframes pulse {
0%, 100% { opacity: 1; }
50% { opacity: 0.4; }
}
@media (max-width: 480px) {
.big-btn { max-width: 100%; }
}
</style>
@@ -0,0 +1,164 @@
<script>
import { onMount } from 'svelte';
let sessions = [];
let expanded = null;
let messages = {};
let loading = true;
onMount(async () => {
try {
const res = await fetch('/api/sessions?limit=30');
sessions = await res.json();
} catch (e) {
console.error(e);
} finally {
loading = false;
}
});
async function toggleSession(id) {
if (expanded === id) { expanded = null; return; }
expanded = id;
if (!messages[id]) {
const res = await fetch(`/api/sessions/${id}/messages`);
messages[id] = await res.json();
}
}
function fmt(ts) {
if (!ts) return '';
return new Date(ts * 1000).toLocaleString();
}
function duration(s) {
if (!s.ended_at || !s.started_at) return '';
const mins = Math.round((s.ended_at - s.started_at) / 60);
return mins < 1 ? '<1m' : `${mins}m`;
}
</script>
<main>
<header>
<a href="/" class="back">← Back</a>
<h1>Session History</h1>
</header>
{#if loading}
<p class="hint">Loading…</p>
{:else if sessions.length === 0}
<p class="hint">No sessions yet.</p>
{:else}
<div class="session-list">
{#each sessions as s}
<div class="session-card" class:open={expanded === s.id}>
<button class="session-header" on:click={() => toggleSession(s.id)}>
<div class="session-meta">
<span class="session-date">{fmt(s.started_at)}</span>
<span class="session-chips">
<span class="chip mode-chip">{s.mode || 'coaching'}</span>
{#if duration(s)}
<span class="chip">{duration(s)}</span>
{/if}
</span>
</div>
{#if s.summary}
<p class="session-summary">{s.summary}</p>
{/if}
<span class="toggle-icon">{expanded === s.id ? '▲' : '▼'}</span>
</button>
{#if expanded === s.id && messages[s.id]}
<div class="transcript">
{#each messages[s.id] as msg}
<div class="msg msg-{msg.role}">
<span class="msg-label">
{msg.role === 'user' ? 'You' : 'Assistant'}
{#if msg.vocal_emotion && msg.vocal_emotion !== 'neutral'}
<span class="vocal-badge">{msg.vocal_emotion}</span>
{/if}
</span>
<span class="msg-text">{msg.content}</span>
</div>
{/each}
</div>
{/if}
</div>
{/each}
</div>
{/if}
</main>
<style>
main {
max-width: 680px; margin: 0 auto; padding: 1rem;
min-height: 100dvh;
}
header {
display: flex; align-items: center; gap: 1rem;
padding: 1rem 0; border-bottom: 1px solid #2a2d3e; margin-bottom: 1rem;
}
header h1 { font-size: 1.1rem; font-weight: 600; }
.back { font-size: 0.875rem; color: var(--muted); }
.hint { color: var(--muted); text-align: center; padding: 2rem; }
.session-list { display: flex; flex-direction: column; gap: 0.75rem; }
.session-card {
background: var(--surface);
border-radius: var(--radius);
overflow: hidden;
border: 1px solid transparent;
transition: border-color 0.2s;
}
.session-card.open { border-color: var(--accent); }
.session-header {
width: 100%; text-align: left;
background: none; padding: 1rem;
display: flex; flex-direction: column; gap: 0.4rem;
border-radius: 0; color: var(--text);
position: relative;
}
.session-meta { display: flex; align-items: center; gap: 0.5rem; flex-wrap: wrap; }
.session-date { font-size: 0.8rem; color: var(--muted); }
.session-chips { display: flex; gap: 0.35rem; }
.chip {
font-size: 0.7rem; padding: 0.15rem 0.5rem;
border-radius: 999px; background: var(--surface2); color: var(--muted);
}
.mode-chip { color: var(--accent2); }
.session-summary { font-size: 0.85rem; color: var(--text); line-height: 1.4; }
.toggle-icon {
position: absolute; right: 1rem; top: 1rem;
font-size: 0.7rem; color: var(--muted);
}
.transcript {
padding: 1rem; border-top: 1px solid #2a2d3e;
display: flex; flex-direction: column; gap: 0.75rem;
}
.msg { display: flex; flex-direction: column; gap: 0.1rem; }
.msg-label {
font-size: 0.7rem; font-weight: 600; text-transform: uppercase;
letter-spacing: 0.05em; color: var(--muted);
display: flex; align-items: center; gap: 0.4rem;
}
.msg-user .msg-label { color: var(--accent2); }
.msg-assistant .msg-label { color: var(--accent); }
.msg-text { font-size: 0.9rem; line-height: 1.5; }
.vocal-badge {
font-size: 0.65rem; background: var(--surface2);
padding: 0.1rem 0.4rem; border-radius: 999px;
text-transform: capitalize; color: var(--muted);
font-weight: 400; letter-spacing: 0;
}
</style>
@@ -0,0 +1,7 @@
import adapter from '@sveltejs/adapter-static';
export default {
kit: {
adapter: adapter({ fallback: 'index.html' }),
},
};
@@ -0,0 +1,8 @@
import { sveltekit } from '@sveltejs/kit/vite'
/** @type {import('vite').UserConfig} */
const config = {
plugins: [sveltekit()]
}
export default config
@@ -1,8 +1,10 @@
services: services:
gitea: gitea:
image: gitea/gitea:latest image: gitea/gitea:1.26.1
container_name: gitea container_name: gitea
restart: unless-stopped restart: unless-stopped
labels:
- "com.centurylinklabs.watchtower.enable=false"
ports: ports:
- "3003:3000" - "3003:3000"
- "2222:22" - "2222:22"
+38 -53
View File
@@ -1,67 +1,52 @@
services: services:
unifi: unifi-os-server:
image: lscr.io/linuxserver/unifi-network-application:latest image: ghcr.io/lemker/unifi-os-server:latest
container_name: unifi container_name: unifi-os-server
restart: unless-stopped restart: unless-stopped
cgroup: host
cap_add:
- NET_RAW
- NET_ADMIN
extra_hosts:
host.docker.internal: host-gateway
tmpfs:
- /run:exec
- /run/lock
- /tmp:exec
- /var/lib/journal
- /var/opt/unifi/tmp:size=64m
environment: environment:
- PUID=1000 - UOS_SYSTEM_IP=192.168.86.100
- PGID=1000
- TZ=America/Chicago
- MONGO_USER=${UNIFI_MONGO_USER}
- MONGO_PASS=${UNIFI_MONGO_PASS}
- MONGO_HOST=unifi-db
- MONGO_PORT=27017
- MONGO_DBNAME=unifi
volumes: volumes:
- /mnt/docker-data/configs/unifi/data:/config - /sys/fs/cgroup:/sys/fs/cgroup:rw
# /persistent on NAS (config backups, survives reinstall)
- /mnt/docker-data/configs/unifi-os/persistent:/persistent
# All other dirs local — NFS root_squash blocks the chown calls these services need
- /var/lib/unifi-os/var-log:/var/log
- /var/lib/unifi-os/data:/data
- /var/lib/unifi-os/srv:/srv
- /var/lib/unifi-os/var-lib-unifi:/var/lib/unifi
- /var/lib/unifi-os/var-lib-mongodb:/var/lib/mongodb
- /var/lib/unifi-os/etc-rabbitmq-ssl:/etc/rabbitmq/ssl
ports: ports:
- "8443:8443" - "11443:443"
- "3478:3478/udp"
- "10001:10001/udp"
- "8080:8080" - "8080:8080"
- "1900:1900/udp" - "3478:3478/udp"
- "8843:8843" - "10003:10003/udp"
- "8880:8880"
- "6789:6789" - "6789:6789"
- "8444:8444"
- "8880:8880"
- "8881:8881"
- "8882:8882"
- "5671:5671"
- "5514:5514/udp" - "5514:5514/udp"
depends_on: - "9543:9543"
unifi-db: - "5005:5005"
condition: service_healthy - "127.0.0.1:11084:11084"
networks:
- unifi-internal
mem_limit: 1G
memswap_limit: 1G
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
unifi-db:
image: mongo:7.0
container_name: unifi-db
restart: unless-stopped
user: "1024:100"
volumes:
- /mnt/docker-data/databases/unifi-db:/data/db
networks:
- unifi-internal
labels: labels:
- "com.centurylinklabs.watchtower.enable=false" - "com.centurylinklabs.watchtower.enable=false"
mem_limit: 1G
memswap_limit: 1G
healthcheck:
test: ["CMD-SHELL", "mongosh --eval 'db.adminCommand(\"ping\")' --quiet"]
interval: 30s
timeout: 10s
retries: 3
start_period: 30s
logging: logging:
driver: "json-file" driver: "json-file"
options: options:
max-size: "50m" max-size: "100m"
max-file: "3" max-file: "3"
networks:
unifi-internal:
driver: bridge
@@ -0,0 +1,303 @@
# LinkedStorm — Home Audio Assistant Design
**Date:** 2026-05-27
**Status:** Approved
---
## Context
The goal is a whole-home audio assistant that passively records household conversations, identifies who is speaking, produces searchable transcripts, and maintains a daily activity log. The motivation is to have a reliable, private, locally-hosted record of decisions and conversations that can be queried naturally ("what did we decide about the vacation?") without relying on cloud services.
The system reuses as much existing homelab infrastructure as possible: Whisper STT on the GPU machine, LiveKit for WebRTC, Ollama for LLM queries, and SpeechBrain for audio ML — all already deployed. Speaker labeling mirrors the proven frigate-notify person-identification pattern (CLIP embeddings → voice embeddings, cosine similarity, three-tier matching, review-and-tag UI).
---
## Architecture
**Hybrid approach:** Lightweight WebSocket audio capture from browser tabs → server-side processing pipeline → LiveKit for live monitoring and replay.
### Container Breakdown
Every deployable component is its own container, following the same pattern as VoiceCoach (7 containers) and the rest of the homelab. LinkedStorm is 3 containers:
| Container | Purpose | Port | Host |
|---|---|---|---|
| `linkedstorm-pipeline` | WebSocket ingestion, Silero VAD, Whisper calls, resemblyzer speaker embedding, cosine similarity matching, SQLite writes, LiveKit publish | 8300 | hf-pdocker-01 |
| `linkedstorm-api` | FastAPI web dashboard + REST API, static UI, FTS5 search, RAG queries, speaker labeling | 8310 | hf-pdocker-01 |
| `linkedstorm-cron` | Nightly Ollama summary, unknown speaker clustering, WAV clip purge | — (no port) | hf-pdocker-01 |
All three share a single Docker volume for the SQLite database and clips directory.
### External Services (Reused Without Modification)
| Service | What | Where |
|---|---|---|
| Speaches/Whisper | `POST /v1/audio/transcriptions` | GPU machine :8000 |
| LiveKit | Per-room replay rooms (`linkedstorm-{room}`) | GPU machine :7880 |
| Ollama | RAG queries + daily summaries (`qwen2.5:14b`) | GPU machine :11434 |
Speaker voice embedding uses `resemblyzer` (ECAPA-TDNN, ~50MB) running in-process inside `linkedstorm-pipeline` — no separate microservice, no coupling to VoiceCoach.
### Data Flow
```
Browser tab (capture.html)
└─ MediaRecorder → WebSocket → linkedstorm-pipeline:8300
└─ Silero VAD detects speech segment
└─ [Whisper STT via Speaches ‖ resemblyzer embed] concurrent
└─ cosine similarity → speaker match
└─ SQLite write + WAV clip save
└─ LiveKit publish → dashboard replay
linkedstorm-api:8310 ← reads SQLite → serves dashboard + API
linkedstorm-cron ← runs nightly at midnight via entrypoint sleep loop
```
---
## Components
### 1. Browser Capture Page (`linkedstorm-capture`)
Single HTML page opened on any device to make it a room microphone.
- `MediaRecorder` at 16kHz mono, 250ms chunks streamed over WebSocket
- Room name set via URL param (`?room=Kitchen`) or prompt on first load
- Persistent room name stored in `localStorage`
- Mute toggle: stops `MediaRecorder`, sends `{"type":"mute"}` control message, server discards buffered audio for that room
- Visual indicator: green pulsing dot (streaming) / red dot (muted) / gray (disconnected)
- Auto-reconnect with exponential backoff on WebSocket drop
### 2. Audio Ingestion Service (`linkedstorm-ingestion`)
Python asyncio service on hf-pdocker-01. Single process, per-room coroutines.
- WebSocket endpoint: `ws://linkedstorm:8300/ws/{room_id}`
- Per-room `asyncio.Queue` of audio chunks
- **Silero VAD** (reused from VoiceCoach): processes each 250ms chunk; accumulates on speech-detected, emits segment on ≥600ms silence; discards segments <0.5s
- On segment emit: calls `process_utterance(room_id, audio_bytes, start_time, end_time)`
- Room registry: upsert `rooms` table on first connection, update `last_seen` on each chunk, set `is_active=False` on disconnect
### 3. Processing Pipeline
Runs as an async task per emitted speech segment.
**Step 1 — Concurrent inference** (asyncio gather):
- `POST http://192.168.86.150:8000/v1/audio/transcriptions` with audio WAV → transcript text
- `resemblyzer.preprocess_wav()` + `encoder.embed_utterance()` in-process → voice embedding (256-dim float32 array)
**Step 2 — Speaker matching:**
- Load all `voice_embeddings` rows from SQLite
- Compute cosine similarity between utterance embedding and each reference
- Group by `speaker_id`, take MAX per speaker
- Thresholds: ≥0.85 → `known`, 0.60–0.84 → `ambiguous`, <0.60 → `unknown`
**Step 3 — Persist:**
- Save audio clip to `/data/linkedstorm/clips/{date}/{room_id}/{utterance_id}.wav`
- Insert `utterances` row: transcript, embedding, room_id, speaker_id, match_status, match_confidence, start_time, end_time, audio_clip_path
**Step 4 — LiveKit publish:**
- Publish audio segment to LiveKit room `linkedstorm-{room_id}` for live monitoring/replay
### 4. Web UI Service (`linkedstorm-ui`)
FastAPI on hf-pdocker-01 port 8310. Static HTML/JS frontend, JSON API backend. Same pattern as frigate-notify.
**API endpoints:**
```
GET /api/utterances # list with filters: date, room_id, speaker_id, match_status
GET /api/utterances/{id}/clip # stream audio clip WAV
POST /api/utterances/{id}/tag # label utterance: {speaker_id} or {name, create_new: true}
POST /api/utterances/{id}/dismiss # mark as noise/unknown, no review needed
GET /api/speakers # list all known speakers with utterance counts
POST /api/speakers # create new speaker
GET /api/rooms # list rooms with is_active, last_seen
POST /api/rooms/{id}/mute # send mute command to room's WebSocket connection
GET /api/search?q=... # full-text search across utterances.transcript (SQLite FTS5)
POST /api/ask # RAG query: builds context from utterances, calls Ollama
GET /api/summary/{date} # fetch or trigger daily summary for a date
```
**UI screens (5 tabs, single-page app):**
1. **Daily Log** — timeline grouped by time blocks, color-coded by speaker, inline Play/Tag/Dismiss on unknowns
2. **Search** — full-text with keyword highlighting, date-range filter
3. **Speakers** — known speaker cards + review-queue banner for unreviewed utterances
4. **Ask AI** — RAG chat backed by Ollama, responses cite room + timestamp sources
5. **Rooms** — live capture device status, per-room mute toggle, link to open capture page
### 5. Nightly Cron Jobs
Run at midnight via cron on hf-pdocker-01.
**Daily summary:** Fetch all utterances for the day → batch into prompt → send to Ollama (`qwen2.5:14b`) → store in `daily_summaries`. Prompt instructs the model to summarize key decisions, activities, and topics — not to transcribe verbatim.
**Unknown speaker clustering:** Load all `unknown` and `ambiguous` utterance embeddings from the past 7 days → cluster by cosine similarity (threshold 0.70) → group consistent voices as `Unknown #1`, `Unknown #2`, etc. in the review queue rather than scattered individuals.
---
## Data Model
SQLite at `/data/linkedstorm/linkedstorm.db`. WAL mode, foreign keys enabled. Thread-local connections (same pattern as frigate-notify).
```sql
CREATE TABLE rooms (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
device_label TEXT,
is_active BOOLEAN DEFAULT 0,
last_seen DATETIME,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE speakers (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE voice_embeddings (
id INTEGER PRIMARY KEY,
speaker_id INTEGER NOT NULL REFERENCES speakers(id),
embedding BLOB NOT NULL, -- packed float32
audio_sample BLOB, -- short reference WAV
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE utterances (
id INTEGER PRIMARY KEY,
room_id INTEGER NOT NULL REFERENCES rooms(id),
speaker_id INTEGER REFERENCES speakers(id), -- nullable
transcript TEXT NOT NULL,
embedding BLOB, -- packed float32
match_status TEXT DEFAULT 'unreviewed', -- known/unknown/ambiguous/unreviewed
match_confidence REAL,
start_time DATETIME NOT NULL,
end_time DATETIME NOT NULL,
audio_clip_path TEXT,
livekit_room TEXT,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE VIRTUAL TABLE utterances_fts USING fts5(
transcript, content=utterances, content_rowid=id
);
CREATE TABLE daily_summaries (
id INTEGER PRIMARY KEY,
date DATE NOT NULL UNIQUE,
summary_text TEXT NOT NULL,
generated_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
```
---
## Speaker Identification
Mirrors frigate-notify's CLIP embedding pattern, adapted for voice:
| Similarity | Status | Action |
|---|---|---|
| ≥ 0.85 | `known` | Auto-label speaker. Store utterance embedding as additional reference. |
| 0.60–0.84 | `ambiguous` | Store utterance. Appear in review queue with Play + Tag buttons. |
| < 0.60 | `unknown` | Store utterance. Cluster with other unknowns nightly. |
| No embed | `unreviewed` | SpeechBrain unavailable. Transcript stored, embed missing. |
**Labeling flow (same as frigate-notify tag workflow):**
`POST /api/utterances/{id}/tag` with `{speaker_id}` or `{name, create_new: true}` →
sets `match_status='known'`, `match_confidence=1.0`, stores utterance's embedding in `voice_embeddings` for future matching.
---
## Deployment
**3 containers on hf-pdocker-01**
**Compose file:** `compose/hf-pdocker-01/linkedstorm/docker-compose.yml`
**Ports:**
- `8300` — `linkedstorm-pipeline` WebSocket ingestion (capture devices connect here)
- `8310` — `linkedstorm-api` FastAPI web UI and API
**Volumes:**
- `/data/linkedstorm/` → `/mnt/docker-data/configs/linkedstorm/` (NAS)
- `linkedstorm.db` — SQLite database
- `clips/` — audio clip WAV files
**Clip retention:** A VAD-triggered household produces roughly 50–200MB of WAV clips per day. The nightly cron should delete clips older than 30 days (configurable via `CLIP_RETENTION_DAYS` env var) while keeping the transcript and embedding rows permanently. The `audio_clip_path` column is set to `NULL` for purged clips; the Play button in the UI shows "clip expired" for those utterances.
**Environment:**
```
SPEACHES_URL=http://192.168.86.150:8000
CLIP_RETENTION_DAYS=30
OLLAMA_URL=http://192.168.86.150:11434
LIVEKIT_URL=ws://192.168.86.150:7880
LIVEKIT_API_KEY=...
LIVEKIT_API_SECRET=...
```
**Traefik:** `linkedstorm.hobbs.farm` → port 8310 (dashboard). Capture page accessed at `linkedstorm.hobbs.farm/capture?room=Kitchen`.
**Cron entries on hf-pdocker-01:**
```
0 0 * * * docker exec linkedstorm python -m cron.daily_summary
5 0 * * * docker exec linkedstorm python -m cron.cluster_unknowns
10 0 * * * docker exec linkedstorm python -m cron.purge_clips
```
---
## File Structure
```
compose/hf-pdocker-01/linkedstorm/
├── docker-compose.yml
├── .env.example
├── pipeline/
│ └── Dockerfile # linkedstorm-pipeline image
├── api/
│ └── Dockerfile # linkedstorm-api image
└── cron/
└── Dockerfile # linkedstorm-cron image
# Shared app code (mounted into pipeline + api containers, or baked into images)
app/
├── config.py # Settings from env vars (shared)
├── database.py # SQLite thread-local connections, schema init (shared)
├── models.py # Dataclasses for rooms, speakers, utterances (shared)
│
├── pipeline/ # linkedstorm-pipeline
│ ├── main.py # asyncio entrypoint: WebSocket server + per-room VAD loops
│ ├── ingestion.py # WebSocket handler, per-room audio buffer + Silero VAD
│ ├── processor.py # process_utterance: Whisper + resemblyzer + matching + persist
│ ├── recognition.py # resemblyzer embed + cosine similarity, speaker matching
│ └── livekit.py # Publish audio segments to LiveKit rooms
│
├── api/ # linkedstorm-api
│ ├── main.py # FastAPI entrypoint, mounts routes, serves static files
│ ├── routes/
│ │ ├── utterances.py # /api/utterances CRUD + tag + dismiss
│ │ ├── speakers.py # /api/speakers CRUD
│ │ ├── rooms.py # /api/rooms status + mute relay to pipeline
│ │ ├── search.py # /api/search (SQLite FTS5)
│ │ └── rag.py # /api/ask, /api/summary
│ └── static/
│ ├── index.html # Dashboard SPA (5-tab layout)
│ ├── capture.html # Minimal capture page (one per room device)
│ └── app.js / style.css
│
└── cron/ # linkedstorm-cron
├── main.py # Sleep-loop entrypoint: runs all jobs at midnight
├── daily_summary.py # Ollama daily summary generation
├── cluster_unknowns.py # Nightly unknown speaker clustering
└── purge_clips.py # Delete WAV files older than CLIP_RETENTION_DAYS
```
---
## Verification
1. **Capture → ingestion:** Open `capture.html?room=Kitchen` in a browser, speak — confirm WebSocket connects, room appears active in the Rooms tab of the dashboard
2. **VAD:** Speak briefly then stay silent — confirm a single utterance appears in the log (not multiple fragments), and background noise is not logged
3. **Transcription:** Read a known sentence — confirm transcript text in the utterance matches
4. **Speaker labeling:** Tag an unknown utterance as "Jeremy" → speak again → confirm next utterances auto-resolve to Jeremy with high confidence
5. **Mute:** Click mute on the capture page — confirm no utterances logged while muted, muted status shown in Rooms tab
6. **Search:** Search for a word spoken earlier — confirm it appears highlighted in results
7. **Ask AI:** Ask "What did we talk about today?" — confirm Ollama returns a coherent summary with room/time citations
8. **LiveKit replay:** From the dashboard, join a room's LiveKit channel — confirm processed audio plays back
9. **Nightly cron:** Trigger `daily_summary` manually — confirm `daily_summaries` row appears and is shown in the log view
10. **Prometheus probe:** Add `http://192.168.86.192:8310/health` to blackbox targets and confirm it's green
+7 -6
View File
@@ -118,27 +118,28 @@ groups:
summary: "{{ $labels.hostname }} rebooted" summary: "{{ $labels.hostname }} rebooted"
description: "{{ $labels.hostname }} was rebooted less than 10 minutes ago." description: "{{ $labels.hostname }} was rebooted less than 10 minutes ago."
# High swap usage # High swap usage - uses absolute free swap threshold to avoid node-exporter v1.10.x
# SwapTotal double-counting bug on swapfile systems. Triggers when free swap < 1.5GB.
- alert: HighSwapUsage - alert: HighSwapUsage
expr: (node_memory_SwapTotal_bytes > 0) and ((node_memory_SwapTotal_bytes - node_memory_SwapFree_bytes) / node_memory_SwapTotal_bytes * 100 > 50) expr: (node_memory_SwapTotal_bytes > 0) and (node_memory_SwapFree_bytes < 1.5 * 1024 * 1024 * 1024)
for: 5m for: 5m
labels: labels:
severity: warning severity: warning
type: server type: server
annotations: annotations:
summary: "{{ $labels.hostname }}: High swap usage" summary: "{{ $labels.hostname }}: High swap usage"
description: "Swap usage on {{ $labels.hostname }} is above 50% for 5 minutes. Check container memory limits." description: "Swap usage on {{ $labels.hostname }} is high (free swap below 1.5GB for 5 minutes). Check container memory limits."
# Critical swap usage # Critical swap usage - triggers when free swap < 512MB
- alert: CriticalSwapUsage - alert: CriticalSwapUsage
expr: (node_memory_SwapTotal_bytes > 0) and ((node_memory_SwapTotal_bytes - node_memory_SwapFree_bytes) / node_memory_SwapTotal_bytes * 100 > 80) expr: (node_memory_SwapTotal_bytes > 0) and (node_memory_SwapFree_bytes < 512 * 1024 * 1024)
for: 2m for: 2m
labels: labels:
severity: critical severity: critical
type: server type: server
annotations: annotations:
summary: "{{ $labels.hostname }}: Critical swap usage" summary: "{{ $labels.hostname }}: Critical swap usage"
description: "Swap usage on {{ $labels.hostname }} is above 80%. System may become unresponsive." description: "Swap usage on {{ $labels.hostname }} is critical (free swap below 512MB). System may become unresponsive."
# NFS mount failure (all data is on NAS - this is critical) # NFS mount failure (all data is on NAS - this is critical)
- alert: NFSMountDown - alert: NFSMountDown
+4 -10
View File
@@ -57,7 +57,6 @@ scrape_configs:
- http://192.168.86.192:5000 # Frigate NVR (standalone, host network) - http://192.168.86.192:5000 # Frigate NVR (standalone, host network)
- http://192.168.86.192:5199/health # Frigate Notify - http://192.168.86.192:5199/health # Frigate Notify
- http://192.168.86.192:5200/health # clip-service (CLIP embeddings) - http://192.168.86.192:5200/health # clip-service (CLIP embeddings)
- http://192.168.86.192:3010 # Open WebUI
- http://192.168.86.192:8888/health # Farm Manager API - http://192.168.86.192:8888/health # Farm Manager API
- http://192.168.86.192:8889/health # Farm Agent - http://192.168.86.192:8889/health # Farm Agent
- http://192.168.86.192:8123 # Home Assistant - http://192.168.86.192:8123 # Home Assistant
@@ -69,8 +68,10 @@ scrape_configs:
- http://192.168.86.192:3001 # Grafana - http://192.168.86.192:3001 # Grafana
- http://192.168.86.192:9095 # Prometheus - http://192.168.86.192:9095 # Prometheus
- http://192.168.86.192:9093 # Alertmanager - http://192.168.86.192:9093 # Alertmanager
- http://192.168.86.192:18815 # Travis (NemoClaw agent)
- http://192.168.86.192:18795 # Spirit (infra ops agent) - http://192.168.86.192:18795 # Spirit (infra ops agent)
- http://192.168.86.192:8085 # VoiceCoach web client
- http://192.168.86.192:8181/health # Rongorongo (probe internal — Traefik basicAuth blocks external)
- http://192.168.86.192:7880 # VoiceCoach LiveKit server
relabel_configs: relabel_configs:
- source_labels: [__address__] - source_labels: [__address__]
target_label: __param_target target_label: __param_target
@@ -93,9 +94,6 @@ scrape_configs:
- http://192.168.86.100:5006 # Actual Budget - http://192.168.86.100:5006 # Actual Budget
- http://192.168.86.100:8001/openapi.json # MCPO - http://192.168.86.100:8001/openapi.json # MCPO
- http://192.168.86.100:8889/health # Farm Agent - http://192.168.86.100:8889/health # Farm Agent
- http://192.168.86.100:8888 # SearXNG
- http://192.168.86.100:8000/healthcheck # Unstructured API
- http://192.168.86.100:9000 # Portainer
relabel_configs: relabel_configs:
- source_labels: [__address__] - source_labels: [__address__]
target_label: __param_target target_label: __param_target
@@ -135,10 +133,6 @@ scrape_configs:
- http://192.168.86.167:5055 # Jellyseerr - http://192.168.86.167:5055 # Jellyseerr
- http://192.168.86.167:9392 # OpenVAS - http://192.168.86.167:9392 # OpenVAS
- http://192.168.86.167:8889/health # Farm Agent - http://192.168.86.167:8889/health # Farm Agent
- targets:
- http://192.168.86.167:3005 # Onyx
labels:
name: onyx
relabel_configs: relabel_configs:
- source_labels: [__address__] - source_labels: [__address__]
target_label: __param_target target_label: __param_target
@@ -159,7 +153,7 @@ scrape_configs:
- https://yoda.hobbs.farm # Home Assistant - https://yoda.hobbs.farm # Home Assistant
- https://vault.hobbs.farm # Vaultwarden - https://vault.hobbs.farm # Vaultwarden
- https://git.hobbs.farm # Gitea - https://git.hobbs.farm # Gitea
- https://open.hobbs.farm # Open WebUI - https://coach.hobbs.farm # VoiceCoach web client
relabel_configs: relabel_configs:
- source_labels: [__address__] - source_labels: [__address__]
target_label: __param_target target_label: __param_target
+15
View File
@@ -0,0 +1,15 @@
http:
routers:
linkedstorm:
rule: "Host(`linkedstorm.hobbs.farm`)"
entryPoints:
- websecure
service: linkedstorm
tls:
certResolver: letsencrypt
services:
linkedstorm:
loadBalancer:
servers:
- url: "http://192.168.86.192:8310"
+35
View File
@@ -0,0 +1,35 @@
http:
routers:
rongorongo:
rule: "Host(`rongorongo.hobbs.farm`)"
entryPoints:
- websecure
middlewares:
- rongorongo-auth
- security-headers
service: rongorongo
tls:
certResolver: letsencrypt
rongorongo-http:
rule: "Host(`rongorongo.hobbs.farm`)"
entryPoints:
- web
middlewares:
- rongorongo-https-redirect
service: rongorongo
middlewares:
rongorongo-https-redirect:
redirectScheme:
scheme: https
permanent: true
rongorongo-auth:
basicAuth:
realm: "rongorongo.hobbs.farm"
# users entries are appended at deploy time as: <user>:<bcrypt-hash>
users:
- "REPLACE_AT_DEPLOY"
services:
rongorongo:
loadBalancer:
servers:
- url: "http://192.168.86.192:8181"