Compare commits

5 Commits

Author SHA1 Message Date
jeremy e9a98f4855 chore: pin jellyfin to 2026060106
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-03 21:37:46 -05:00
jeremy 97e3cf12a7 frigate: sync config to live state, fix /tmp/cache size, enable VAAPI on house_east/pool_room, throttle shop_west detection
- docker-compose.yml: increase /tmp/cache tmpfs from 4G to 8G, add Coral USB
  device passthrough (/dev/bus/usb), fix memswap_limit to 16G (was equal to mem_limit)
- config.yml: replace OpenVINO detector with Coral edgetpu (usb), update model
  dimensions to 320x320 and pixel format to rgb, enable VAAPI on house_east and
  pool_room, throttle shop_west to 2fps with stricter person/car filters (min_area
  + threshold), add offline note for gate_camera, enable semantic_search (large),
  face_recognition (small), LPR, and bird classification

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-01 15:07:36 -05:00
jeremy 06e9db70d2 feat: replace legacy Unifi with UniFi OS Server v5.1.15
Deploys ghcr.io/lemker/unifi-os-server replacing linuxserver/unifi-network-application.
Data dirs moved to /var/lib/unifi-os/ locally (NFS root_squash blocks chown needed by
postgres/mongodb/nginx init). Only /persistent remains on NAS for config backups.
Access: https://192.168.86.100:11443

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-31 20:36:02 -05:00
jeremy b3bef75981 feat: VoiceCoach v2 — livekit-agents 1.5 personal voice assistant
Replaces pipecat 0.0.48 (6 manual patches, DuplicateIdentity reconnect storms)
with livekit-agents 1.5 SDK. New features:
- SvelteKit UI with live transcript, VAD ring, mode bar, vocal meter, history view
- SQLite session store with WAL mode (cross-session memory)
- Async SpeechBrain vocal/emotion analysis injected per turn
- Homelab tools: SSH/docker, Prometheus alerts, Home Assistant, SearXNG search
- OpenClaw LLM gateway preserved (qwen2.5:14b via Ollama)
- 4 modes: coaching, vent, practice, briefing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-31 18:05:30 -05:00
jeremy 55f05a88d0 infra: migrate GPU workstation to Ubuntu, deploy vLLM-Hermes + voice stack
- Migrated primary workstation (192.168.86.27) from Windows/WSL2 to Ubuntu 24.04 (2026-05-28)
- Upgraded Ollama 0.15.6 → 0.24.0 (all 21 models preserved)
- Fixed Watchtower crash loop (DOCKER_API_VERSION=1.44)
- Fixed Open WebUI OLLAMA_BASE_URL to use Docker service name
- Deployed Speaches :8000 (CUDA, STT/TTS for VoiceCoach)
- Deployed LiveKit :7880/:7881 (WebRTC for VoiceCoach)
- Deployed vLLM-Hermes :8001 (Hermes 4.3 36B AWQ, tool-call-parser hermes)
  - Endpoint: http://192.168.86.27:8001/v1, model: hermes
  - Tool-call test passed: structured tool_calls array confirmed
- Added vllm-local provider to voicecoach-openclaw and margaret OpenClaw configs
- Added OPENAI_API_BASE_URL to Open WebUI for Hermes access in chat UI
- Updated CLAUDE.md: removed WSL2 references, added GPU workstation section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-28 18:48:44 -05:00
127 changed files with 2399 additions and 16100 deletions
-1
View File
@@ -142,7 +142,6 @@ When creating documentation:
2. Full content in markdown format
3. Mermaid diagrams embedded within the documentation
4. Summary of what was documented and any follow-up actions needed
5. Update the change logs
When providing information to other agents:
1. The specific information requested
+1 -2
View File
@@ -36,8 +36,7 @@
"Bash(docker version:*)",
"Bash(/tmp/mcpo-config.json << 'EOF'\n{\n \"servers\": []\n}\nEOF)",
"Bash(/tmp/mcpo-config.json << 'EOF'\n{\n \"mcpServers\": {}\n}\nEOF)",
"Bash(/tmp/mcpo-config.json << 'EOF'\n{\n \"mcpServers\": {\n \"example\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-everything\"],\n \"env\": {}\n }\n }\n}\nEOF)",
"Bash(sleep 20 && sshpass -p '&Rip8a115' ssh -o StrictHostKeyChecking=no glyptic@192.168.86.100 \"docker ps --filter name=n8n --format 'table {{.Names}}\\\\t{{.Status}}' && echo '---' && docker logs n8n --tail 15 2>&1\")"
"Bash(/tmp/mcpo-config.json << 'EOF'\n{\n \"mcpServers\": {\n \"example\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-everything\"],\n \"env\": {}\n }\n }\n}\nEOF)"
]
}
}
-7
View File
@@ -17,10 +17,3 @@ Thumbs.db
.vscode/
*.swp
*.swo
# Tool scratch output (not infrastructure)
.playwright-mcp/
.superpowers/
# Has its own repo: git.hobbs.farm/jeremy/friday-voice.git
/friday-voice/
+49 -46
View File
@@ -23,7 +23,7 @@ This repository manages a **homelab infrastructure** with self-hosted services r
hf-pdocker-01 (23GB RAM, i7-7700T, NVMe, HD 630 GPU) -- RESTORED (2026-03-08, hardware repaired)
├── [COMPOSE] Frigate NVR (:5000) -- Coral USB TPU (edgetpu), ~11ms inference, 13 cameras active
├── [COMPOSE] Frigate-Notify v2 (:5199) -- MQTT->AI->Matrix notifications
├── [REMOVED] clip-service -- decommissioned 2026-07-13 (CLIP embeddings; ~2GB RAM freed, no config references from frigate-notify existed)
├── [COMPOSE] clip-service (:5200) -- CLIP embeddings (ViT-L-14, CPU)
├── [COMPOSE] Home Assistant (:8123)
├── [COMPOSE] ESPHome (:6052)
├── [COMPOSE] Mosquitto (:1883, :8883)
@@ -31,38 +31,47 @@ hf-pdocker-01 (23GB RAM, i7-7700T, NVMe, HD 630 GPU) -- RESTORED (2026-03-08, ha
├── [COMPOSE] Guacamole (:8199) + guacd + guacamole-db (mysql, LOCAL /var/lib/guacamole-db)
├── [COMPOSE] AI Agents (margaret:18790, spirit:18795 [infra ops])
├── [COMPOSE] Traefik (reverse proxy, ports 80/443/8090)
├── [REMOVED] AdGuard Home -- decommissioned 2026-07-13 (conf dir had been empty since 2026-05-17, DNS engine never ran, LAN uses router DNS). Ports 53/3000 freed on .192.
├── [COMPOSE] AdGuard Home (:3000, :53)
├── [COMPOSE] Monitoring (Grafana :3001, Prometheus :9095, Alertmanager :9093, InfluxDB :8086, Blackbox :9115, matrix-receiver :9094)
├── [COMPOSE] Farm Manager API (:8888) -- dashboard + API server
├── [COMPOSE] Farm Agent (:8889)
├── [COMPOSE] Rongorongo decipherment engine -- 2 containers (rongorongo-web :8181, rongorongo-api). Server-synced state at /mnt/docker-data/configs/rongorongo/data. Traefik basicAuth on rongorongo.hobbs.farm.
├── [COMPOSE] LinkedStorm (:8300 pipeline, :8310 api, cron no-port) -- whole-home audio assistant. VAD+Whisper STT+speaker-ID. Deployed 2026-05-28. Public: linkedstorm.hobbs.farm -> :8310.
├── [REMOVED] VoiceCoach -- moved to GPU machine (192.168.86.150) 2026-04-04
├── [COMPOSE] litellm (:4000?) + litellm-db -- LLM proxy (see Known Issue #15)
├── [COMPOSE] synapse-admin -- Synapse admin UI
├── [REMOVED] VoiceCoach -- moved to GPU machine (192.168.86.27) 2026-04-04, now Ubuntu
├── [COMPOSE] Node-exporter (:9100) -- standalone with pid:host
├── [COMPOSE] Telegraf -- metrics collection to InfluxDB
├── [CRON] sync-frigate-to-nas (daily 2am), cleanup-frigate-media (weekly Sun 3am), restart-frigate (4x daily 4am/10am/4pm/10pm, log ~/frigate-restart.log)
├── [CRON] sync-frigate-to-nas (daily 2am), cleanup-frigate-media (weekly Sun 3am), restart-frigate (daily 4am)
├── 5TB USB disk at /mnt/frigate-recordings (Frigate recordings storage)
bart (15GB RAM, i5-4570T, HDD)
├── [COMPOSE] Media Stack (Jellyfin, Sonarr, Radarr, SABnzbd, NZBHydra2, Jellyseerr) -- Sonarr config on LOCAL /var/lib/sonarr, NZBHydra config on LOCAL /var/lib/nzbhydra (2026-07-13), **Jellyfin config on LOCAL /var/lib/jellyfin (2026-07-19)** -- all moved off NFS after SQLite/lock crashes. **Jellyfin PINNED to jellyfin/jellyfin:2026060106 (v12.0.0 unstable)** -- its DB is on the v12 schema (ExtraIds column dropped); :latest is still 10.11.x and breaks on it. Do not unpin until official stable >= 12.
├── [REMOVED] Onyx -- fully decommissioned 2026-07-13 (had been down since ~2026-06-08 unnoticed; containers, volumes, images removed)
├── [REMOVED] OpenVAS -- decommissioned 2026-07-19 per user (containers, volumes incl. PG DB, image removed; ~39GB freed with other cleanup). Compose dir kept for reference. History: pg_upgrade crash-loop (Known Issue #21), DB wiped fresh 2026-07-15.
├── [COMPOSE] Media Stack (Jellyfin, Sonarr, Radarr, SABnzbd, NZBHydra2, Jellyseerr)
├── [COMPOSE] Onyx (:3005) -- redeployed 2026-05-20, all services :latest, Watchtower disabled on all 10
├── [COMPOSE] OpenVAS (vulnerability scanner)
├── [COMPOSE] Farm Agent (:8889)
├── [COMPOSE] Node-exporter (:9100) -- standalone with pid:host
├── [COMPOSE] Telegraf -- metrics collection to InfluxDB
├── [COMPOSE] Watchtower
GPU workstation / Jeremy's PC (192.168.86.27, Ubuntu 24.04 — migrated from Windows 2026-05-28)
├── RTX 3090 (24GB VRAM), i7-13700KF, 32GB RAM
├── [COMPOSE] Ollama (:11434) -- 21 local models, v0.24.0
├── [COMPOSE] Open WebUI (:8080) -- AI chat, points to local Ollama
├── [REMOVED] vLLM-Hermes -- removed 2026-05-29, VRAM freed for Speaches CUDA
├── [COMPOSE] Speaches (:8000) -- STT/TTS via faster-whisper + Kokoro, CUDA
├── [COMPOSE] LiveKit (:7880/:7881) -- WebRTC for VoiceCoach, key: devkey
├── [COMPOSE] Portainer (:9000, :8002)
├── [COMPOSE] Prometheus (:9090), DCGM-exporter (:9400), node-exporter (:9100)
├── [COMPOSE] Watchtower (label-enable mode)
├── Compose file: /home/glyptic/Docker/compose/docker-compose.yml
├── vLLM compose: /home/glyptic/Docker/vllm-hermes/docker-compose.yml
└── Model cache: /home/glyptic/Docker/data/models/
hf-pdocker-02 (7.6GB RAM, i7-6700T, HDD)
├── [COMPOSE] UniFi OS Server (container `unifi-os-server`, HTTPS on host :11443)
├── [COMPOSE] Gitea + gitea-db (PostgreSQL) -- LevelDB queues on LOCAL /var/lib/gitea-queues (2026-07-13, stale NFS lock caused 7,972-restart loop)
├── [COMPOSE] Semaphore + semaphore-db (MySQL, LOCAL /var/lib/semaphore-db since 2026-07-13)
├── [COMPOSE] UniFi OS Server (:11443 HTTPS, :8080 inform, :3478/udp STUN) -- ghcr.io/lemker/unifi-os-server v5.1.15, replaced legacy linuxserver/unifi 2026-05-31. Data local at /var/lib/unifi-os/, /persistent on NAS.
├── [COMPOSE] Gitea + gitea-db (PostgreSQL)
├── [COMPOSE] Semaphore + semaphore-db (MySQL)
├── [COMPOSE] Wiki.js + wikijs-db (PostgreSQL)
├── [COMPOSE] Vaultwarden, MCPO, Trivy
├── [REMOVED] Actual Budget -- decommissioned 2026-07-13 (data kept at /mnt/docker-data/configs/actual-budget)
├── [COMPOSE] Vaultwarden, Actual Budget, MCPO, Trivy
├── [COMPOSE] n8n + n8n-db (PostgreSQL) -- workflow automation
├── [COMPOSE] CouchDB (:5984) -- Obsidian LiveSync backend for the Second Brain vault. LAN-only. LOCAL disk (NOT NFS).
├── [COMPOSE] Farm Agent (:8889)
├── [COMPOSE] Node-exporter (:9100) -- standalone with pid:host
├── [COMPOSE] Telegraf -- metrics collection to InfluxDB
@@ -74,9 +83,11 @@ hf-pdocker-02 (7.6GB RAM, i7-6700T, HDD)
### Environment Context
You are working in a **WSL 2 environment** on Windows. Docker is NOT directly available in WSL - all Docker operations must be performed by connecting to the remote nodes via SSH.
You are working on a **native Ubuntu 24.04 workstation** (previously WSL2 on Windows — migrated 2026-05-28). This machine IS the GPU inference box (RTX 3090, 192.168.86.27). Docker is available locally and runs the inference stack. All homelab node operations (hf-pdocker-01, hf-pdocker-02, bart) still require SSH.
**Critical:** Docker is not available locally in WSL. All Docker operations must be performed by connecting to the remote nodes via SSH.
**Local Docker on this machine** runs: Ollama, Open WebUI, vLLM-Hermes, Speaches, LiveKit, Portainer, Prometheus, DCGM-exporter.
**Critical:** For homelab nodes, use the `docker-swarm-ops` agent via SSH. For local inference services, Docker commands run directly.
### Storage Architecture
@@ -145,21 +156,22 @@ compose/ # Standalone Compose definitions (deployed per-node)
- Home Assistant: `yoda.hobbs.farm`
- Vaultwarden: `vault.hobbs.farm`
- Gitea: `git.hobbs.farm`
- FRIDAY Voice: `coach.hobbs.farm` (replaced VoiceCoach 2026-07-05; runs on .150 :8320)
- VoiceCoach: `coach.hobbs.farm` (LiveKit WebRTC: `voice.hobbs.farm`)
- Rongorongo: `rongorongo.hobbs.farm`
- LinkedStorm: `linkedstorm.hobbs.farm`
### Internal Services by Node
**hf-pdocker-01 (192.168.86.192):**
- Frigate: :5000
- Frigate Alerts (v2): :5199 (web UI + REST API)
- clip-service: :5200 (CLIP embeddings, ViT-L-14, CPU)
- Home Assistant: :8123
- ESPHome: :6052
- Mosquitto: :1883
- Matrix/Synapse: :8008
- Well-known (Matrix federation): :8010
- Guacamole: :8199
- AdGuard Home: :3000 (DNS: :53)
- Traefik Dashboard: :8090 (HTTP: :80, HTTPS: :443)
- Grafana: :3001
- Prometheus: :9095
@@ -171,19 +183,18 @@ compose/ # Standalone Compose definitions (deployed per-node)
- Farm Manager: :8888 (dashboard + API)
- Farm Agent: :8889
- Rongorongo: :8181 (rongorongo-web nginx; reverse-proxies /api/* to rongorongo-api Flask backend on the internal `rongorongo` bridge net). State + images at /mnt/docker-data/configs/rongorongo/data/. Public URL gated by Traefik basicAuth (creds in Vaultwarden).
- LinkedStorm: :8300 (pipeline/VAD+STT+speaker-ID), :8310 (API/dashboard), cron (no port). Public: linkedstorm.hobbs.farm -> :8310. Source: /opt/linkedstorm (rsync from WSL). Data: /mnt/docker-data/configs/linkedstorm/. STT via Speaches GPU machine :8000, LLM via Ollama :11434 (qwen2.5:14b).
- ~~VoiceCoach~~ -- fully decommissioned 2026-07-05, replaced by FRIDAY Voice on .150 (see GPU Machine section + memory `friday-voice`)
- ~~VoiceCoach~~ -- moved to GPU machine (192.168.86.150) 2026-04-04
**hf-pdocker-02 (192.168.86.100):**
- Farm Agent: :8889
- UniFi OS Server: :11443 (HTTPS, container 443)
- Unifi: :8443
- Gitea: :3003 (web), :2222 (SSH)
- Semaphore: :3004
- Wiki.js: :3002
- Vaultwarden: :26226
- Actual Budget: :5006
- MCPO: :8001
- Trivy: :4954
- CouchDB: :5984 -- Obsidian LiveSync backend. DB `secondbrain`; admin creds in Vaultwarden ("CouchDB - Obsidian LiveSync"). LAN-only (no Traefik route). Data on LOCAL disk `/var/lib/couchdb` (NFS breaks CouchDB file locking). **`local.ini` must be mounted READ-WRITE** -- with `:ro` CouchDB exits 1 and logs nothing. Auth required on everything except `/_up` (left open for the blackbox probe). Vault repo: git.hobbs.farm/jeremy/second-brain (private); vault lives at `C:\Users\Jeremy\second-brain` on the Windows box.
**bart (192.168.86.167):**
- Farm Agent: :8889
@@ -193,21 +204,25 @@ compose/ # Standalone Compose definitions (deployed per-node)
- SABnzbd: :8082
- NZBHydra2: :5076
- Jellyseerr: :5055
- OpenVAS: :9392
**GPU Machine (192.168.86.150, RTX 3090):**
- **NOTE (2026-07-05):** .150 is actually the user's **Windows workstation** (runs the WSL/Claude Code session), not a headless Linux server. It runs **Docker Desktop** (WSL2 backend, RTX 3090 GPU passthrough working) alongside **UniFi OS Server** (Podman). Docker Desktop panics if UniFi/Podman claims `\\.\pipe\docker_engine` first — start Docker Desktop before UniFi OS Server. See memory `docker-desktop-unifi-podman-conflict`.
- **FRIDAY Voice replaced VoiceCoach (2026-07-05).** VoiceCoach fully decommissioned (v1+v2 removed from hf-pdocker-01; voice.hobbs.farm route removed). See memory `friday-voice`.
- [COMPOSE] **FRIDAY Voice stack (all on .150 Docker Desktop, via `docker.exe -c desktop-linux`):**
- **voicecoach-speaches-gpu**: :8000 — GPU Whisper large-v3-turbo (STT) + Kokoro-82M af_nova (TTS), CUDA float16. Compose: `compose/gpu-machine/voicecoach-speaches/`.
- **hermes**: `nousresearch/hermes-agent:latest` gateway — FRIDAY's reasoning agent. Settings transferred from WSL `~/.hermes` into named volume `hermes-data`. Compose: `compose/gpu-machine/hermes/`.
- **friday-voice**: :8320 — push-to-talk console. Calls speaches (`host.docker.internal:8000`) + hermes (`docker exec hermes hermes -Q` via wrapper). Public: **coach.hobbs.farm** → :8320. Compose: `compose/gpu-machine/friday-voice/`.
- Ollama: :11434 (qwen2.5:14b, Hermes-4.3-36B-GGUF, others)
**GPU Machine (192.168.86.150, RTX 3090, 15GB RAM):**
- Ollama: :11434 (llama3.1:8b for VoiceCoach LLM, qwen2.5:14b, others)
- [COMPOSE] VoiceCoach (all services co-located):
- voicecoach-livekit: :7880/:7881, UDP 50000-50100 (public at voice.hobbs.farm)
- voicecoach-speaches-gpu: :8000 (GPU Whisper STT + Kokoro TTS, CUDA float16)
- voicecoach-speechbrain: :8001 (emotion analysis)
- voicecoach-token-server: :8080 (token + settings API)
- voicecoach-pipecat: pipeline agent (LLM via local Ollama llama3.1:8b)
- voicecoach-web: :8085 (public at coach.hobbs.farm)
- voicecoach-openclaw: :3100 (idle, kept for future Claude API routing)
- Compose: ~/voicecoach-speaches/docker-compose.yml
### Port Conflict Notes
Port conflicts already resolved:
- Port 8000: Freed (MCPO moved to 8001, Portainer moved elsewhere)
- Port 3000: freed 2026-07-13 (was AdGuard Home, decommissioned; Gitea remains on 3003)
- Port 3000: AdGuard Home (Gitea moved to 3003)
- Port 3001: Grafana (Semaphore moved to 3004)
- Port 5000: Frigate uses host networking (registry container stopped)
- AI Agent ports re-spaced (2026-03-15): openclaw 2026.3.13 uses gateway+2 and gateway+3 for browser/chrome-relay. Agents spaced 5 apart: margaret=18790, spirit=18795
@@ -231,7 +246,6 @@ After ANY infrastructure change (deploy, remove, reconfigure, add service, chang
2. **MEMORY.md** — Add/update relevant memory entries
3. **Prometheus probes** — Add/remove blackbox targets, hot-reload Prometheus
4. **Wiki** — Launch documentation-keeper agent to update wiki pages (run in background)
5. **Farm Manager links page** — If a service was added/removed or its URL/port changed: edit the `serverLinks` array in `/opt/farm-manager/server/static/app.js` on hf-pdocker-01, then rebuild + redeploy: `docker build -f /opt/farm-manager/Dockerfile.server -t farm-manager:latest /opt/farm-manager && cd /mnt/docker-data/compose/hf-pdocker-01/farm-manager && docker compose up -d` (links are baked into the image, not mounted)
Do NOT wait for the user to ask. Do NOT skip "because the change was small." Every change gets documented.
@@ -294,19 +308,8 @@ All persistent data on NAS at `/mnt/docker-data/`:
4. **InfluxDB Connection**: Telegraf may show timeout errors if InfluxDB is not running
5. **Frigate cameras**: Cameras on 192.168.44.x subnet may show RTSP timeouts (check camera network)
6. **Registry container**: Stopped on hf-pdocker-01 to free port 5000 for Frigate
7. **systemd-resolved DNSStubListener**: disabled on hf-pdocker-01 and hf-pdocker-02 via `/etc/systemd/resolved.conf.d/nostub.conf` (`DNSStubListener=no`) — originally for AdGuard Home's port 53. AdGuard was decommissioned 2026-07-13; the stub-listener change is harmless and left in place (re-enable only if something needs 127.0.0.53).
7. **systemd-resolved DNSStubListener**: Must be disabled on any node running AdGuard Home (port 53 conflict). Fix: `/etc/systemd/resolved.conf.d/nostub.conf` with `DNSStubListener=no`, then `systemctl restart systemd-resolved`. Applied on hf-pdocker-01 and hf-pdocker-02.
8. **NFS boot-mount race (hf-pdocker-01)**: NFS mounts failed at boot (network not ready in time). Fixed 2026-05-20 by adding `nofail,x-systemd.after=network-online.target,x-systemd.mount-timeout=60s` to both fstab entries. If mounts are missing after reboot: `sudo mount /mnt/docker-data && sudo mount /mnt/infrastructure`.
9. **Guacamole schema**: App image pinned to `guacamole/guacamole:1.6.0` + `guacd:1.6.0` with Watchtower disabled. DB schema was reset to fresh 1.6.0 on 2026-05-20 (old schema was non-standard). Recreate connections after any DB reset.
10. **Gitea image**: Pinned to `gitea/gitea:1.26.1` with Watchtower disabled (2026-05-20). Watchtower previously pulled broken `:latest` images causing missing git binary errors.
11. **Home Assistant reverse proxy**: Config at `/mnt/docker-data/configs/homeassistant/configuration.yaml` (no hyphen). Must have `http: use_x_forwarded_for: true` + `trusted_proxies: [172.16.0.0/12, 127.0.0.1]` or proxied requests return 400.
12. **hf-pdocker-01 memory pressure (2026-07-05, retrimmed 2026-07-11)**: Node runs hot (~15-18/23GB, chronic swap). Mitigations applied: `vm.swappiness` lowered 60→10 (persisted in `/etc/sysctl.d/99-swappiness.conf`), and VoiceCoach (v1+v2, ~10 containers) fully removed (replaced by FRIDAY Voice on .150). Frigate is the main consumer. **2026-07-11:** Frigate's built-in AI features (`semantic_search: large`, `face_recognition`, `lpr`, `classification.bird`) were found *enabled* (contradicting the intended baseline) and re-disabled — dropped the embeddings CLIP model ~1.2GB→168MB and cut CPU inference; host mem 18→~9GB, swap 3.9→1.3GB right after. Config `/mnt/docker-data/configs/frigate/config/config.yml` (backup `.bak.20260711-aitrim`). Recognition is instead handled externally by frigate-notify v2 (clip-service was removed 2026-07-13 — frigate-notify's config never referenced it). **2026-07-12 — WHOLE-NODE CRASH:** .192 froze hard (~14:15, sshd/HA/DNS all dead, required hard reboot). Cause: Frigate `memswap_limit` had drifted to **16G** vs `mem_limit` 12G, letting it spill ~4GB into host swap; with the ffmpeg leak pinning it at 12G all day (cgroup OOM every ~45min), the host swap-thrashed to death. **Fix: node compose `memswap_limit: 12G` (= mem_limit) so Frigate can never use host swap** — a hit now trims its own ffmpeg (contained cgroup OOM), not the host. Repo already had 12G; node had drifted (never redeployed). NOTE: recording ffmpeg still leaks ~0.8GB/h atop ~6GB baseline (fills 12G in ~7.5h); twice-daily 4am+4pm restart-frigate cron mitigates but doesn't fully beat it — consider every-6h restart + fixing broken cameras (gate_camera, garage_2). Always keep `memswap_limit == mem_limit` on this node.
13. **Docker Desktop vs UniFi OS Server pipe conflict (Windows box .150, 2026-07-05)**: Both fight over `\\.\pipe\docker_engine`; Docker Desktop's backend nil-panics ("backend exited before becoming ready") if Podman wins. Fix: let Docker Desktop (auto-starts at login) claim the pipe first, launch UniFi OS Server manually after. Full detail in memory `docker-desktop-unifi-podman-conflict`.
14. **MySQL InnoDB corruption on NFS**: `semaphore-db` (mysql:8.0 on NFS) hit system-tablespace page corruption and crash-looped 3789× (2026-07-11), same class as the Guacamole-on-NFS issue (#9). Recovered in-place via `--innodb-force-recovery=1` → mysqldump → fresh volume → reimport (DB was empty, no data lost); admin recreated with `semaphore user add`. Any MySQL DB on NFS is at risk — prefer LOCAL storage like guacamole-db. See memory `semaphore-db-nfs-corruption`.
15. **litellm hangs after DB blip**: litellm does NOT self-recover from a transient `litellm-db` disconnect — it loops on "Prisma DB reconnect failed" forever; needs manual `docker restart litellm`. Its healthcheck was fixed from `curl` (absent from image, permanent false-unhealthy) to a python urllib check (2026-07-11).
16. **Stale NFS locks after unclean reboot/power loss (2026-07-12 power event)**: NFS file locks held by pre-reboot client identities survive on the Synology and block relocking — gitea crash-looped 7,972× on its LevelDB queue lock, semaphore-db 975× on `ibdata1`, and nzbhydra2 sat unhealthy on its H2 database lock. Fix applied 2026-07-13: lock-prone data moved to LOCAL disk (gitea queues → `/var/lib/gitea-queues`, semaphore-db → `/var/lib/semaphore-db` on .100; sonarr config → `/var/lib/sonarr`, nzbhydra config → `/var/lib/nzbhydra` on bart; jellyfin config → `/var/lib/jellyfin` on bart 2026-07-19 after 60-100s "database is locked" stalls on jellyfin.db caused playback lockups). **Rule: anything holding long-lived file locks (MySQL, SQLite, LevelDB) goes on local disk, never NFS.**
17. **Synapse rate-limited Alertmanager notifications**: default `rc_message` (0.2 msg/s) made every matrix-alertmanager-receiver send fail with M_LIMIT_EXCEEDED 429 — **no alerts reached Matrix for weeks** while outages accumulated (Onyx down 5 weeks, AdGuard dead 2 months, unnoticed). Fixed 2026-07-13: `rc_message: {per_second: 5, burst_count: 50}` in homeserver.yaml (backup `.bak-20260713-ratelimit`).
18. **AdGuard Home decommissioned 2026-07-13**: its conf dir had been empty since 2026-05-17 (container ran the first-launch wizard, served no DNS, no backup existed) and the LAN had been using router DNS ever since. Container, data, compose dir, and Prometheus probe all removed; ports 53/3000 freed on .192. LAN DNS = the router.
19. **Duplicate failover stacks removed from .100 and bart (2026-07-13)**: .100 ran a second monitoring stack (prometheus/grafana/alertmanager/influxdb/blackbox), an idle traefik (shared acme.json with .192 — ACME conflict risk), and a stray guacamole; bart ran a failover guacamole. All removed (~2GB RAM reclaimed). Compose dirs kept for reference; data volumes untouched.
20. **UniFi OS Server probe port**: `unifi-os-server` on .100 serves HTTPS on host port **11443** (container 443), not 8443. Blackbox probe fixed 2026-07-13. solo.hobbs.farm returns 401 (Traefik basicAuth) when healthy — probed via `http_2xx_or_401` module.
21. **OpenVAS pg_upgrade loop saturated bart's HDD (2026-07-15)**: `immauss/openvas:latest` looped on the PG 13→15 auto-upgrade for ~21h (158 restarts) — each cycle re-copied the 21GB DB, pinning sda at ~90% util/40% iowait and making Jellyfin (same disk) hang. The upgrade itself succeeded every time but the final `mv /data/database /data/database_13_old` failed with "Device or resource busy" because the openvas-db volume is mounted **at** `/data/database` (can't rename a mountpoint). Fixed by wiping the DB (`docker volume rm openvas_openvas-db`) and letting it initdb fresh on PG15 — scan history lost, feeds re-synced. **On the next major PG bump this will recur** unless the upgrade is done out-of-band (content-level swap) or the volume is remounted at `/data`. General rule: a service "hanging" on bart usually means another container is eating the single HDD — check `iostat -x`/`pidstat -d` first. **OpenVAS fully decommissioned 2026-07-19** (moot unless redeployed).
22. **Jellyfin SQLite-on-NFS lockups (fixed 2026-07-19)**: jellyfin.db (WAL mode) on `/mnt/docker-data/configs/jellyfin` hit escalating `SQLite Error 5: 'database is locked'` (22→33→125/day), DB commands blocking 60-102s → healthcheck flaps + playback stalls ("Unexpected end of request content" on /Sessions/Playing/Progress). Config moved to LOCAL `/var/lib/jellyfin` (old copy left on NFS as backup). Same class as Known Issue #16.
-322
View File
@@ -1,322 +0,0 @@
# RONGORONGO_DEPLOY.md
# Rongorongo Decipherment Engine — Claude Code Deployment Guide
# Target: hobbs.farm Docker Swarm infrastructure
# Subdomain: rongorongo.hobbs.farm
---
## CONTEXT
This document instructs Claude Code to build, push, and deploy the Rongorongo
Decipherment Engine as a containerized static web app to the hobbs.farm Swarm.
The app is a single-file HTML/JS application served by nginx. It calls the
Anthropic API directly from the browser. No backend, no database. All session
state is stored in browser localStorage.
Infrastructure assumptions (from existing hobbs.farm stack):
- Docker Swarm: nodes hf-pdocker-01, hf-pdocker-02, bart
- Traefik reverse proxy with wildcard *.hobbs.farm TLS via Cloudflare DNS
- External overlay network: traefik-public
- Registry: registry.hobbs.farm (if local) OR use Docker Hub as fallback
- Portainer CE available for stack management
- DNS: AdGuard Home with hairpin NAT for *.hobbs.farm
---
## REPOSITORY STRUCTURE
After cloning or creating the project, verify this structure exists:
```
rongorongo/
├── Dockerfile
├── nginx.conf
├── stack.yml
├── frontend/
│ └── index.html
└── RONGORONGO_DEPLOY.md (this file)
```
---
## STEP 1 — VERIFY PREREQUISITES
Run these checks before proceeding. Stop and report any failures.
```bash
# Confirm Docker and Swarm context
docker info | grep -E "Swarm:|Manager:|Nodes:"
# Confirm traefik-public network exists
docker network ls | grep traefik-public
# Confirm Traefik is running
docker service ls | grep traefik
# Test registry connectivity (if using local registry)
curl -s https://registry.hobbs.farm/v2/_catalog || echo "Registry unreachable — will use build-and-load method"
# Confirm you are on a Swarm manager node
docker node ls
```
If registry.hobbs.farm is unreachable, use the ALTERNATIVE DEPLOYMENT method
at the bottom of this document.
---
## STEP 2 — BUILD THE IMAGE
```bash
cd rongorongo/
# Build image
docker build -t registry.hobbs.farm/rongorongo:latest .
# Verify image was created
docker images | grep rongorongo
```
Expected output: image tagged `registry.hobbs.farm/rongorongo:latest`, size
should be ~25–40MB (nginx:alpine base + small HTML file).
---
## STEP 3 — PUSH TO REGISTRY
```bash
# If registry requires auth:
docker login registry.hobbs.farm
# Push
docker push registry.hobbs.farm/rongorongo:latest
```
If the push fails due to registry issues, fall through to ALTERNATIVE DEPLOYMENT.
---
## STEP 4 — DEPLOY STACK
```bash
# Deploy or update the stack
docker stack deploy -c stack.yml rongorongo --with-registry-auth
# Watch rollout
docker service ps rongorongo_rongorongo --no-trunc
```
Wait for the service to show `Running` status. This typically takes 15–30 seconds.
```bash
# Confirm healthy
docker service ls | grep rongorongo
curl -s http://localhost/health # or via container exec
```
---
## STEP 5 — DNS VERIFICATION
The stack.yml assumes `rongorongo.hobbs.farm` resolves to the Traefik entrypoint.
```bash
# Check AdGuard Home has the record (or wildcard covers it)
nslookup rongorongo.hobbs.farm
# Test internal access
curl -sk https://rongorongo.hobbs.farm/health
```
If using AdGuard Home for internal DNS and the wildcard *.hobbs.farm is already
configured for hairpin NAT, no additional DNS entry is needed.
---
## STEP 6 — SMOKE TEST
Open https://rongorongo.hobbs.farm in a browser and verify:
1. App loads with "Rongorongo · Decipherment Engine" title
2. Sidebar navigation works (Image Corpus, Glyph Catalog, etc.)
3. Enter an Anthropic API key in the sidebar footer input
4. Upload a test image to the corpus
5. Navigate to Pattern Analyzer and run an analysis
6. Confirm AI response appears in the result box
---
## UPDATING THE APP
When index.html is modified:
```bash
cd rongorongo/
# Rebuild
docker build -t registry.hobbs.farm/rongorongo:latest .
# Push
docker push registry.hobbs.farm/rongorongo:latest
# Force service update (pull new image)
docker service update --image registry.hobbs.farm/rongorongo:latest --force rongorongo_rongorongo
# Watch update
docker service ps rongorongo_rongorongo
```
---
## ROLLBACK
If something breaks after an update:
```bash
docker service rollback rongorongo_rongorongo
docker service ps rongorongo_rongorongo
```
---
## TEARDOWN
To remove the service without destroying data (there is no persistent volume):
```bash
docker stack rm rongorongo
```
---
## ALTERNATIVE DEPLOYMENT (no registry)
Use this if registry.hobbs.farm is unavailable. This method builds the image
directly on the Swarm manager and uses `docker service create` instead of
`stack deploy`.
```bash
# On the Swarm manager node:
cd rongorongo/
# Build locally
docker build -t rongorongo:latest .
# Create service manually (adjust Traefik labels as needed)
docker service create \
--name rongorongo_rongorongo \
--network traefik-public \
--label "traefik.enable=true" \
--label "traefik.docker.network=traefik-public" \
--label "traefik.http.routers.rongorongo.rule=Host(\`rongorongo.hobbs.farm\`)" \
--label "traefik.http.routers.rongorongo.entrypoints=websecure" \
--label "traefik.http.routers.rongorongo.tls=true" \
--label "traefik.http.routers.rongorongo.tls.certresolver=cloudflare" \
--label "traefik.http.services.rongorongo.loadbalancer.server.port=80" \
--replicas 1 \
--restart-condition on-failure \
rongorongo:latest
# Monitor
docker service ps rongorongo_rongorongo
```
NOTE: With this method, to update you must rebuild locally and run:
```bash
docker service update --image rongorongo:latest --force rongorongo_rongorongo
```
This only works if your current node is the one running the replica. If you
have a multi-node swarm, you should set up the registry to avoid placement issues.
---
## PORTAINER DEPLOYMENT (optional GUI method)
If you prefer Portainer CE:
1. Navigate to https://portainer.hobbs.farm (or your Portainer URL)
2. Select the Swarm endpoint
3. Go to Stacks → Add Stack
4. Name: `rongorongo`
5. Paste the contents of stack.yml into the web editor
6. Set `registry.hobbs.farm/rongorongo:latest` as the image
(you still need to build and push first)
7. Click Deploy the stack
---
## ENVIRONMENT NOTES
The app does NOT use environment variables or Docker secrets. The Anthropic API
key is entered by the user at runtime and stored in their browser's localStorage.
This is intentional — the app is designed to be used by a single user (Jeremy)
and no server-side key storage is needed or appropriate.
If you want to pre-populate the API key for convenience (only if the deployment
is strictly private/internal), you can inject it at build time by modifying
frontend/index.html before building:
Find this line:
```javascript
function getApiKey() {
return document.getElementById('apiKeyInput').value.trim()
```
And prepend a hardcoded fallback (NOT recommended for any externally accessible URL).
---
## TRAEFIK ENTRYPOINT NAMES
The stack.yml assumes:
- HTTP entrypoint name: `web`
- HTTPS entrypoint name: `websecure`
- Cert resolver name: `cloudflare`
These match the standard hobbs.farm Traefik configuration. If your Traefik uses
different names, edit the labels in stack.yml before deploying.
---
## TROUBLESHOOTING
**Service won't start:**
```bash
docker service ps rongorongo_rongorongo --no-trunc
docker inspect $(docker ps -q --filter name=rongorongo) 2>/dev/null | grep -A5 '"State"'
```
**Traefik not routing:**
```bash
# Check Traefik logs for rongorongo
docker service logs traefik_traefik 2>&1 | grep -i rongorongo | tail -20
# Confirm labels are applied
docker service inspect rongorongo_rongorongo | grep -A2 Labels
```
**TLS cert not issuing:**
```bash
# Wildcard cert should already cover rongorongo.hobbs.farm
# If not, check Cloudflare DNS API token has Zone:Read and DNS:Edit
docker service logs traefik_traefik 2>&1 | grep -i "acme\|certificate\|cloudflare" | tail -20
```
**API calls blocked (CORS/CSP):**
The nginx.conf includes a Content-Security-Policy header that allows
`connect-src https://api.anthropic.com`. If you see CORS errors in the browser
console, verify the CSP header is present:
```bash
curl -I https://rongorongo.hobbs.farm/ | grep -i content-security
```
**App loads but AI calls fail:**
- Check API key is entered correctly in the sidebar
- Anthropic API key must start with `sk-ant-`
- Confirm outbound HTTPS to api.anthropic.com is not blocked by firewall rules
- The app sends `anthropic-dangerous-direct-browser-access: true` header which
is required for direct browser API access
---
## END OF DEPLOYMENT GUIDE
@@ -1,18 +0,0 @@
FROM nginx:1.25-alpine
# Remove default nginx static content
RUN rm -rf /usr/share/nginx/html/*
# Copy app
COPY frontend/index.html /usr/share/nginx/html/index.html
# Copy nginx config
COPY nginx.conf /etc/nginx/conf.d/default.conf
# Healthcheck
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD wget -qO- http://localhost/health || exit 1
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
@@ -1,322 +0,0 @@
# RONGORONGO_DEPLOY.md
# Rongorongo Decipherment Engine — Claude Code Deployment Guide
# Target: hobbs.farm Docker Swarm infrastructure
# Subdomain: rongorongo.hobbs.farm
---
## CONTEXT
This document instructs Claude Code to build, push, and deploy the Rongorongo
Decipherment Engine as a containerized static web app to the hobbs.farm Swarm.
The app is a single-file HTML/JS application served by nginx. It calls the
Anthropic API directly from the browser. No backend, no database. All session
state is stored in browser localStorage.
Infrastructure assumptions (from existing hobbs.farm stack):
- Docker Swarm: nodes hf-pdocker-01, hf-pdocker-02, bart
- Traefik reverse proxy with wildcard *.hobbs.farm TLS via Cloudflare DNS
- External overlay network: traefik-public
- Registry: registry.hobbs.farm (if local) OR use Docker Hub as fallback
- Portainer CE available for stack management
- DNS: AdGuard Home with hairpin NAT for *.hobbs.farm
---
## REPOSITORY STRUCTURE
After cloning or creating the project, verify this structure exists:
```
rongorongo/
├── Dockerfile
├── nginx.conf
├── stack.yml
├── frontend/
│ └── index.html
└── RONGORONGO_DEPLOY.md (this file)
```
---
## STEP 1 — VERIFY PREREQUISITES
Run these checks before proceeding. Stop and report any failures.
```bash
# Confirm Docker and Swarm context
docker info | grep -E "Swarm:|Manager:|Nodes:"
# Confirm traefik-public network exists
docker network ls | grep traefik-public
# Confirm Traefik is running
docker service ls | grep traefik
# Test registry connectivity (if using local registry)
curl -s https://registry.hobbs.farm/v2/_catalog || echo "Registry unreachable — will use build-and-load method"
# Confirm you are on a Swarm manager node
docker node ls
```
If registry.hobbs.farm is unreachable, use the ALTERNATIVE DEPLOYMENT method
at the bottom of this document.
---
## STEP 2 — BUILD THE IMAGE
```bash
cd rongorongo/
# Build image
docker build -t registry.hobbs.farm/rongorongo:latest .
# Verify image was created
docker images | grep rongorongo
```
Expected output: image tagged `registry.hobbs.farm/rongorongo:latest`, size
should be ~25–40MB (nginx:alpine base + small HTML file).
---
## STEP 3 — PUSH TO REGISTRY
```bash
# If registry requires auth:
docker login registry.hobbs.farm
# Push
docker push registry.hobbs.farm/rongorongo:latest
```
If the push fails due to registry issues, fall through to ALTERNATIVE DEPLOYMENT.
---
## STEP 4 — DEPLOY STACK
```bash
# Deploy or update the stack
docker stack deploy -c stack.yml rongorongo --with-registry-auth
# Watch rollout
docker service ps rongorongo_rongorongo --no-trunc
```
Wait for the service to show `Running` status. This typically takes 15–30 seconds.
```bash
# Confirm healthy
docker service ls | grep rongorongo
curl -s http://localhost/health # or via container exec
```
---
## STEP 5 — DNS VERIFICATION
The stack.yml assumes `rongorongo.hobbs.farm` resolves to the Traefik entrypoint.
```bash
# Check AdGuard Home has the record (or wildcard covers it)
nslookup rongorongo.hobbs.farm
# Test internal access
curl -sk https://rongorongo.hobbs.farm/health
```
If using AdGuard Home for internal DNS and the wildcard *.hobbs.farm is already
configured for hairpin NAT, no additional DNS entry is needed.
---
## STEP 6 — SMOKE TEST
Open https://rongorongo.hobbs.farm in a browser and verify:
1. App loads with "Rongorongo · Decipherment Engine" title
2. Sidebar navigation works (Image Corpus, Glyph Catalog, etc.)
3. Enter an Anthropic API key in the sidebar footer input
4. Upload a test image to the corpus
5. Navigate to Pattern Analyzer and run an analysis
6. Confirm AI response appears in the result box
---
## UPDATING THE APP
When index.html is modified:
```bash
cd rongorongo/
# Rebuild
docker build -t registry.hobbs.farm/rongorongo:latest .
# Push
docker push registry.hobbs.farm/rongorongo:latest
# Force service update (pull new image)
docker service update --image registry.hobbs.farm/rongorongo:latest --force rongorongo_rongorongo
# Watch update
docker service ps rongorongo_rongorongo
```
---
## ROLLBACK
If something breaks after an update:
```bash
docker service rollback rongorongo_rongorongo
docker service ps rongorongo_rongorongo
```
---
## TEARDOWN
To remove the service without destroying data (there is no persistent volume):
```bash
docker stack rm rongorongo
```
---
## ALTERNATIVE DEPLOYMENT (no registry)
Use this if registry.hobbs.farm is unavailable. This method builds the image
directly on the Swarm manager and uses `docker service create` instead of
`stack deploy`.
```bash
# On the Swarm manager node:
cd rongorongo/
# Build locally
docker build -t rongorongo:latest .
# Create service manually (adjust Traefik labels as needed)
docker service create \
--name rongorongo_rongorongo \
--network traefik-public \
--label "traefik.enable=true" \
--label "traefik.docker.network=traefik-public" \
--label "traefik.http.routers.rongorongo.rule=Host(\`rongorongo.hobbs.farm\`)" \
--label "traefik.http.routers.rongorongo.entrypoints=websecure" \
--label "traefik.http.routers.rongorongo.tls=true" \
--label "traefik.http.routers.rongorongo.tls.certresolver=cloudflare" \
--label "traefik.http.services.rongorongo.loadbalancer.server.port=80" \
--replicas 1 \
--restart-condition on-failure \
rongorongo:latest
# Monitor
docker service ps rongorongo_rongorongo
```
NOTE: With this method, to update you must rebuild locally and run:
```bash
docker service update --image rongorongo:latest --force rongorongo_rongorongo
```
This only works if your current node is the one running the replica. If you
have a multi-node swarm, you should set up the registry to avoid placement issues.
---
## PORTAINER DEPLOYMENT (optional GUI method)
If you prefer Portainer CE:
1. Navigate to https://portainer.hobbs.farm (or your Portainer URL)
2. Select the Swarm endpoint
3. Go to Stacks → Add Stack
4. Name: `rongorongo`
5. Paste the contents of stack.yml into the web editor
6. Set `registry.hobbs.farm/rongorongo:latest` as the image
(you still need to build and push first)
7. Click Deploy the stack
---
## ENVIRONMENT NOTES
The app does NOT use environment variables or Docker secrets. The Anthropic API
key is entered by the user at runtime and stored in their browser's localStorage.
This is intentional — the app is designed to be used by a single user (Jeremy)
and no server-side key storage is needed or appropriate.
If you want to pre-populate the API key for convenience (only if the deployment
is strictly private/internal), you can inject it at build time by modifying
frontend/index.html before building:
Find this line:
```javascript
function getApiKey() {
return document.getElementById('apiKeyInput').value.trim()
```
And prepend a hardcoded fallback (NOT recommended for any externally accessible URL).
---
## TRAEFIK ENTRYPOINT NAMES
The stack.yml assumes:
- HTTP entrypoint name: `web`
- HTTPS entrypoint name: `websecure`
- Cert resolver name: `cloudflare`
These match the standard hobbs.farm Traefik configuration. If your Traefik uses
different names, edit the labels in stack.yml before deploying.
---
## TROUBLESHOOTING
**Service won't start:**
```bash
docker service ps rongorongo_rongorongo --no-trunc
docker inspect $(docker ps -q --filter name=rongorongo) 2>/dev/null | grep -A5 '"State"'
```
**Traefik not routing:**
```bash
# Check Traefik logs for rongorongo
docker service logs traefik_traefik 2>&1 | grep -i rongorongo | tail -20
# Confirm labels are applied
docker service inspect rongorongo_rongorongo | grep -A2 Labels
```
**TLS cert not issuing:**
```bash
# Wildcard cert should already cover rongorongo.hobbs.farm
# If not, check Cloudflare DNS API token has Zone:Read and DNS:Edit
docker service logs traefik_traefik 2>&1 | grep -i "acme\|certificate\|cloudflare" | tail -20
```
**API calls blocked (CORS/CSP):**
The nginx.conf includes a Content-Security-Policy header that allows
`connect-src https://api.anthropic.com`. If you see CORS errors in the browser
console, verify the CSP header is present:
```bash
curl -I https://rongorongo.hobbs.farm/ | grep -i content-security
```
**App loads but AI calls fail:**
- Check API key is entered correctly in the sidebar
- Anthropic API key must start with `sk-ant-`
- Confirm outbound HTTPS to api.anthropic.com is not blocked by firewall rules
- The app sends `anthropic-dangerous-direct-browser-access: true` header which
is required for direct browser API access
---
## END OF DEPLOYMENT GUIDE
File diff suppressed because it is too large Load Diff
@@ -1,41 +0,0 @@
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
index index.html;
# Security headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
# CSP — allows Anthropic API calls and Google Fonts
add_header Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; font-src 'self' https://fonts.gstatic.com; img-src 'self' data: blob:; connect-src 'self' https://api.anthropic.com; worker-src blob:;" always;
# Gzip
gzip on;
gzip_types text/html text/css application/javascript;
gzip_min_length 1024;
# Health endpoint
location /health {
access_log off;
return 200 "ok\n";
add_header Content-Type text/plain;
}
# SPA fallback
location / {
try_files $uri $uri/ /index.html;
expires -1;
add_header Cache-Control "no-store, no-cache, must-revalidate";
}
# Cache static assets aggressively if you add any
location ~* \.(js|css|png|jpg|jpeg|gif|ico|woff2)$ {
expires 1y;
add_header Cache-Control "public, immutable";
}
}
@@ -1,48 +0,0 @@
version: "3.8"
services:
rongorongo:
image: registry.hobbs.farm/rongorongo:latest
networks:
- traefik-public
deploy:
replicas: 1
restart_policy:
condition: on-failure
delay: 10s
max_attempts: 3
update_config:
parallelism: 1
delay: 10s
failure_action: rollback
labels:
- "traefik.enable=true"
- "traefik.docker.network=traefik-public"
# HTTP → HTTPS redirect
- "traefik.http.routers.rongorongo-http.rule=Host(`rongorongo.hobbs.farm`)"
- "traefik.http.routers.rongorongo-http.entrypoints=web"
- "traefik.http.routers.rongorongo-http.middlewares=redirect-to-https"
# HTTPS router
- "traefik.http.routers.rongorongo.rule=Host(`rongorongo.hobbs.farm`)"
- "traefik.http.routers.rongorongo.entrypoints=websecure"
- "traefik.http.routers.rongorongo.tls=true"
- "traefik.http.routers.rongorongo.tls.certresolver=cloudflare"
# Service
- "traefik.http.services.rongorongo.loadbalancer.server.port=80"
# Middlewares
- "traefik.http.routers.rongorongo.middlewares=rongorongo-headers"
- "traefik.http.middlewares.rongorongo-headers.headers.customResponseHeaders.X-Robots-Tag=noindex,nofollow"
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost/health"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
networks:
traefik-public:
external: true
Binary file not shown.
Binary file not shown.
File diff suppressed because it is too large Load Diff
-1973
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+4 -22
View File
@@ -1,30 +1,16 @@
services:
jellyfin:
# PINNED to the 12.0.0 unstable build that migrated the DB to the v12
# schema (DropExtraIdsColumn) — :latest is still 10.11.x and CANNOT read
# this DB ("no such column: b.ExtraIds"). Do not unpin until official
# stable >= 12 ships. (2026-07-14: a :latest deploy downgraded it and
# broke Resume/home sections; repinned.)
image: jellyfin/jellyfin:2026060106
container_name: jellyfin
restart: unless-stopped
environment:
- PUID=1000
- PGID=1000
- LIBVA_DRIVER_NAME=i965 # Haswell HD 4600 uses legacy i965 (iHD is Broadwell+)
volumes:
# LOCAL disk, not NFS: jellyfin.db (SQLite WAL) on NFS caused 60-100s
# "database is locked" stalls -> server lockups during playback
# (2026-07-19). Old copy left at /mnt/docker-data/configs/jellyfin.
- /var/lib/jellyfin:/config
- /mnt/docker-data/configs/jellyfin:/config
- /mnt/nas-media:/media
ports:
- "8096:8096"
devices:
- /dev/dri:/dev/dri # Intel iGPU for VAAPI hardware transcoding
group_add:
- "993" # render
- "44" # video
mem_limit: 2G
healthcheck:
test: ["CMD-SHELL", "curl -f http://localhost:8096/health || exit 1"]
@@ -47,9 +33,7 @@ services:
- PGID=1000
- TZ=America/Chicago
volumes:
# LOCAL disk, not NFS: SQLite-on-NFS locking crashed Sonarr's DB migration
# after the 2026-07-12 power event ("waiting for user intervention")
- /var/lib/sonarr:/config
- /mnt/docker-data/configs/sonarr:/config
- /mnt/nas-media:/media
ports:
- "8989:8989"
@@ -127,12 +111,10 @@ services:
- PGID=1000
- TZ=America/Chicago
volumes:
# LOCAL disk, not NFS: stale NFS lock on the H2 database after the
# 2026-07-12 power event kept hydra unhealthy in a DB-retry loop
- /var/lib/nzbhydra:/config
- /mnt/docker-data/configs/nzbhydra:/config
ports:
- "5076:5076"
mem_limit: 1G
mem_limit: 512M
healthcheck:
test: ["CMD-SHELL", "curl -f http://localhost:5076/ || exit 1"]
interval: 30s
@@ -1,24 +0,0 @@
services:
node-exporter:
image: prom/node-exporter:v1.7.0
container_name: node-exporter
hostname: bart
network_mode: host
pid: host
restart: unless-stopped
volumes:
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /:/rootfs:ro
command:
- '--path.procfs=/host/proc'
- '--path.rootfs=/rootfs'
- '--path.sysfs=/host/sys'
- '--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)'
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
labels:
- "com.centurylinklabs.watchtower.enable=false"
+231
View File
@@ -0,0 +1,231 @@
services:
api_server:
image: onyxdotapp/onyx-backend:latest
command: >
/bin/sh -c "alembic upgrade head &&
echo Starting Onyx API Server &&
uvicorn onyx.main:app --host 0.0.0.0 --port 8080"
depends_on:
relational_db:
condition: service_healthy
index:
condition: service_healthy
cache:
condition: service_healthy
restart: "no"
mem_limit: 2G
memswap_limit: 2G
env_file:
- .env
environment:
- AUTH_TYPE=basic
- DISABLE_TELEMETRY=true
- POSTGRES_HOST=relational_db
- VESPA_HOST=index
- REDIS_HOST=cache
- MODEL_SERVER_HOST=inference_model_server
- INDEXING_MODEL_SERVER_HOST=indexing_model_server
- OPENSEARCH_HOST=opensearch
- ENABLE_OPENSEARCH_INDEXING_FOR_ONYX=false
- FILE_STORE_BACKEND=postgres
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
- onyx_file_connector:/home/file_connector_storage
networks:
- onyx
labels:
com.centurylinklabs.watchtower.enable: "false"
background:
image: onyxdotapp/onyx-backend:latest
command: >
/bin/sh -c "/usr/bin/supervisord -c /etc/supervisor/conf.d/supervisord.conf"
depends_on:
relational_db:
condition: service_healthy
index:
condition: service_healthy
cache:
condition: service_healthy
restart: "no"
mem_limit: 3G
memswap_limit: 3G
env_file:
- .env
environment:
- AUTH_TYPE=basic
- DISABLE_TELEMETRY=true
- POSTGRES_HOST=relational_db
- VESPA_HOST=index
- REDIS_HOST=cache
- MODEL_SERVER_HOST=inference_model_server
- INDEXING_MODEL_SERVER_HOST=indexing_model_server
- OPENSEARCH_HOST=opensearch
- ENABLE_OPENSEARCH_INDEXING_FOR_ONYX=false
- FILE_STORE_BACKEND=postgres
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
- onyx_file_connector:/home/file_connector_storage
networks:
- onyx
labels:
com.centurylinklabs.watchtower.enable: "false"
web_server:
image: onyxdotapp/onyx-web-server:latest
depends_on:
- api_server
restart: "no"
mem_limit: 512M
memswap_limit: 512M
environment:
- INTERNAL_URL=http://api_server:8080
- WEB_DOMAIN=https://onyx.hobbs.farm
- NEXT_PUBLIC_DISABLE_STREAMING=false
- AUTH_TYPE=basic
networks:
- onyx
labels:
com.centurylinklabs.watchtower.enable: "false"
inference_model_server:
image: onyxdotapp/onyx-model-server:latest
command: >
/bin/sh -c "exec uvicorn model_server.main:app --host 0.0.0.0 --port 9000"
restart: "no"
mem_limit: 2G
memswap_limit: 2G
environment:
- MIN_THREADS_ML_MODELS=1
- LOG_LEVEL=info
volumes:
- onyx_model_cache_huggingface:/root/.cache/huggingface/
networks:
- onyx
labels:
com.centurylinklabs.watchtower.enable: "false"
indexing_model_server:
image: onyxdotapp/onyx-model-server:latest
command: >
/bin/sh -c "exec uvicorn model_server.main:app --host 0.0.0.0 --port 9000"
restart: "no"
mem_limit: 2G
memswap_limit: 2G
environment:
- MIN_THREADS_ML_MODELS=1
- INDEXING_ONLY=true
- LOG_LEVEL=info
volumes:
- onyx_model_cache_huggingface:/root/.cache/huggingface/
networks:
- onyx
labels:
com.centurylinklabs.watchtower.enable: "false"
relational_db:
image: postgres:15.2-alpine
restart: "no"
mem_limit: 512M
memswap_limit: 512M
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=onyx_postgres_pw
volumes:
- onyx_db:/var/lib/postgresql/data
networks:
- onyx
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
labels:
com.centurylinklabs.watchtower.enable: "false"
index:
image: vespaengine/vespa:8.609.39
restart: "no"
mem_limit: 4G
memswap_limit: 4G
volumes:
- onyx_vespa:/opt/vespa/var
networks:
- onyx
healthcheck:
test: ["CMD-SHELL", "curl -sf http://localhost:19071/state/v1/health || exit 1"]
interval: 30s
timeout: 10s
retries: 10
start_period: 120s
labels:
com.centurylinklabs.watchtower.enable: "false"
opensearch:
image: opensearchproject/opensearch:2.11.1
restart: "no"
mem_limit: 2G
memswap_limit: 2G
environment:
- discovery.type=single-node
- plugins.security.disabled=true
- OPENSEARCH_JAVA_OPTS=-Xms1g -Xmx1g
- DISABLE_INSTALL_DEMO_CONFIG=true
volumes:
- onyx_opensearch:/usr/share/opensearch/data
networks:
- onyx
healthcheck:
test: ["CMD-SHELL", "curl -sf http://localhost:9200/_cluster/health || exit 1"]
interval: 30s
timeout: 10s
retries: 10
start_period: 120s
labels:
com.centurylinklabs.watchtower.enable: "false"
cache:
image: redis:7.4-alpine
restart: "no"
mem_limit: 256M
memswap_limit: 256M
command: redis-server --maxmemory 128mb --maxmemory-policy allkeys-lru
networks:
- onyx
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 5s
retries: 5
labels:
com.centurylinklabs.watchtower.enable: "false"
nginx:
image: nginx:1.25.5-alpine
restart: "no"
mem_limit: 128M
memswap_limit: 128M
depends_on:
- api_server
- web_server
ports:
- "3005:80"
volumes:
- ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
networks:
- onyx
labels:
com.centurylinklabs.watchtower.enable: "false"
volumes:
onyx_file_connector:
onyx_model_cache_huggingface:
onyx_db:
onyx_vespa:
onyx_opensearch:
networks:
onyx:
driver: bridge
+36
View File
@@ -0,0 +1,36 @@
server {
listen 80;
server_name _;
# Use Docker's internal DNS with a short TTL so container IP changes
# (e.g. after docker compose up -d for a single service) are picked up
# without requiring an nginx reload.
resolver 127.0.0.11 valid=10s;
client_max_body_size 100M;
location /api/ {
set $api_upstream http://api_server:8080;
rewrite ^/api/(.*)$ /$1 break;
proxy_pass $api_upstream;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 300s;
proxy_connect_timeout 75s;
proxy_buffering off;
proxy_cache off;
}
location / {
set $web_upstream http://web_server:3000;
proxy_pass $web_upstream;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 300s;
proxy_connect_timeout 75s;
}
}
-4
View File
@@ -3,10 +3,6 @@ services:
image: immauss/openvas:latest
container_name: openvas
restart: unless-stopped
labels:
# Watchtower pulled a new major that forced a pg_upgrade crash-loop
# (212 restarts, 2026-07-13); update OpenVAS deliberately instead
- "com.centurylinklabs.watchtower.enable=false"
environment:
- PASSWORD=${OPENVAS_PASSWORD}
- USERNAME=${OPENVAS_USERNAME}
-22
View File
@@ -1,22 +0,0 @@
services:
telegraf:
image: telegraf:latest
container_name: telegraf
hostname: bart
restart: unless-stopped
environment:
- HOST_PROC=/host/proc
- HOST_SYS=/host/sys
- HOST_ETC=/host/etc
volumes:
- /mnt/docker-data/configs/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- /sys:/host/sys:ro
- /proc:/host/proc:ro
- /etc:/host/etc:ro
mem_limit: 512M
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
@@ -1,17 +0,0 @@
# Overlay on the friday-voice app image that adds the docker CLI + a wrapper so
# the container can reach the hermes agent via `docker exec hermes hermes ...`.
# Runs as root to access the mounted /var/run/docker.sock.
FROM friday-voice:latest
USER root
# Static docker *client* only (~small), for `docker exec` into the hermes container.
RUN curl -fsSL https://download.docker.com/linux/static/stable/x86_64/docker-27.5.1.tgz \
| tar -xz --strip-components=1 -C /usr/local/bin docker/docker \
&& chmod +x /usr/local/bin/docker \
&& docker --version
COPY hermes-wrap /usr/local/bin/hermes-wrap
RUN chmod +x /usr/local/bin/hermes-wrap
# Keep the base image's CMD (uvicorn friday_voice.app:app :8320).
@@ -1,49 +0,0 @@
# FRIDAY Voice on .150 (Docker Desktop) — replaces VoiceCoach.
# STT/TTS via local GPU speaches (:8000). Reasoning via the hermes container
# (invoked through the docker-exec wrapper). Runs as root for socket access.
services:
friday-voice:
build:
context: .
dockerfile: Dockerfile.hermes
image: friday-voice-hermes:latest
container_name: friday-voice
restart: unless-stopped
user: "0:0"
ports:
- "8320:8320"
volumes:
- friday-voice-data:/data
- /var/run/docker.sock:/var/run/docker.sock
environment:
- DATA_DIR=/data
- SPEACHES_URL=http://host.docker.internal:8000
- HERMES_COMMAND=/usr/local/bin/hermes-wrap
- HERMES_SOURCE=friday-voice
- WHISPER_MODEL=deepdml/faster-whisper-large-v3-turbo-ct2
- TTS_MODEL=speaches-ai/Kokoro-82M-v1.0-ONNX
- TTS_VOICE=af_nova
- SPEAKER_CONFIRMED_THRESHOLD=0.85
- SPEAKER_PROBABLE_THRESHOLD=0.70
extra_hosts:
- "host.docker.internal:host-gateway"
labels:
- "com.centurylinklabs.watchtower.enable=false"
healthcheck:
test: ["CMD", "curl", "-fsS", "http://127.0.0.1:8320/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 20s
logging:
driver: "json-file"
options:
max-size: "20m"
max-file: "3"
volumes:
friday-voice-data:
networks:
default:
name: friday-voice
@@ -1,5 +0,0 @@
#!/bin/sh
# friday-voice invokes: hermes-wrap chat -q "<text>" --source friday-voice
# We forward into the hermes container and add -Q (quiet) so only the final
# response text is returned (no banner/spinner/box art) for clean TTS.
exec docker exec hermes hermes "$@" -Q
@@ -1,30 +0,0 @@
# Hermes Agent on .150 (Docker Desktop) — FRIDAY's reasoning agent.
# Data/settings live in the `hermes-data` named volume, seeded from the user's
# WSL ~/.hermes (settings preserved; SQLite state.db kept on the VM ext4, not a
# bridged bind mount). friday-voice calls this via `docker exec hermes hermes ...`.
services:
gateway:
image: nousresearch/hermes-agent:latest
container_name: hermes
restart: unless-stopped
volumes:
- hermes-data:/opt/data
environment:
- HERMES_UID=10000
- HERMES_GID=10000
command: ["gateway", "run"]
labels:
- "com.centurylinklabs.watchtower.enable=false"
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
# dashboard: disabled — crash-loops under s6 on Docker Desktop and not needed
# by friday-voice (gateway + `docker exec hermes` is all that's required).
# Re-enable + debug later if the localhost:9119 web UI is wanted.
volumes:
hermes-data:
external: true
@@ -1,36 +0,0 @@
services:
speaches:
image: ghcr.io/speaches-ai/speaches:latest-cuda
container_name: voicecoach-speaches-gpu
restart: unless-stopped
environment:
- WHISPER__INFERENCE_DEVICE=cuda
- WHISPER__COMPUTE_TYPE=float16
- WHISPER__TTL=-1
volumes:
- speaches-models:/home/ubuntu/.cache
ports:
- "8000:8000"
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
labels:
- "com.centurylinklabs.watchtower.enable=false"
healthcheck:
test: ["CMD", "curl", "-sf", "http://localhost:8000/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 180s
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
volumes:
speaches-models:
@@ -0,0 +1,26 @@
services:
adguardhome:
image: adguard/adguardhome:latest
container_name: adguardhome
restart: unless-stopped
volumes:
- /mnt/docker-data/configs/adguardhome/work:/opt/adguardhome/work
- /mnt/docker-data/configs/adguardhome/conf:/opt/adguardhome/conf
ports:
- "53:53/tcp"
- "53:53/udp"
- "3000:3000/tcp"
- "853:853/tcp"
mem_limit: 256M
memswap_limit: 256M
healthcheck:
test: ["CMD-SHELL", "wget -q --spider http://localhost:3000/ || exit 1"]
interval: 30s
timeout: 10s
retries: 3
start_period: 30s
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
@@ -36,7 +36,6 @@ services:
- HOME=/home/glyptic
- USER=glyptic
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
- GEMINI_API_KEY=${GEMINI_API_KEY}
- WIKIJS_API_KEY=${WIKIJS_API_KEY}
- OPENCLAW_STATE_DIR=/home/glyptic/.spirit
- OPENCLAW_CONFIG_PATH=/home/glyptic/.spirit/openclaw.json
@@ -44,8 +43,6 @@ services:
- /mnt/infrastructure/spirit:/home/glyptic/.spirit
- /var/run/docker.sock:/var/run/docker.sock
- /mnt/infrastructure/spirit/ssh:/root/.ssh:ro
- /mnt/docker-data/configs/prometheus:/prometheus-config
- /mnt/docker-data/configs/alertmanager:/alertmanager-config
command: ["sh", "-c", "mkdir -p /home/glyptic && node --max-old-space-size=1500 /usr/local/lib/node_modules/openclaw/dist/index.js gateway --allow-unconfigured"]
mem_limit: 3G
memswap_limit: 3G
@@ -55,3 +52,74 @@ services:
max-size: "50m"
max-file: "3"
bob:
image: margaret-openclaw:latest
container_name: bob
restart: unless-stopped
network_mode: host
user: "0:0"
environment:
- NODE_ENV=production
- HOME=/home/glyptic
- USER=glyptic
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
- OPENCLAW_STATE_DIR=/home/glyptic/.bob
- OPENCLAW_CONFIG_PATH=/home/glyptic/.bob/openclaw.json
volumes:
- /mnt/infrastructure/bob:/home/glyptic/.bob
command: ["sh", "-c", "mkdir -p /home/glyptic && node --max-old-space-size=1500 /usr/local/lib/node_modules/openclaw/dist/index.js gateway --allow-unconfigured"]
mem_limit: 2G
memswap_limit: 2G
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
clippy:
image: margaret-openclaw:latest
container_name: clippy
restart: unless-stopped
network_mode: host
user: "0:0"
environment:
- NODE_ENV=production
- HOME=/home/glyptic
- USER=glyptic
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
- OPENCLAW_STATE_DIR=/home/glyptic/.clippy
- OPENCLAW_CONFIG_PATH=/home/glyptic/.clippy/openclaw.json
volumes:
- /mnt/infrastructure/clippy:/home/glyptic/.clippy
command: ["sh", "-c", "mkdir -p /home/glyptic && node --max-old-space-size=1500 /usr/local/lib/node_modules/openclaw/dist/index.js gateway --allow-unconfigured"]
mem_limit: 2G
memswap_limit: 2G
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
travis:
image: travis-nemoclaw:latest
container_name: travis
restart: unless-stopped
network_mode: host
user: "0:0"
environment:
- NODE_ENV=production
- HOME=/home/glyptic
- USER=glyptic
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
- OPENCLAW_STATE_DIR=/home/glyptic/.travis
- OPENCLAW_CONFIG_PATH=/home/glyptic/.travis/openclaw.json
volumes:
- /mnt/infrastructure/travis:/home/glyptic/.travis
command: ["sh", "-c", "mkdir -p /home/glyptic && openclaw plugins install /opt/nemoclaw 2>/dev/null; node --max-old-space-size=1500 /usr/local/lib/node_modules/openclaw/dist/index.js gateway --allow-unconfigured"]
mem_limit: 2G
memswap_limit: 2G
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
@@ -1,8 +0,0 @@
SPEACHES_URL=http://192.168.86.150:8000
HERMES_COMMAND=hermes
HERMES_SOURCE=friday-voice
WHISPER_MODEL=deepdml/faster-whisper-large-v3-turbo-ct2
TTS_MODEL=speaches-ai/Kokoro-82M-v1.0-ONNX
TTS_VOICE=af_nova
SPEAKER_CONFIRMED_THRESHOLD=0.85
SPEAKER_PROBABLE_THRESHOLD=0.70
@@ -1,39 +0,0 @@
services:
friday-voice:
build:
context: /opt/friday-voice
dockerfile: Dockerfile
image: friday-voice:latest
container_name: friday-voice
restart: unless-stopped
ports:
- "8320:8320"
volumes:
- /mnt/docker-data/configs/friday-voice:/data
environment:
- DATA_DIR=/data
- SPEACHES_URL=${SPEACHES_URL:-http://192.168.86.150:8000}
- HERMES_COMMAND=${HERMES_COMMAND:-hermes}
- HERMES_SOURCE=${HERMES_SOURCE:-friday-voice}
- WHISPER_MODEL=${WHISPER_MODEL:-deepdml/faster-whisper-large-v3-turbo-ct2}
- TTS_MODEL=${TTS_MODEL:-speaches-ai/Kokoro-82M-v1.0-ONNX}
- TTS_VOICE=${TTS_VOICE:-af_nova}
- SPEAKER_CONFIRMED_THRESHOLD=${SPEAKER_CONFIRMED_THRESHOLD:-0.85}
- SPEAKER_PROBABLE_THRESHOLD=${SPEAKER_PROBABLE_THRESHOLD:-0.70}
labels:
- "com.centurylinklabs.watchtower.enable=false"
healthcheck:
test: ["CMD", "curl", "-fsS", "http://127.0.0.1:8320/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 20s
logging:
driver: "json-file"
options:
max-size: "20m"
max-file: "3"
networks:
default:
name: friday-voice
@@ -9,8 +9,6 @@ RUN pip install --no-cache-dir -r requirements.txt
COPY app/ .
RUN mkdir -p /data
EXPOSE 5199
CMD ["python", "-u", "main.py"]
@@ -1,35 +0,0 @@
"""Automatic event cleanup based on retention settings."""
import asyncio
import logging
from database import get_connection
logger = logging.getLogger(__name__)
async def cleanup_old_events():
"""Delete events older than the configured retention period. Runs hourly."""
while True:
try:
conn = get_connection()
row = conn.execute(
"SELECT value FROM settings WHERE key='event_retention_days'"
).fetchone()
days = int(row["value"]) if row else 90
result = conn.execute(
"DELETE FROM events WHERE created_at < datetime('now', ? || ' days')",
(f"-{days}",),
)
conn.commit()
if result.rowcount > 0:
logger.info(
"Cleaned up %d events older than %d days",
result.rowcount,
days,
)
except Exception:
logger.exception("Cleanup failed")
await asyncio.sleep(3600)
@@ -1,56 +0,0 @@
"""Centralized configuration for frigate-notify v2."""
import os
class Config:
"""Application configuration read from environment variables."""
# MQTT
MQTT_HOST: str = os.environ.get("MQTT_HOST", "192.168.86.192")
MQTT_PORT: int = int(os.environ.get("MQTT_PORT", "1883"))
# Frigate
FRIGATE_URL: str = os.environ.get("FRIGATE_URL", "http://192.168.86.192:5000")
# Matrix
MATRIX_HOMESERVER: str = os.environ.get(
"MATRIX_HOMESERVER", "http://192.168.86.192:8008"
)
MATRIX_ACCESS_TOKEN: str = os.environ.get("MATRIX_ACCESS_TOKEN", "")
MATRIX_ROOM_ID: str = os.environ.get("MATRIX_ROOM_ID", "")
# Vision - CLIP service
CLIP_SERVICE_URL: str = os.environ.get(
"CLIP_SERVICE_URL", "http://192.168.86.150:5200"
)
# Vision - Ollama
OLLAMA_URL: str = os.environ.get("OLLAMA_URL", "http://192.168.86.150:11434")
OLLAMA_MODEL: str = os.environ.get("OLLAMA_MODEL", "llava:13b")
# Recognition thresholds
RECOGNITION_THRESHOLD_HIGH: float = float(
os.environ.get("RECOGNITION_THRESHOLD_HIGH", "0.85")
)
RECOGNITION_THRESHOLD_LOW: float = float(
os.environ.get("RECOGNITION_THRESHOLD_LOW", "0.60")
)
# Filtering
ALLOWED_LABELS: set = set(
os.environ.get("ALLOWED_LABELS", "person,car,dog,cat,bird").split(",")
)
MIN_SCORE: float = float(os.environ.get("MIN_SCORE", "0.7"))
RATE_LIMIT_SECONDS: int = int(os.environ.get("RATE_LIMIT_SECONDS", "60"))
DEDUP_WINDOW_SECONDS: int = int(os.environ.get("DEDUP_WINDOW_SECONDS", "30"))
# Server
SERVER_HOST: str = os.environ.get("SERVER_HOST", "0.0.0.0")
SERVER_PORT: int = int(os.environ.get("SERVER_PORT", "5199"))
# Database
DB_PATH: str = os.environ.get("DB_PATH", "/data/frigate-notify.db")
config = Config()
@@ -1,109 +0,0 @@
"""SQLite database schema and connection management for frigate-notify v2."""
import logging
import sqlite3
import threading
logger = logging.getLogger(__name__)
# Thread-local storage for database connections
_local = threading.local()
_db_path: str = "/data/frigate-notify.db"
def set_db_path(path: str) -> None:
"""Set the database path. Closes any existing connection first."""
global _db_path
close_connection()
_db_path = path
def get_connection() -> sqlite3.Connection:
"""Get a thread-local database connection with WAL mode and foreign keys enabled."""
if not hasattr(_local, "connection") or _local.connection is None:
conn = sqlite3.connect(_db_path)
conn.execute("PRAGMA journal_mode=WAL")
conn.execute("PRAGMA foreign_keys=ON")
conn.row_factory = sqlite3.Row
_local.connection = conn
return _local.connection
def close_connection() -> None:
"""Close the thread-local database connection if one exists."""
if hasattr(_local, "connection") and _local.connection is not None:
_local.connection.close()
_local.connection = None
def init_db() -> None:
"""Create all tables and default settings."""
conn = get_connection()
cursor = conn.cursor()
cursor.executescript(
"""
CREATE TABLE IF NOT EXISTS known_objects (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
type TEXT NOT NULL CHECK(type IN ('person', 'vehicle', 'pet')),
action TEXT NOT NULL DEFAULT 'quiet' CHECK(action IN ('quiet', 'suppress', 'alert')),
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS embeddings (
id INTEGER PRIMARY KEY AUTOINCREMENT,
known_object_id INTEGER NOT NULL,
embedding BLOB NOT NULL,
snapshot_url TEXT,
thumbnail BLOB,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (known_object_id) REFERENCES known_objects(id) ON DELETE CASCADE
);
CREATE TABLE IF NOT EXISTS events (
id INTEGER PRIMARY KEY AUTOINCREMENT,
frigate_event_id TEXT UNIQUE NOT NULL,
camera TEXT NOT NULL,
label TEXT NOT NULL,
score REAL NOT NULL,
snapshot BLOB,
embedding BLOB,
match_status TEXT NOT NULL DEFAULT 'unreviewed'
CHECK(match_status IN ('known', 'unknown', 'ambiguous', 'unreviewed')),
matched_object_id INTEGER,
match_confidence REAL,
ai_description TEXT,
notified INTEGER NOT NULL DEFAULT 0,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (matched_object_id) REFERENCES known_objects(id) ON DELETE SET NULL
);
CREATE TABLE IF NOT EXISTS settings (
key TEXT PRIMARY KEY,
value TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_events_created_at ON events(created_at);
CREATE INDEX IF NOT EXISTS idx_events_match_status ON events(match_status);
CREATE INDEX IF NOT EXISTS idx_events_camera ON events(camera);
CREATE INDEX IF NOT EXISTS idx_embeddings_known_object_id ON embeddings(known_object_id);
"""
)
# Insert default settings if they don't exist
defaults = {
"vision_backend": "haiku",
"recognition_enabled": "false",
"event_retention_days": "90",
}
for key, value in defaults.items():
cursor.execute(
"INSERT OR IGNORE INTO settings (key, value) VALUES (?, ?)",
(key, value),
)
conn.commit()
logger.info("Database initialized at %s", _db_path)
+86 -190
View File
@@ -1,5 +1,3 @@
"""frigate-notify v2 — FastAPI application with MQTT listener."""
import asyncio
import json
import logging
@@ -8,28 +6,29 @@ import signal
import sys
import time
from collections import defaultdict
from contextlib import asynccontextmanager
from pathlib import Path
from http.server import HTTPServer, BaseHTTPRequestHandler
from threading import Thread
import aiohttp
import anthropic
import paho.mqtt.client as mqtt
import uvicorn
from fastapi import FastAPI
from fastapi.responses import FileResponse, JSONResponse
from fastapi.staticfiles import StaticFiles
from config import config
from database import close_connection, get_connection, init_db, set_db_path
from vision import analyze_snapshot
from matrix_client import MatrixClient
from routes.events import router as events_router
from routes.known_objects import router as known_objects_router
from routes.settings import router as settings_router
from ollama_client import OllamaClient
from recognition import recognize_event, get_clip_embedding, embedding_to_bytes
from vision import get_description
from cleanup import cleanup_old_events
# --- Configuration ---
MQTT_HOST = os.environ.get("MQTT_HOST", "192.168.86.192")
MQTT_PORT = int(os.environ.get("MQTT_PORT", "1883"))
FRIGATE_URL = os.environ.get("FRIGATE_URL", "http://192.168.86.192:5000")
MATRIX_HOMESERVER = os.environ.get("MATRIX_HOMESERVER", "http://192.168.86.192:8008")
MATRIX_ACCESS_TOKEN = os.environ["MATRIX_ACCESS_TOKEN"]
MATRIX_ROOM_ID = os.environ["MATRIX_ROOM_ID"]
HEALTH_PORT = int(os.environ.get("HEALTH_PORT", "5199"))
ALLOWED_LABELS = {"person", "car", "dog", "cat", "bird"}
MIN_SCORE = 0.7
RATE_LIMIT_SECONDS = 60
DEDUP_WINDOW_SECONDS = 30
# --- Logging ---
logging.basicConfig(
@@ -45,12 +44,46 @@ last_notify_per_camera: defaultdict[str, float] = defaultdict(float)
events_processed = 0
last_event_time: str | None = None
mqtt_connected = False
mqtt_client_instance: mqtt.Client | None = None
# --- Health endpoint ---
class HealthHandler(BaseHTTPRequestHandler):
protocol_version = "HTTP/1.1"
def do_GET(self):
if self.path == "/health":
body = json.dumps(
{
"status": "ok",
"mqtt_connected": mqtt_connected,
"events_processed": events_processed,
"last_event": last_event_time,
}
)
encoded = body.encode()
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(encoded)))
self.end_headers()
self.wfile.write(encoded)
else:
self.send_response(404)
self.end_headers()
def log_message(self, format, *args):
pass # suppress access logs
def start_health_server():
server = HTTPServer(("0.0.0.0", HEALTH_PORT), HealthHandler)
thread = Thread(target=server.serve_forever, daemon=True)
thread.start()
logger.info("Health server started on port %d", HEALTH_PORT)
# --- Snapshot fetching ---
async def fetch_snapshot(event_id: str) -> bytes | None:
url = f"{config.FRIGATE_URL}/api/events/{event_id}/snapshot.jpg"
url = f"{FRIGATE_URL}/api/events/{event_id}/snapshot.jpg"
try:
async with aiohttp.ClientSession() as session:
async with session.get(url) as resp:
@@ -70,8 +103,7 @@ async def fetch_snapshot(event_id: str) -> bytes | None:
# --- Event processing ---
async def process_event(
event_data: dict, claude: anthropic.Anthropic, matrix: MatrixClient,
ollama_client: OllamaClient = None,
event_data: dict, claude: anthropic.Anthropic, matrix: MatrixClient
):
global events_processed, last_event_time
@@ -87,30 +119,27 @@ async def process_event(
has_snapshot = after.get("has_snapshot", False)
# --- Filtering ---
if label not in config.ALLOWED_LABELS:
if label not in ALLOWED_LABELS:
return
if score < config.MIN_SCORE:
if score < MIN_SCORE:
return
if not has_snapshot:
return
# Dedup
now = time.time()
if (
event_id in recent_events
and now - recent_events[event_id] < config.DEDUP_WINDOW_SECONDS
):
if event_id in recent_events and now - recent_events[event_id] < DEDUP_WINDOW_SECONDS:
return
recent_events[event_id] = now
# Rate limit per camera
if now - last_notify_per_camera[camera] < config.RATE_LIMIT_SECONDS:
if now - last_notify_per_camera[camera] < RATE_LIMIT_SECONDS:
logger.info("Rate limited: %s on %s", label, camera)
return
last_notify_per_camera[camera] = now
# Cleanup old dedup entries
cutoff = now - config.DEDUP_WINDOW_SECONDS * 2
cutoff = now - DEDUP_WINDOW_SECONDS * 2
for eid in [k for k, v in recent_events.items() if v < cutoff]:
del recent_events[eid]
@@ -121,84 +150,18 @@ async def process_event(
# --- Fetch snapshot ---
snapshot = await fetch_snapshot(event_id)
# --- Recognition ---
match_status = "unreviewed"
matched_id = None
match_conf = 0.0
matched_name = None
# Check if recognition is enabled
recognition_enabled = False
try:
conn = get_connection()
row = conn.execute("SELECT value FROM settings WHERE key='recognition_enabled'").fetchone()
if row:
recognition_enabled = row["value"].lower() == "true"
except Exception:
pass
if recognition_enabled and snapshot:
match_status, matched_id, match_conf, matched_name = await recognize_event(
snapshot, ollama_client
)
# Store embedding with event
embedding = await get_clip_embedding(snapshot)
if embedding:
embedding_blob = embedding_to_bytes(embedding)
try:
conn = get_connection()
conn.execute(
"UPDATE events SET embedding=?, match_status=?, matched_object_id=?, match_confidence=? WHERE frigate_event_id=?",
(embedding_blob, match_status, matched_id, match_conf, event_id),
)
conn.commit()
except Exception:
logger.exception("Failed to update event recognition")
# If known, check action - skip notification for quiet/suppress
if match_status == "known" and matched_id:
conn = get_connection()
obj = conn.execute("SELECT action FROM known_objects WHERE id=?", (matched_id,)).fetchone()
if obj and obj["action"] in ("quiet", "suppress"):
logger.info("Known object '%s' (%s), skipping notification", matched_name, obj["action"])
events_processed += 1
last_event_time = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
return
# --- AI description ---
vision_backend = "haiku"
try:
conn = get_connection()
row = conn.execute("SELECT value FROM settings WHERE key='vision_backend'").fetchone()
if row:
vision_backend = row["value"]
except Exception:
pass
# --- AI analysis ---
description = None
if snapshot:
description = await get_description(snapshot, vision_backend, claude, ollama_client)
description = analyze_snapshot(claude, snapshot)
if not description:
description = f"{label.capitalize()} detected on {camera}."
# --- Store event in database ---
try:
conn = get_connection()
conn.execute(
"""INSERT OR IGNORE INTO events
(frigate_event_id, camera, label, score, snapshot, match_status, ai_description, notified)
VALUES (?, ?, ?, ?, ?, ?, ?, 1)""",
(event_id, camera, label, score, snapshot, match_status, description),
)
conn.commit()
except Exception:
logger.exception("Failed to store event %s in database", event_id)
# --- Format message ---
score_pct = int(score * 100)
ts = time.strftime("%I:%M:%S %p", time.localtime())
date_str = time.strftime("%b %d, %Y", time.localtime())
frigate_link = f"{config.FRIGATE_URL}/review?id={event_id}"
frigate_link = f"{FRIGATE_URL}/review?id={event_id}"
plain = (
f"{camera} - {label.capitalize()} - {score_pct}%\n"
@@ -232,8 +195,7 @@ def create_mqtt_client(
loop: asyncio.AbstractEventLoop,
claude: anthropic.Anthropic,
matrix: MatrixClient,
ollama_client: OllamaClient = None,
) -> mqtt.Client:
):
global mqtt_connected
def on_connect(client, userdata, flags, reason_code, properties):
@@ -251,7 +213,7 @@ def create_mqtt_client(
try:
payload = json.loads(msg.payload)
asyncio.run_coroutine_threadsafe(
process_event(payload, claude, matrix, ollama_client), loop
process_event(payload, claude, matrix), loop
)
except json.JSONDecodeError:
logger.warning("Invalid JSON on %s", msg.topic)
@@ -265,99 +227,33 @@ def create_mqtt_client(
return client
def start_mqtt(loop: asyncio.AbstractEventLoop) -> None:
"""Start MQTT client in a background thread."""
global mqtt_client_instance
# --- Main ---
def main():
logger.info("Starting frigate-notify")
claude = anthropic.Anthropic()
matrix = MatrixClient(
config.MATRIX_HOMESERVER,
config.MATRIX_ACCESS_TOKEN,
config.MATRIX_ROOM_ID,
)
ollama_client = OllamaClient(config.OLLAMA_URL, config.OLLAMA_MODEL)
matrix = MatrixClient(MATRIX_HOMESERVER, MATRIX_ACCESS_TOKEN, MATRIX_ROOM_ID)
mqtt_client_instance = create_mqtt_client(loop, claude, matrix, ollama_client)
mqtt_client_instance.connect(config.MQTT_HOST, config.MQTT_PORT)
loop = asyncio.new_event_loop()
loop_thread = Thread(target=loop.run_forever, daemon=True)
loop_thread.start()
thread = Thread(target=mqtt_client_instance.loop_forever, daemon=True)
thread.start()
logger.info("MQTT client started in background thread")
start_health_server()
mqtt_client = create_mqtt_client(loop, claude, matrix)
mqtt_client.connect(MQTT_HOST, MQTT_PORT)
def shutdown(signum, frame):
logger.info("Shutting down...")
mqtt_client.disconnect()
loop.call_soon_threadsafe(loop.stop)
sys.exit(0)
signal.signal(signal.SIGTERM, shutdown)
signal.signal(signal.SIGINT, shutdown)
mqtt_client.loop_forever()
# --- FastAPI lifespan ---
@asynccontextmanager
async def lifespan(app: FastAPI):
"""Startup and shutdown logic for the FastAPI application."""
logger.info("Starting frigate-notify v2")
# Initialize database
set_db_path(config.DB_PATH)
init_db()
# Start MQTT listener in background
loop = asyncio.get_event_loop()
start_mqtt(loop)
# Start periodic event cleanup
asyncio.create_task(cleanup_old_events())
yield
# Shutdown
logger.info("Shutting down frigate-notify v2")
if mqtt_client_instance:
mqtt_client_instance.disconnect()
close_connection()
# --- FastAPI app ---
app = FastAPI(
title="frigate-notify",
version="2.0.0",
lifespan=lifespan,
)
# Include API routers
app.include_router(events_router)
app.include_router(known_objects_router)
app.include_router(settings_router)
# Mount static files directory
static_dir = Path(__file__).parent / "static"
static_dir.mkdir(exist_ok=True)
app.mount("/static", StaticFiles(directory=str(static_dir)), name="static")
@app.get("/health")
async def health():
"""Health check endpoint."""
return {
"status": "ok",
"version": "2.0.0",
"mqtt_connected": mqtt_connected,
"events_processed": events_processed,
"last_event": last_event_time,
}
@app.get("/")
async def index():
"""Serve the web UI index page."""
index_path = static_dir / "index.html"
if index_path.exists():
return FileResponse(str(index_path))
return JSONResponse(
{"message": "frigate-notify v2", "docs": "/docs"},
status_code=200,
)
# --- Entrypoint ---
if __name__ == "__main__":
uvicorn.run(
"main:app",
host=config.SERVER_HOST,
port=config.SERVER_PORT,
log_level="info",
)
main()
@@ -1,95 +0,0 @@
"""Pydantic models for frigate-notify v2 API."""
from datetime import datetime
from typing import Optional
from pydantic import BaseModel, Field
# --- Known Objects ---
class KnownObjectCreate(BaseModel):
"""Request model for creating a known object."""
name: str = Field(..., min_length=1, max_length=255)
type: str = Field(..., pattern="^(person|vehicle|pet)$")
action: str = Field(default="quiet", pattern="^(quiet|suppress|alert)$")
class KnownObjectUpdate(BaseModel):
"""Request model for updating a known object."""
name: Optional[str] = Field(default=None, min_length=1, max_length=255)
type: Optional[str] = Field(default=None, pattern="^(person|vehicle|pet)$")
action: Optional[str] = Field(default=None, pattern="^(quiet|suppress|alert)$")
class KnownObjectResponse(BaseModel):
"""Response model for a known object."""
id: int
name: str
type: str
action: str
created_at: str
updated_at: str
# --- Events ---
class TagEventRequest(BaseModel):
"""Request model for tagging an event with a known object.
Either provide known_object_id to link to an existing object,
or provide name + type to create a new known object.
"""
known_object_id: Optional[int] = None
name: Optional[str] = Field(default=None, min_length=1, max_length=255)
type: Optional[str] = Field(default=None, pattern="^(person|vehicle|pet)$")
class EventResponse(BaseModel):
"""Response model for an event."""
id: int
frigate_event_id: str
camera: str
label: str
score: float
has_snapshot: bool = False
snapshot_base64: Optional[str] = None
match_status: str
matched_object_id: Optional[int] = None
matched_object_name: Optional[str] = None
match_confidence: Optional[float] = None
ai_description: Optional[str] = None
notified: bool
created_at: str
# --- Settings ---
class SettingsResponse(BaseModel):
"""Response model for application settings."""
vision_backend: str
recognition_enabled: bool
event_retention_days: int
# --- Stats ---
class StatsResponse(BaseModel):
"""Response model for application statistics."""
total_events: int = 0
known_matches: int = 0
unknown_events: int = 0
unreviewed: int = 0
known_objects_count: int = 0
events_today: int = 0
@@ -1,132 +0,0 @@
"""Ollama vision LLM client for frigate-notify v2."""
import base64
import logging
import aiohttp
logger = logging.getLogger(__name__)
class OllamaClient:
"""Client for Ollama vision models (e.g., llava)."""
def __init__(self, base_url: str, model: str):
self.base_url = base_url.rstrip("/")
self.model = model
async def analyze_image(self, image_bytes: bytes, prompt: str) -> str | None:
"""Send an image to Ollama for analysis.
Args:
image_bytes: Raw image bytes (JPEG).
prompt: Text prompt for the vision model.
Returns:
Response text or None on failure.
"""
b64 = base64.b64encode(image_bytes).decode("utf-8")
payload = {
"model": self.model,
"prompt": prompt,
"images": [b64],
"stream": False,
"options": {"num_predict": 200},
}
try:
async with aiohttp.ClientSession() as session:
async with session.post(
f"{self.base_url}/api/generate",
json=payload,
timeout=aiohttp.ClientTimeout(total=30),
) as resp:
if resp.status == 200:
data = await resp.json()
text = data.get("response", "").strip()
logger.info(
"Ollama analysis complete: %d chars", len(text)
)
return text if text else None
body = await resp.text()
logger.error(
"Ollama analyze_image failed %s: %s", resp.status, body
)
except Exception:
logger.exception("Ollama analyze_image request failed")
return None
async def compare_images(
self,
image_bytes: bytes,
reference_bytes: bytes,
object_name: str,
) -> bool | None:
"""Compare two images to determine if they show the same object/person.
Args:
image_bytes: The new event image bytes.
reference_bytes: The reference (known) image bytes.
object_name: Name of the known object for context.
Returns:
True if same, False if different, None on failure/inconclusive.
"""
b64_new = base64.b64encode(image_bytes).decode("utf-8")
b64_ref = base64.b64encode(reference_bytes).decode("utf-8")
prompt = (
f"Look at these two images. The first image is from a security camera. "
f"The second image is a reference photo of '{object_name}'. "
f"Are these the same person/vehicle/animal? "
f"Answer with ONLY 'yes' or 'no'."
)
payload = {
"model": self.model,
"prompt": prompt,
"images": [b64_new, b64_ref],
"stream": False,
"options": {"num_predict": 200},
}
try:
async with aiohttp.ClientSession() as session:
async with session.post(
f"{self.base_url}/api/generate",
json=payload,
timeout=aiohttp.ClientTimeout(total=30),
) as resp:
if resp.status == 200:
data = await resp.json()
text = data.get("response", "").strip().lower()
logger.info("Ollama compare result: %s", text)
if "yes" in text.split()[:3]:
return True
if "no" in text.split()[:3]:
return False
logger.warning(
"Ollama compare inconclusive: %s", text
)
return None
body = await resp.text()
logger.error(
"Ollama compare_images failed %s: %s",
resp.status,
body,
)
except Exception:
logger.exception("Ollama compare_images request failed")
return None
async def is_available(self) -> bool:
"""Check if Ollama is reachable and responding.
Returns:
True if Ollama responds with HTTP 200, False otherwise.
"""
try:
async with aiohttp.ClientSession() as session:
async with session.get(
f"{self.base_url}/api/tags",
timeout=aiohttp.ClientTimeout(total=3),
) as resp:
return resp.status == 200
except Exception:
return False
@@ -1,232 +0,0 @@
"""Recognition pipeline using CLIP embeddings with Ollama vision fallback."""
import base64
import logging
import struct
from typing import Optional
import aiohttp
import numpy as np
from config import config
from database import get_connection
from ollama_client import OllamaClient
logger = logging.getLogger(__name__)
def embedding_to_bytes(embedding: list[float]) -> bytes:
"""Pack a float32 embedding list into bytes for SQLite storage.
Args:
embedding: List of float values (typically 768-dimensional).
Returns:
Packed bytes (len(embedding) * 4 bytes).
"""
return struct.pack(f"{len(embedding)}f", *embedding)
def bytes_to_embedding(data: bytes) -> np.ndarray:
"""Unpack bytes back into a float32 numpy array.
Args:
data: Packed bytes from embedding_to_bytes().
Returns:
Numpy array of float32 values.
"""
count = len(data) // 4
values = struct.unpack(f"{count}f", data)
return np.array(values, dtype=np.float32)
async def get_clip_embedding(image_bytes: bytes) -> list[float] | None:
"""Get CLIP embedding for an image from the clip-service sidecar.
Args:
image_bytes: Raw image bytes (JPEG).
Returns:
List of float embedding values, or None on failure.
"""
b64 = base64.b64encode(image_bytes).decode("utf-8")
try:
async with aiohttp.ClientSession() as session:
async with session.post(
f"{config.CLIP_SERVICE_URL}/embed",
json={"image": b64},
timeout=aiohttp.ClientTimeout(total=10),
) as resp:
if resp.status == 200:
data = await resp.json()
embedding = data.get("embedding")
if embedding:
logger.info(
"CLIP embedding received: %d dimensions",
len(embedding),
)
return embedding
logger.warning("CLIP response missing embedding field")
else:
body = await resp.text()
logger.error(
"CLIP embed failed %s: %s", resp.status, body
)
except Exception:
logger.exception("CLIP embedding request failed")
return None
async def find_best_match(
query_embedding: list[float],
) -> tuple[int | None, float, str | None]:
"""Find the best matching known object for a query embedding.
Loads all embeddings from the database, computes cosine similarity,
groups by known_object_id and takes the max score per object.
Args:
query_embedding: The query embedding to match against.
Returns:
Tuple of (best_object_id, best_score, best_name).
Returns (None, 0.0, None) if no embeddings exist.
"""
conn = get_connection()
cursor = conn.execute(
"""
SELECT e.id, e.known_object_id, e.embedding, ko.name
FROM embeddings e
JOIN known_objects ko ON e.known_object_id = ko.id
"""
)
rows = cursor.fetchall()
if not rows:
return (None, 0.0, None)
query_vec = np.array(query_embedding, dtype=np.float32)
query_norm = np.linalg.norm(query_vec)
if query_norm == 0:
return (None, 0.0, None)
query_vec = query_vec / query_norm
# Group scores by known_object_id, keeping max per object
best_per_object: dict[int, tuple[float, str]] = {}
for row in rows:
stored_vec = bytes_to_embedding(row["embedding"])
stored_norm = np.linalg.norm(stored_vec)
if stored_norm == 0:
continue
stored_vec = stored_vec / stored_norm
similarity = float(np.dot(query_vec, stored_vec))
obj_id = row["known_object_id"]
obj_name = row["name"]
if obj_id not in best_per_object or similarity > best_per_object[obj_id][0]:
best_per_object[obj_id] = (similarity, obj_name)
if not best_per_object:
return (None, 0.0, None)
best_id = max(best_per_object, key=lambda k: best_per_object[k][0])
best_score, best_name = best_per_object[best_id]
return (best_id, best_score, best_name)
async def recognize_event(
image_bytes: bytes,
ollama_client: Optional[OllamaClient] = None,
) -> tuple[str, int | None, float, str | None]:
"""Full recognition pipeline for a Frigate event image.
Steps:
1. Get CLIP embedding for the image.
2. Find the best matching known object.
3. High confidence (>= threshold_high): return "known".
4. Low confidence (< threshold_low): return "unknown".
5. Ambiguous range: use Ollama vision LLM to confirm/deny.
6. If LLM unavailable or inconclusive: return "ambiguous".
7. If no embedding could be obtained: return "unreviewed".
Args:
image_bytes: Raw image bytes (JPEG).
ollama_client: Optional OllamaClient for ambiguous case fallback.
Returns:
Tuple of (match_status, object_id, score, object_name).
match_status is one of: "known", "unknown", "ambiguous", "unreviewed".
"""
# Step 1: Get CLIP embedding
embedding = await get_clip_embedding(image_bytes)
if embedding is None:
logger.warning("Could not get CLIP embedding, returning unreviewed")
return ("unreviewed", None, 0.0, None)
# Step 2: Find best match
best_id, best_score, best_name = await find_best_match(embedding)
# No embeddings in DB at all
if best_id is None:
return ("unknown", None, 0.0, None)
high_threshold = config.RECOGNITION_THRESHOLD_HIGH
low_threshold = config.RECOGNITION_THRESHOLD_LOW
# Step 3: High confidence match
if best_score >= high_threshold:
logger.info(
"High confidence match: %s (%.3f)", best_name, best_score
)
return ("known", best_id, best_score, best_name)
# Step 4: Low confidence - clearly not a match
if best_score < low_threshold:
logger.info("Low confidence, no match (%.3f)", best_score)
return ("unknown", None, best_score, None)
# Step 5: Ambiguous range - try Ollama vision LLM
if ollama_client is not None:
logger.info(
"Ambiguous score %.3f for %s, trying Ollama comparison",
best_score,
best_name,
)
# Get a reference thumbnail from the embeddings table
conn = get_connection()
cursor = conn.execute(
"""
SELECT thumbnail FROM embeddings
WHERE known_object_id = ? AND thumbnail IS NOT NULL
LIMIT 1
""",
(best_id,),
)
ref_row = cursor.fetchone()
if ref_row and ref_row["thumbnail"] is not None:
result = await ollama_client.compare_images(
image_bytes, ref_row["thumbnail"], best_name
)
if result is True:
logger.info(
"Ollama confirmed match: %s (%.3f)",
best_name,
best_score,
)
return ("known", best_id, best_score, best_name)
elif result is False:
logger.info(
"Ollama denied match: %s (%.3f)", best_name, best_score
)
return ("unknown", None, best_score, None)
# result is None: inconclusive, fall through to ambiguous
# Step 6: LLM unavailable or inconclusive
logger.info(
"Ambiguous match: %s (%.3f)", best_name, best_score
)
return ("ambiguous", best_id, best_score, best_name)
@@ -1,7 +1,3 @@
paho-mqtt==2.1.0
anthropic>=0.40.0
aiohttp>=3.9.0
fastapi>=0.115.0
uvicorn[standard]>=0.34.0
python-multipart>=0.0.18
numpy>=1.26.0
@@ -1,226 +0,0 @@
"""Event API endpoints for frigate-notify v2."""
import base64
import logging
from typing import Optional
from fastapi import APIRouter, HTTPException, Query
from fastapi.responses import Response
from database import get_connection
from models import EventResponse, TagEventRequest
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/events", tags=["events"])
@router.get("", response_model=list[EventResponse])
async def list_events(
camera: Optional[str] = Query(default=None),
label: Optional[str] = Query(default=None),
status: Optional[str] = Query(default=None),
limit: int = Query(default=50, ge=1, le=500),
offset: int = Query(default=0, ge=0),
):
"""List events with optional filters and pagination.
Returns events without snapshot/embedding BLOBs but with has_snapshot bool.
Joins to known_objects for matched_object_name.
"""
conn = get_connection()
query = """
SELECT
e.id, e.frigate_event_id, e.camera, e.label, e.score,
CASE WHEN e.snapshot IS NOT NULL THEN 1 ELSE 0 END AS has_snapshot,
e.match_status, e.matched_object_id,
ko.name AS matched_object_name,
e.match_confidence, e.ai_description, e.notified, e.created_at
FROM events e
LEFT JOIN known_objects ko ON e.matched_object_id = ko.id
"""
conditions = []
params: list = []
if camera:
conditions.append("e.camera = ?")
params.append(camera)
if label:
conditions.append("e.label = ?")
params.append(label)
if status:
conditions.append("e.match_status = ?")
params.append(status)
if conditions:
query += " WHERE " + " AND ".join(conditions)
query += " ORDER BY e.created_at DESC LIMIT ? OFFSET ?"
params.extend([limit, offset])
cursor = conn.execute(query, params)
rows = cursor.fetchall()
return [
EventResponse(
id=row["id"],
frigate_event_id=row["frigate_event_id"],
camera=row["camera"],
label=row["label"],
score=row["score"],
has_snapshot=bool(row["has_snapshot"]),
match_status=row["match_status"],
matched_object_id=row["matched_object_id"],
matched_object_name=row["matched_object_name"],
match_confidence=row["match_confidence"],
ai_description=row["ai_description"],
notified=bool(row["notified"]),
created_at=row["created_at"],
)
for row in rows
]
@router.get("/{event_id}", response_model=EventResponse)
async def get_event(event_id: int):
"""Get a single event detail. Includes base64-encoded snapshot if exists."""
conn = get_connection()
cursor = conn.execute(
"""
SELECT
e.id, e.frigate_event_id, e.camera, e.label, e.score,
e.snapshot,
CASE WHEN e.snapshot IS NOT NULL THEN 1 ELSE 0 END AS has_snapshot,
e.match_status, e.matched_object_id,
ko.name AS matched_object_name,
e.match_confidence, e.ai_description, e.notified, e.created_at
FROM events e
LEFT JOIN known_objects ko ON e.matched_object_id = ko.id
WHERE e.id = ?
""",
(event_id,),
)
row = cursor.fetchone()
if not row:
raise HTTPException(status_code=404, detail="Event not found")
snapshot_b64 = None
if row["snapshot"] is not None:
snapshot_b64 = base64.b64encode(row["snapshot"]).decode("ascii")
return EventResponse(
id=row["id"],
frigate_event_id=row["frigate_event_id"],
camera=row["camera"],
label=row["label"],
score=row["score"],
has_snapshot=bool(row["has_snapshot"]),
snapshot_base64=snapshot_b64,
match_status=row["match_status"],
matched_object_id=row["matched_object_id"],
matched_object_name=row["matched_object_name"],
match_confidence=row["match_confidence"],
ai_description=row["ai_description"],
notified=bool(row["notified"]),
created_at=row["created_at"],
)
@router.get("/{event_id}/snapshot")
async def get_event_snapshot(event_id: int):
"""Return raw JPEG snapshot for an event. 404 if no snapshot."""
conn = get_connection()
cursor = conn.execute(
"SELECT snapshot FROM events WHERE id = ?",
(event_id,),
)
row = cursor.fetchone()
if not row:
raise HTTPException(status_code=404, detail="Event not found")
if row["snapshot"] is None:
raise HTTPException(status_code=404, detail="No snapshot available")
return Response(content=row["snapshot"], media_type="image/jpeg")
@router.post("/{event_id}/tag")
async def tag_event(event_id: int, body: TagEventRequest):
"""Tag an event as a known object.
Either provide known_object_id to link to an existing object,
or provide name + type to create a new known object.
If the event has an embedding, store it in the embeddings table.
"""
conn = get_connection()
# Verify event exists
cursor = conn.execute(
"SELECT id, embedding FROM events WHERE id = ?",
(event_id,),
)
event_row = cursor.fetchone()
if not event_row:
raise HTTPException(status_code=404, detail="Event not found")
known_object_id = body.known_object_id
if known_object_id is not None:
# Verify the known object exists
cursor = conn.execute(
"SELECT id FROM known_objects WHERE id = ?",
(known_object_id,),
)
if not cursor.fetchone():
raise HTTPException(status_code=404, detail="Known object not found")
elif body.name and body.type:
# Create a new known object
cursor = conn.execute(
"INSERT INTO known_objects (name, type) VALUES (?, ?)",
(body.name, body.type),
)
known_object_id = cursor.lastrowid
conn.commit()
else:
raise HTTPException(
status_code=400,
detail="Must provide either known_object_id or both name and type",
)
# If event has an embedding, store it in the embeddings table
if event_row["embedding"] is not None:
conn.execute(
"INSERT INTO embeddings (known_object_id, embedding) VALUES (?, ?)",
(known_object_id, event_row["embedding"]),
)
# Update event
conn.execute(
"""UPDATE events
SET match_status = 'known',
matched_object_id = ?,
match_confidence = 1.0
WHERE id = ?""",
(known_object_id, event_id),
)
conn.commit()
return {"status": "ok", "matched_object_id": known_object_id}
@router.post("/{event_id}/dismiss")
async def dismiss_event(event_id: int):
"""Set event match_status to 'unknown'."""
conn = get_connection()
cursor = conn.execute("SELECT id FROM events WHERE id = ?", (event_id,))
if not cursor.fetchone():
raise HTTPException(status_code=404, detail="Event not found")
conn.execute(
"UPDATE events SET match_status = 'unknown' WHERE id = ?",
(event_id,),
)
conn.commit()
return {"status": "ok"}
@@ -1,177 +0,0 @@
"""Known objects API endpoints for frigate-notify v2."""
import logging
from typing import Optional
from fastapi import APIRouter, HTTPException, Query
from fastapi.responses import Response
from database import get_connection
from models import KnownObjectResponse, KnownObjectUpdate
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/known-objects", tags=["known-objects"])
@router.get("")
async def list_known_objects(
type: Optional[str] = Query(default=None, pattern="^(person|vehicle|pet)$"),
):
"""List all known objects with embedding_count. Optional type filter."""
conn = get_connection()
query = """
SELECT
ko.id, ko.name, ko.type, ko.action,
ko.created_at, ko.updated_at,
COUNT(e.id) AS embedding_count
FROM known_objects ko
LEFT JOIN embeddings e ON ko.id = e.known_object_id
"""
params: list = []
if type:
query += " WHERE ko.type = ?"
params.append(type)
query += " GROUP BY ko.id ORDER BY ko.name"
cursor = conn.execute(query, params)
rows = cursor.fetchall()
return [
{
"id": row["id"],
"name": row["name"],
"type": row["type"],
"action": row["action"],
"created_at": row["created_at"],
"updated_at": row["updated_at"],
"embedding_count": row["embedding_count"],
}
for row in rows
]
@router.get("/{object_id}")
async def get_known_object(object_id: int):
"""Get a single known object with list of its embeddings (id, created_at only)."""
conn = get_connection()
cursor = conn.execute(
"SELECT id, name, type, action, created_at, updated_at FROM known_objects WHERE id = ?",
(object_id,),
)
row = cursor.fetchone()
if not row:
raise HTTPException(status_code=404, detail="Known object not found")
# Get embeddings list
emb_cursor = conn.execute(
"SELECT id, created_at FROM embeddings WHERE known_object_id = ? ORDER BY created_at",
(object_id,),
)
embeddings = [
{"id": e["id"], "created_at": e["created_at"]} for e in emb_cursor.fetchall()
]
return {
"id": row["id"],
"name": row["name"],
"type": row["type"],
"action": row["action"],
"created_at": row["created_at"],
"updated_at": row["updated_at"],
"embeddings": embeddings,
}
@router.get("/{object_id}/thumbnails/{embedding_id}")
async def get_embedding_thumbnail(object_id: int, embedding_id: int):
"""Return raw JPEG thumbnail from embeddings table."""
conn = get_connection()
cursor = conn.execute(
"SELECT thumbnail FROM embeddings WHERE id = ? AND known_object_id = ?",
(embedding_id, object_id),
)
row = cursor.fetchone()
if not row:
raise HTTPException(status_code=404, detail="Embedding not found")
if row["thumbnail"] is None:
raise HTTPException(status_code=404, detail="No thumbnail available")
return Response(content=row["thumbnail"], media_type="image/jpeg")
@router.put("/{object_id}")
async def update_known_object(object_id: int, body: KnownObjectUpdate):
"""Update name/type/action for a known object. Sets updated_at."""
conn = get_connection()
cursor = conn.execute(
"SELECT id FROM known_objects WHERE id = ?",
(object_id,),
)
if not cursor.fetchone():
raise HTTPException(status_code=404, detail="Known object not found")
updates = []
params: list = []
if body.name is not None:
updates.append("name = ?")
params.append(body.name)
if body.type is not None:
updates.append("type = ?")
params.append(body.type)
if body.action is not None:
updates.append("action = ?")
params.append(body.action)
if not updates:
raise HTTPException(status_code=400, detail="No fields to update")
updates.append("updated_at = CURRENT_TIMESTAMP")
params.append(object_id)
conn.execute(
f"UPDATE known_objects SET {', '.join(updates)} WHERE id = ?",
params,
)
conn.commit()
# Return the updated object
cursor = conn.execute(
"SELECT id, name, type, action, created_at, updated_at FROM known_objects WHERE id = ?",
(object_id,),
)
row = cursor.fetchone()
return KnownObjectResponse(
id=row["id"],
name=row["name"],
type=row["type"],
action=row["action"],
created_at=row["created_at"],
updated_at=row["updated_at"],
)
@router.delete("/{object_id}")
async def delete_known_object(object_id: int):
"""Delete a known object. Embeddings cascade delete automatically."""
conn = get_connection()
cursor = conn.execute(
"SELECT id FROM known_objects WHERE id = ?",
(object_id,),
)
if not cursor.fetchone():
raise HTTPException(status_code=404, detail="Known object not found")
conn.execute("DELETE FROM known_objects WHERE id = ?", (object_id,))
conn.commit()
return {"status": "ok"}
@@ -1,78 +0,0 @@
"""Settings and stats API endpoints for frigate-notify v2."""
import logging
from datetime import date
from fastapi import APIRouter
from config import config
from database import get_connection
from models import StatsResponse
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["settings"])
@router.get("/settings")
async def get_settings():
"""Return all settings from DB plus runtime config values."""
conn = get_connection()
cursor = conn.execute("SELECT key, value FROM settings")
db_settings = {row["key"]: row["value"] for row in cursor.fetchall()}
# Merge with runtime config values
return {
**db_settings,
"recognition_threshold_high": config.RECOGNITION_THRESHOLD_HIGH,
"recognition_threshold_low": config.RECOGNITION_THRESHOLD_LOW,
"rate_limit_seconds": config.RATE_LIMIT_SECONDS,
"allowed_labels": list(config.ALLOWED_LABELS),
"min_score": config.MIN_SCORE,
}
@router.put("/settings")
async def update_settings(body: dict):
"""Accept dict body, upsert each key/value into settings table."""
conn = get_connection()
for key, value in body.items():
conn.execute(
"INSERT INTO settings (key, value) VALUES (?, ?) "
"ON CONFLICT(key) DO UPDATE SET value = excluded.value",
(key, str(value)),
)
conn.commit()
return {"status": "ok"}
@router.get("/stats", response_model=StatsResponse)
async def get_stats():
"""Return event and object counts."""
conn = get_connection()
total = conn.execute("SELECT COUNT(*) FROM events").fetchone()[0]
known = conn.execute(
"SELECT COUNT(*) FROM events WHERE match_status = 'known'"
).fetchone()[0]
unknown = conn.execute(
"SELECT COUNT(*) FROM events WHERE match_status = 'unknown'"
).fetchone()[0]
unreviewed = conn.execute(
"SELECT COUNT(*) FROM events WHERE match_status = 'unreviewed'"
).fetchone()[0]
objects_count = conn.execute("SELECT COUNT(*) FROM known_objects").fetchone()[0]
today = date.today().isoformat()
events_today = conn.execute(
"SELECT COUNT(*) FROM events WHERE DATE(created_at) = ?",
(today,),
).fetchone()[0]
return StatsResponse(
total_events=total,
known_matches=known,
unknown_events=unknown,
unreviewed=unreviewed,
known_objects_count=objects_count,
events_today=events_today,
)
@@ -1,578 +0,0 @@
// Frigate Alerts - Web UI
(function () {
'use strict';
let currentView = 'dashboard';
let currentTagEventId = null;
let refreshTimer = null;
let knownObjectsCache = [];
// --- Initialization ---
document.addEventListener('DOMContentLoaded', () => {
setupNavigation();
setupTagModal();
loadView('dashboard');
loadStats();
});
// --- Navigation ---
function setupNavigation() {
document.querySelectorAll('.sidebar-item').forEach(item => {
item.addEventListener('click', (e) => {
e.preventDefault();
const view = item.dataset.view;
loadView(view);
});
});
}
function setActiveNav(view) {
document.querySelectorAll('.sidebar-item').forEach(item => {
item.classList.toggle('active', item.dataset.view === view);
});
}
// --- View Router ---
function loadView(view) {
currentView = view;
setActiveNav(view);
clearAutoRefresh();
switch (view) {
case 'dashboard':
loadDashboard();
startAutoRefresh();
break;
case 'review':
loadReview();
startAutoRefresh();
break;
case 'known':
loadKnown();
break;
case 'settings':
loadSettings();
break;
}
}
// --- Auto Refresh ---
function startAutoRefresh() {
refreshTimer = setInterval(() => {
if (currentView === 'dashboard') {
loadDashboard();
} else if (currentView === 'review') {
loadReview();
}
loadStats();
}, 30000);
}
function clearAutoRefresh() {
if (refreshTimer) {
clearInterval(refreshTimer);
refreshTimer = null;
}
}
// --- Dashboard View ---
function loadDashboard() {
const content = document.getElementById('main-content');
content.innerHTML = '<div class="loading">Loading events</div>';
fetch('/api/events?limit=50')
.then(r => r.json())
.then(events => {
if (!events.length) {
content.innerHTML = `
<h1>Dashboard</h1>
<div class="empty-state">
<p>No events recorded yet.</p>
<p>Events will appear here when Frigate detects objects.</p>
</div>`;
return;
}
content.innerHTML = `
<div class="page-header">
<h1>Dashboard</h1>
</div>
<div class="filters">
<select id="filter-camera" onchange="window._applyFilters()">
<option value="">All Cameras</option>
</select>
<select id="filter-label" onchange="window._applyFilters()">
<option value="">All Labels</option>
</select>
<select id="filter-status" onchange="window._applyFilters()">
<option value="">All Statuses</option>
<option value="known">Known</option>
<option value="unknown">Unknown</option>
<option value="ambiguous">Ambiguous</option>
<option value="unreviewed">Unreviewed</option>
</select>
</div>
<div id="events-list"></div>`;
populateFilterOptions(events);
renderEvents(events);
})
.catch(err => {
content.innerHTML = `<h1>Dashboard</h1><div class="empty-state"><p>Error loading events: ${err.message}</p></div>`;
});
}
function populateFilterOptions(events) {
const cameras = [...new Set(events.map(e => e.camera).filter(Boolean))].sort();
const labels = [...new Set(events.map(e => e.label).filter(Boolean))].sort();
const cameraSelect = document.getElementById('filter-camera');
const labelSelect = document.getElementById('filter-label');
if (cameraSelect) {
cameras.forEach(c => {
const opt = document.createElement('option');
opt.value = c;
opt.textContent = c;
cameraSelect.appendChild(opt);
});
}
if (labelSelect) {
labels.forEach(l => {
const opt = document.createElement('option');
opt.value = l;
opt.textContent = l;
labelSelect.appendChild(opt);
});
}
}
// Expose filter function globally for inline onchange handlers
window._applyFilters = function () {
const camera = document.getElementById('filter-camera')?.value || '';
const label = document.getElementById('filter-label')?.value || '';
const status = document.getElementById('filter-status')?.value || '';
let url = '/api/events?limit=50';
if (camera) url += `&camera=${encodeURIComponent(camera)}`;
if (label) url += `&label=${encodeURIComponent(label)}`;
if (status) url += `&status=${encodeURIComponent(status)}`;
fetch(url)
.then(r => r.json())
.then(events => renderEvents(events))
.catch(err => console.error('Filter error:', err));
};
function renderEvents(events) {
const container = document.getElementById('events-list');
if (!container) return;
if (!events.length) {
container.innerHTML = '<div class="empty-state"><p>No events match the current filters.</p></div>';
return;
}
container.innerHTML = events.map(event => `
<div class="event-card" data-event-id="${event.id}">
<img src="/api/events/${event.id}/snapshot"
onerror="this.style.display='none'"
alt="${event.label || 'snapshot'}">
<div class="event-info">
<div class="event-header">
<span>
<span class="event-camera">${escapeHtml(event.camera || 'unknown')}</span>
<span class="event-label"> &mdash; ${escapeHtml(event.label || 'unknown')}</span>
<span class="event-score">(${(event.score * 100).toFixed(0)}%)</span>
</span>
<span class="badge badge-${event.match_status || 'unreviewed'}">${event.match_status || 'unreviewed'}</span>
</div>
<div class="event-time">${formatTime(event.created_at)}</div>
${event.ai_description ? `<div class="event-description">${escapeHtml(event.ai_description)}</div>` : ''}
${event.matched_object_name ? `<div class="event-label">Identified as: <strong>${escapeHtml(event.matched_object_name)}</strong></div>` : ''}
<div class="event-actions">
<button class="btn btn-primary btn-sm" onclick="window._tagEvent('${event.id}')">Tag</button>
<button class="btn btn-sm" onclick="window._dismissEvent('${event.id}')">Dismiss</button>
</div>
</div>
</div>
`).join('');
}
// --- Review View ---
function loadReview() {
const content = document.getElementById('main-content');
content.innerHTML = '<div class="loading">Loading review queue</div>';
fetch('/api/events?limit=50&status=unreviewed')
.then(r => r.json())
.then(events => {
if (!events.length) {
content.innerHTML = `
<h1>Review Queue</h1>
<div class="empty-state">
<p>No events to review.</p>
<p>All events have been reviewed or dismissed.</p>
</div>`;
return;
}
content.innerHTML = `
<div class="page-header">
<h1>Review Queue</h1>
<span>${events.length} event${events.length !== 1 ? 's' : ''} to review</span>
</div>
<div id="events-list"></div>`;
renderEvents(events);
})
.catch(err => {
content.innerHTML = `<h1>Review Queue</h1><div class="empty-state"><p>Error loading events: ${err.message}</p></div>`;
});
}
// --- Known Objects View ---
function loadKnown() {
const content = document.getElementById('main-content');
content.innerHTML = '<div class="loading">Loading known objects</div>';
fetch('/api/known-objects')
.then(r => r.json())
.then(objects => {
knownObjectsCache = objects;
if (!objects.length) {
content.innerHTML = `
<h1>Known Objects</h1>
<div class="empty-state">
<p>No known objects registered.</p>
<p>Tag events from the Dashboard to add known objects.</p>
</div>`;
return;
}
content.innerHTML = `
<div class="page-header">
<h1>Known Objects</h1>
<span>${objects.length} object${objects.length !== 1 ? 's' : ''}</span>
</div>
<div class="known-grid">
${objects.map(obj => `
<div class="known-card" data-object-id="${obj.id}">
<h4>${escapeHtml(obj.name)}</h4>
<span class="type-badge">${escapeHtml(obj.type)}</span>
<div class="embedding-count">${obj.embedding_count || 0} embedding${(obj.embedding_count || 0) !== 1 ? 's' : ''}</div>
${obj.samples && obj.samples.length ? `
<div class="known-card-samples">
${obj.samples.map(s => `<img src="${s}" onerror="this.style.display='none'" alt="sample">`).join('')}
</div>
` : ''}
<div class="known-card-actions">
<button class="btn btn-danger btn-sm" onclick="window._deleteKnownObject('${obj.id}', '${escapeHtml(obj.name)}')">Delete</button>
</div>
</div>
`).join('')}
</div>`;
})
.catch(err => {
content.innerHTML = `<h1>Known Objects</h1><div class="empty-state"><p>Error loading objects: ${err.message}</p></div>`;
});
}
// --- Settings View ---
function loadSettings() {
const content = document.getElementById('main-content');
content.innerHTML = '<div class="loading">Loading settings</div>';
fetch('/api/settings')
.then(r => r.json())
.then(settings => {
content.innerHTML = `
<h1>Settings</h1>
<div class="settings-form">
<div class="settings-section">
<h3>Vision Analysis</h3>
<div class="form-group">
<label>Vision Backend</label>
<select id="setting-vision-backend">
<option value="haiku" ${settings.vision_backend === 'haiku' ? 'selected' : ''}>Claude Haiku</option>
<option value="ollama" ${settings.vision_backend === 'ollama' ? 'selected' : ''}>Ollama (Local)</option>
<option value="both" ${settings.vision_backend === 'both' ? 'selected' : ''}>Both (Haiku + Ollama)</option>
</select>
<div class="help-text">Which AI backend to use for snapshot analysis</div>
</div>
</div>
<div class="settings-section">
<h3>Recognition</h3>
<div class="form-group">
<label class="checkbox-label">
<input type="checkbox" id="setting-recognition-enabled" ${settings.recognition_enabled ? 'checked' : ''}>
Enable object recognition
</label>
<div class="help-text">Match detected objects against known objects database</div>
</div>
</div>
<div class="settings-section">
<h3>Data Retention</h3>
<div class="form-group">
<label>Event Retention (days)</label>
<input type="number" id="setting-retention-days" value="${settings.event_retention_days || 30}" min="1" max="365">
<div class="help-text">How long to keep event records and snapshots</div>
</div>
</div>
<div class="form-group">
<button class="btn btn-primary" onclick="window._saveSettings()">Save Settings</button>
</div>
</div>`;
})
.catch(err => {
content.innerHTML = `<h1>Settings</h1><div class="empty-state"><p>Error loading settings: ${err.message}</p></div>`;
});
}
window._saveSettings = function () {
const settings = {
vision_backend: document.getElementById('setting-vision-backend').value,
recognition_enabled: document.getElementById('setting-recognition-enabled').checked,
event_retention_days: parseInt(document.getElementById('setting-retention-days').value, 10)
};
fetch('/api/settings', {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(settings)
})
.then(r => {
if (!r.ok) throw new Error('Failed to save settings');
return r.json();
})
.then(() => showToast('Settings saved', 'success'))
.catch(err => showToast('Error: ' + err.message, 'error'));
};
// --- Stats ---
function loadStats() {
fetch('/api/stats')
.then(r => r.json())
.then(stats => {
const el = document.getElementById('stats-summary');
if (!el) return;
el.innerHTML = `
<div class="stat-item">
<span>Total Events</span>
<span class="stat-value">${stats.total_events || 0}</span>
</div>
<div class="stat-item">
<span>Unreviewed</span>
<span class="stat-value">${stats.unreviewed || 0}</span>
</div>
<div class="stat-item">
<span>Known</span>
<span class="stat-value">${stats.known_matches || 0}</span>
</div>
<div class="stat-item">
<span>Unknown</span>
<span class="stat-value">${stats.unknown_events || 0}</span>
</div>
<div class="stat-item">
<span>Known Objects</span>
<span class="stat-value">${stats.known_objects_count || 0}</span>
</div>`;
})
.catch(() => {
// Stats are non-critical, silently fail
});
}
// --- Tag Modal ---
function setupTagModal() {
document.getElementById('tag-cancel').addEventListener('click', closeTagModal);
document.getElementById('tag-save').addEventListener('click', saveTag);
document.getElementById('tag-existing-object').addEventListener('change', (e) => {
const newFields = document.getElementById('tag-new-fields');
newFields.style.display = e.target.value ? 'none' : 'block';
});
// Close modal on backdrop click
document.getElementById('tag-modal').addEventListener('click', (e) => {
if (e.target === document.getElementById('tag-modal')) {
closeTagModal();
}
});
// Close modal on Escape key
document.addEventListener('keydown', (e) => {
if (e.key === 'Escape') closeTagModal();
});
}
window._tagEvent = function (eventId) {
currentTagEventId = eventId;
// Show snapshot in modal
const snapshotContainer = document.getElementById('tag-modal-snapshot');
snapshotContainer.innerHTML = `<img src="/api/events/${eventId}/snapshot" onerror="this.style.display='none'" alt="snapshot">`;
// Load known objects for dropdown
fetch('/api/known-objects')
.then(r => r.json())
.then(objects => {
knownObjectsCache = objects;
const select = document.getElementById('tag-existing-object');
select.innerHTML = '<option value="">-- Create new --</option>';
objects.forEach(obj => {
const opt = document.createElement('option');
opt.value = obj.id;
opt.textContent = `${obj.name} (${obj.type})`;
select.appendChild(opt);
});
})
.catch(() => { });
// Reset form
document.getElementById('tag-name').value = '';
document.getElementById('tag-type').value = 'person';
document.getElementById('tag-new-fields').style.display = 'block';
document.getElementById('tag-existing-object').value = '';
document.getElementById('tag-modal').classList.remove('hidden');
};
function closeTagModal() {
document.getElementById('tag-modal').classList.add('hidden');
currentTagEventId = null;
}
function saveTag() {
if (!currentTagEventId) return;
const existingId = document.getElementById('tag-existing-object').value;
let body;
if (existingId) {
body = { known_object_id: existingId };
} else {
const name = document.getElementById('tag-name').value.trim();
const type = document.getElementById('tag-type').value;
if (!name) {
showToast('Please enter a name', 'error');
return;
}
body = { name: name, type: type };
}
fetch(`/api/events/${currentTagEventId}/tag`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body)
})
.then(r => {
if (!r.ok) throw new Error('Failed to tag event');
return r.json();
})
.then(() => {
showToast('Event tagged successfully', 'success');
closeTagModal();
loadView(currentView);
loadStats();
})
.catch(err => showToast('Error: ' + err.message, 'error'));
}
// --- Event Actions ---
window._dismissEvent = function (eventId) {
fetch(`/api/events/${eventId}/dismiss`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' }
})
.then(r => {
if (!r.ok) throw new Error('Failed to dismiss event');
return r.json();
})
.then(() => {
showToast('Event dismissed', 'success');
// Remove card from DOM for immediate feedback
const card = document.querySelector(`.event-card[data-event-id="${eventId}"]`);
if (card) {
card.style.opacity = '0';
card.style.transition = 'opacity 0.3s';
setTimeout(() => card.remove(), 300);
}
loadStats();
})
.catch(err => showToast('Error: ' + err.message, 'error'));
};
// --- Known Object Actions ---
window._deleteKnownObject = function (objectId, name) {
if (!confirm(`Delete known object "${name}"? This cannot be undone.`)) return;
fetch(`/api/known-objects/${objectId}`, {
method: 'DELETE'
})
.then(r => {
if (!r.ok) throw new Error('Failed to delete object');
return r.json();
})
.then(() => {
showToast(`Deleted "${name}"`, 'success');
loadKnown();
loadStats();
})
.catch(err => showToast('Error: ' + err.message, 'error'));
};
// --- Utilities ---
function escapeHtml(str) {
if (!str) return '';
const div = document.createElement('div');
div.textContent = str;
return div.innerHTML;
}
function formatTime(timestamp) {
if (!timestamp) return '';
const date = new Date(timestamp);
const now = new Date();
const diffMs = now - date;
const diffMins = Math.floor(diffMs / 60000);
const diffHours = Math.floor(diffMs / 3600000);
const diffDays = Math.floor(diffMs / 86400000);
if (diffMins < 1) return 'Just now';
if (diffMins < 60) return `${diffMins}m ago`;
if (diffHours < 24) return `${diffHours}h ago`;
if (diffDays < 7) return `${diffDays}d ago`;
return date.toLocaleDateString() + ' ' + date.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit' });
}
function showToast(message, type) {
const toast = document.createElement('div');
toast.className = `toast toast-${type}`;
toast.textContent = message;
document.body.appendChild(toast);
setTimeout(() => {
toast.style.opacity = '0';
toast.style.transition = 'opacity 0.3s';
setTimeout(() => toast.remove(), 300);
}, 3000);
}
})();
@@ -1,61 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Frigate Alerts</title>
<link rel="stylesheet" href="/static/style.css">
</head>
<body>
<nav id="sidebar">
<div class="sidebar-header">
<h2>Frigate Alerts</h2>
</div>
<div class="sidebar-section">
<a href="#" class="sidebar-item active" data-view="dashboard">Dashboard</a>
<a href="#" class="sidebar-item" data-view="review">Review Queue</a>
<a href="#" class="sidebar-item" data-view="known">Known Objects</a>
<a href="#" class="sidebar-item" data-view="settings">Settings</a>
</div>
<div class="sidebar-section sidebar-stats">
<div id="stats-summary"></div>
</div>
</nav>
<main id="content">
<div id="main-content"></div>
</main>
<div id="tag-modal" class="modal hidden">
<div class="modal-content">
<h3>Tag as Known Object</h3>
<div id="tag-modal-snapshot"></div>
<div class="form-group">
<label>Existing object:</label>
<select id="tag-existing-object">
<option value="">-- Create new --</option>
</select>
</div>
<div id="tag-new-fields">
<div class="form-group">
<label>Name:</label>
<input type="text" id="tag-name" placeholder="e.g., Jeremy, Blue F-150, Max">
</div>
<div class="form-group">
<label>Type:</label>
<select id="tag-type">
<option value="person">Person</option>
<option value="vehicle">Vehicle</option>
<option value="pet">Pet</option>
</select>
</div>
</div>
<div class="modal-actions">
<button id="tag-save" class="btn btn-primary">Save</button>
<button id="tag-cancel" class="btn">Cancel</button>
</div>
</div>
</div>
<script src="/static/app.js"></script>
</body>
</html>
@@ -1,534 +0,0 @@
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
background: #1a1a2e;
color: #e0e0e0;
display: flex;
min-height: 100vh;
}
/* Sidebar */
#sidebar {
width: 220px;
background: #16213e;
padding: 20px 0;
position: fixed;
height: 100vh;
overflow-y: auto;
}
.sidebar-header {
padding: 0 20px 20px;
border-bottom: 1px solid #0f3460;
}
.sidebar-header h2 {
color: #00b4d8;
font-size: 1.2em;
}
.sidebar-section {
padding: 10px 0;
}
.sidebar-item {
display: block;
padding: 10px 20px;
color: #a0a0a0;
text-decoration: none;
transition: all 0.2s;
}
.sidebar-item:hover {
background: #0f3460;
color: #fff;
}
.sidebar-item.active {
background: #0f3460;
color: #00b4d8;
border-left: 3px solid #00b4d8;
}
.sidebar-stats {
padding: 20px;
font-size: 0.85em;
color: #888;
}
/* Main content */
#content {
margin-left: 220px;
flex: 1;
padding: 20px;
}
/* Page headers */
h1 {
margin-bottom: 20px;
}
.page-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 20px;
}
/* Filters */
.filters {
display: flex;
gap: 10px;
margin-bottom: 20px;
flex-wrap: wrap;
}
.filters select,
.filters input {
padding: 6px 10px;
background: #16213e;
border: 1px solid #0f3460;
color: #e0e0e0;
border-radius: 4px;
}
.filters select:focus,
.filters input:focus {
outline: none;
border-color: #00b4d8;
}
/* Event cards */
.event-card {
background: #16213e;
border-radius: 8px;
padding: 15px;
margin-bottom: 15px;
display: flex;
gap: 15px;
transition: border-color 0.2s;
border: 1px solid transparent;
}
.event-card:hover {
border-color: #0f3460;
}
.event-card img {
width: 160px;
height: 120px;
object-fit: cover;
border-radius: 4px;
background: #1a1a2e;
}
.event-info {
flex: 1;
}
.event-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 8px;
}
.event-camera {
font-weight: bold;
color: #00b4d8;
}
.event-label {
color: #a0a0a0;
}
.event-score {
color: #888;
font-size: 0.85em;
}
.event-description {
color: #ccc;
margin: 8px 0;
font-size: 0.9em;
line-height: 1.4;
}
.event-time {
color: #666;
font-size: 0.8em;
margin-bottom: 6px;
}
.event-actions {
margin-top: 10px;
display: flex;
gap: 8px;
}
/* Status badges */
.badge {
padding: 3px 10px;
border-radius: 12px;
font-size: 0.8em;
font-weight: bold;
}
.badge-known {
background: #2d6a4f;
color: #95d5b2;
}
.badge-unknown {
background: #9a4c1a;
color: #f4a261;
}
.badge-ambiguous {
background: #7c6f09;
color: #ffd166;
}
.badge-unreviewed {
background: #444;
color: #aaa;
}
/* Buttons */
.btn {
padding: 6px 14px;
border: none;
border-radius: 4px;
cursor: pointer;
font-size: 0.85em;
transition: all 0.2s;
color: #ccc;
background: #2a2a4a;
}
.btn:hover {
background: #3a3a5a;
color: #fff;
}
.btn-primary {
background: #0f3460;
color: #fff;
}
.btn-primary:hover {
background: #00b4d8;
}
.btn-danger {
background: #7c2929;
color: #fff;
}
.btn-danger:hover {
background: #c0392b;
}
.btn-sm {
padding: 4px 10px;
font-size: 0.8em;
}
/* Modal */
.modal {
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background: rgba(0, 0, 0, 0.7);
display: flex;
align-items: center;
justify-content: center;
z-index: 1000;
}
.modal.hidden {
display: none;
}
.modal-content {
background: #16213e;
border-radius: 8px;
padding: 25px;
width: 450px;
max-width: 90vw;
}
.modal-content h3 {
margin-bottom: 15px;
color: #00b4d8;
}
.modal-actions {
display: flex;
gap: 10px;
justify-content: flex-end;
margin-top: 15px;
}
#tag-modal-snapshot {
margin-bottom: 15px;
}
#tag-modal-snapshot img {
width: 100%;
max-height: 200px;
object-fit: cover;
border-radius: 4px;
}
/* Form elements */
.form-group {
margin-bottom: 12px;
}
.form-group label {
display: block;
margin-bottom: 4px;
color: #a0a0a0;
font-size: 0.9em;
}
.form-group input,
.form-group select {
width: 100%;
padding: 8px 10px;
background: #1a1a2e;
border: 1px solid #0f3460;
color: #e0e0e0;
border-radius: 4px;
}
.form-group input:focus,
.form-group select:focus {
outline: none;
border-color: #00b4d8;
}
.form-group input[type="checkbox"] {
width: auto;
margin-right: 8px;
}
.form-group .checkbox-label {
display: flex;
align-items: center;
cursor: pointer;
}
.form-group .help-text {
font-size: 0.8em;
color: #666;
margin-top: 4px;
}
/* Known objects grid */
.known-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(250px, 1fr));
gap: 15px;
}
.known-card {
background: #16213e;
border-radius: 8px;
padding: 15px;
border: 1px solid transparent;
transition: border-color 0.2s;
}
.known-card:hover {
border-color: #0f3460;
}
.known-card h4 {
color: #00b4d8;
margin-bottom: 5px;
}
.known-card .type-badge {
font-size: 0.8em;
color: #a0a0a0;
text-transform: capitalize;
}
.known-card .embedding-count {
font-size: 0.8em;
color: #666;
margin-top: 6px;
}
.known-card .known-card-actions {
margin-top: 10px;
display: flex;
gap: 8px;
}
.known-card .known-card-samples {
display: flex;
gap: 5px;
margin-top: 10px;
overflow-x: auto;
}
.known-card .known-card-samples img {
width: 60px;
height: 60px;
object-fit: cover;
border-radius: 4px;
background: #1a1a2e;
}
/* Settings */
.settings-form {
max-width: 600px;
}
.settings-form .form-group {
margin-bottom: 15px;
}
.settings-section {
margin-bottom: 30px;
}
.settings-section h3 {
color: #00b4d8;
margin-bottom: 15px;
padding-bottom: 8px;
border-bottom: 1px solid #0f3460;
}
/* Stats in sidebar */
.stat-item {
display: flex;
justify-content: space-between;
margin-bottom: 8px;
}
.stat-value {
color: #00b4d8;
font-weight: bold;
}
/* Empty state */
.empty-state {
text-align: center;
color: #666;
padding: 60px 20px;
}
.empty-state p {
margin-top: 10px;
font-size: 0.9em;
}
/* Loading spinner */
.loading {
text-align: center;
padding: 40px;
color: #666;
}
.loading::after {
content: '';
display: inline-block;
width: 20px;
height: 20px;
border: 2px solid #0f3460;
border-top-color: #00b4d8;
border-radius: 50%;
animation: spin 0.8s linear infinite;
margin-left: 10px;
vertical-align: middle;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
/* Toast notifications */
.toast {
position: fixed;
bottom: 20px;
right: 20px;
padding: 12px 20px;
border-radius: 6px;
color: #fff;
font-size: 0.9em;
z-index: 2000;
animation: slideIn 0.3s ease-out;
}
.toast-success {
background: #2d6a4f;
}
.toast-error {
background: #7c2929;
}
@keyframes slideIn {
from {
transform: translateY(20px);
opacity: 0;
}
to {
transform: translateY(0);
opacity: 1;
}
}
/* Responsive */
@media (max-width: 768px) {
#sidebar {
width: 100%;
height: auto;
position: relative;
}
.sidebar-section {
display: flex;
flex-wrap: wrap;
}
.sidebar-item {
padding: 8px 15px;
}
.sidebar-stats {
display: none;
}
#content {
margin-left: 0;
}
body {
flex-direction: column;
}
.event-card {
flex-direction: column;
}
.event-card img {
width: 100%;
height: auto;
}
.known-grid {
grid-template-columns: 1fr;
}
}
@@ -1,11 +1,7 @@
import base64
import logging
from typing import Optional
import anthropic
from ollama_client import OllamaClient
logger = logging.getLogger(__name__)
PROMPT = (
@@ -44,66 +40,3 @@ def analyze_snapshot(client: anthropic.Anthropic, image_bytes: bytes) -> str | N
except Exception:
logger.exception("Claude Vision API failed")
return None
async def analyze_snapshot_ollama(
ollama_client: OllamaClient, image_bytes: bytes
) -> str | None:
"""Send a snapshot to Ollama vision model for description.
Args:
ollama_client: Configured OllamaClient instance.
image_bytes: Raw image bytes (JPEG).
Returns:
Description string or None on failure.
"""
return await ollama_client.analyze_image(image_bytes, PROMPT)
async def get_description(
image_bytes: bytes,
vision_backend: str,
claude_client: Optional[anthropic.Anthropic] = None,
ollama_client: Optional[OllamaClient] = None,
) -> str | None:
"""Get a scene description using the configured vision backend.
Args:
image_bytes: Raw image bytes (JPEG).
vision_backend: One of "haiku", "ollama", or "both".
"both" tries Ollama first, falls back to Haiku.
claude_client: Anthropic client (required for "haiku" and "both").
ollama_client: OllamaClient instance (required for "ollama" and "both").
Returns:
Description string or None if all backends fail.
"""
if vision_backend == "ollama":
if ollama_client is None:
logger.error("Ollama backend selected but no client provided")
return None
return await analyze_snapshot_ollama(ollama_client, image_bytes)
if vision_backend == "haiku":
if claude_client is None:
logger.error("Haiku backend selected but no client provided")
return None
return analyze_snapshot(claude_client, image_bytes)
if vision_backend == "both":
# Try Ollama first, fall back to Haiku
if ollama_client is not None:
result = await analyze_snapshot_ollama(ollama_client, image_bytes)
if result is not None:
return result
logger.info("Ollama failed, falling back to Haiku")
if claude_client is not None:
return analyze_snapshot(claude_client, image_bytes)
logger.error("Both backend selected but no clients provided")
return None
logger.error("Unknown vision backend: %s", vision_backend)
return None
@@ -12,8 +12,8 @@ services:
- MQTT_HOST=192.168.86.192
- MQTT_PORT=1883
- MATRIX_HOMESERVER=http://192.168.86.192:8008
- CLIP_SERVICE_URL=http://localhost:5200
- OLLAMA_URL=http://192.168.86.150:11434
- CLIP_SERVICE_URL=http://192.168.86.196:5200
- OLLAMA_URL=http://192.168.86.196:11434
- OLLAMA_MODEL=llava:13b
- DB_PATH=/data/frigate-notify.db
volumes:
+36 -10
View File
@@ -8,7 +8,7 @@
# back_porch 192.168.44.63 kerberos:kerberos
# house_south_corner 192.168.44.61 admin:(blank)
# house_south_yard 192.168.44.62 admin:(blank)
# gate_camera 192.168.86.175 admin:(blank)
# gate_camera 192.168.86.175 admin:(blank) <-- OFFLINE: wrong user/pass RTSP error
#
# Profile cameras (/profile1 + /profile2, H.264):
# house_north 192.168.44.51 admin:123456
@@ -166,17 +166,17 @@ ffmpeg:
# =============================================================================
model:
path: /openvino-model/ssdlite_mobilenet_v2.xml
width: 300
height: 300
width: 320
height: 320
input_tensor: nhwc
input_pixel_format: bgr
input_pixel_format: rgb
labelmap_path: /openvino-model/coco_91cl_bkgr.txt
detectors:
ov:
type: openvino
device: GPU
coral:
type: edgetpu
device: usb
detect:
enabled: true
fps: 3
@@ -229,6 +229,21 @@ birdseye:
enabled: true
mode: objects
semantic_search:
enabled: true
model_size: large
face_recognition:
enabled: true
model_size: small
lpr:
enabled: true
classification:
bird:
enabled: true
# =============================================================================
# Camera definitions
# =============================================================================
@@ -334,8 +349,19 @@ cameras:
roles:
- record
# shop_west: throttled to 2fps + stricter filters to reduce detection load
shop_west:
enabled: true
detect:
fps: 2
objects:
filters:
person:
min_area: 8000
threshold: 0.8
car:
min_area: 15000
threshold: 0.8
ffmpeg:
inputs:
- path: rtsp://127.0.0.1:8554/shop_west_sub
@@ -374,13 +400,12 @@ cameras:
- record
# ---------------------------------------------------------------------------
# house_east - HEVC, VAAPI disabled (pw: admin)
# house_east - HEVC source, go2rtc provides H.264 sub, VAAPI enabled
# ---------------------------------------------------------------------------
house_east:
enabled: true
ffmpeg:
hwaccel_args: []
inputs:
- path: rtsp://127.0.0.1:8554/house_east_sub
input_args: preset-rtsp-restream
@@ -459,6 +484,7 @@ cameras:
# ---------------------------------------------------------------------------
# ASH43-W cameras (HEVC main, H.264 sub - go2rtc transcodes main to H.264)
# VAAPI enabled - sub stream is H.264
# ---------------------------------------------------------------------------
pool_room:
@@ -14,12 +14,12 @@ services:
- /mnt/frigate-recordings:/media/frigate
- /etc/localtime:/etc/localtime:ro
tmpfs:
- /tmp/cache:size=4G
- /tmp/cache:size=8G
devices:
- /dev/dri:/dev/dri
- /dev/bus/usb:/dev/bus/usb
mem_limit: 12G
memswap_limit: 12G
memswap_limit: 16G
healthcheck:
test: ["CMD-SHELL", "curl -f http://localhost:5000/api/version || exit 1"]
interval: 30s
@@ -1,2 +0,0 @@
ANTHROPIC_API_KEY=your-anthropic-api-key
OPENAI_API_KEY=your-openai-api-key
@@ -1,55 +0,0 @@
services:
litellm:
image: ghcr.io/berriai/litellm:main-latest
container_name: litellm
restart: unless-stopped
ports:
- "4000:4000"
volumes:
- /mnt/docker-data/configs/litellm/config.yaml:/app/config.yaml
env_file:
- .env
command: ["--config", "/app/config.yaml", "--port", "4000"]
labels:
- "com.centurylinklabs.watchtower.enable=false"
mem_limit: 1G
depends_on:
litellm-db:
condition: service_healthy
healthcheck:
test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:4000/health/liveliness',timeout=5).status==200 else 1)\""]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
litellm-db:
image: postgres:15-alpine
container_name: litellm-db
restart: unless-stopped
user: "1024:100"
environment:
- POSTGRES_DB=litellm
- POSTGRES_USER=litellm
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
volumes:
- /mnt/docker-data/databases/litellm-db:/var/lib/postgresql/data
labels:
- "com.centurylinklabs.watchtower.enable=false"
mem_limit: 256M
healthcheck:
test: ["CMD-SHELL", "pg_isready -U litellm -d litellm"]
interval: 10s
timeout: 5s
retries: 5
start_period: 30s
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
@@ -1,24 +0,0 @@
services:
node-exporter:
image: prom/node-exporter:v1.7.0
container_name: node-exporter
hostname: hf-pdocker-01
network_mode: host
pid: host
restart: unless-stopped
volumes:
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /:/rootfs:ro
command:
- '--path.procfs=/host/proc'
- '--path.rootfs=/rootfs'
- '--path.sysfs=/host/sys'
- '--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)'
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
labels:
- "com.centurylinklabs.watchtower.enable=false"
@@ -0,0 +1,29 @@
services:
open-webui:
image: ghcr.io/open-webui/open-webui:main
container_name: open-webui
restart: unless-stopped
ports:
- "3010:8080"
env_file:
- .env
environment:
- TZ=America/Chicago
- OLLAMA_BASE_URL=http://192.168.86.196:11434
volumes:
- /mnt/docker-data/configs/openwebui:/app/backend/data
mem_limit: 1G
memswap_limit: 1G
labels:
- "com.centurylinklabs.watchtower.enable=false"
healthcheck:
test: ["CMD-SHELL", "curl -sf http://localhost:8080/health || exit 1"]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
@@ -1,22 +0,0 @@
services:
telegraf:
image: telegraf:latest
container_name: telegraf
hostname: hf-pdocker-01
restart: unless-stopped
environment:
- HOST_PROC=/host/proc
- HOST_SYS=/host/sys
- HOST_ETC=/host/etc
volumes:
- /mnt/docker-data/configs/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- /sys:/host/sys:ro
- /proc:/host/proc:ro
- /etc:/host/etc:ro
mem_limit: 512M
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
@@ -0,0 +1,6 @@
LIVEKIT_API_KEY=voicecoach-89a06ad0
LIVEKIT_API_SECRET=Dn16mQJU5lTCWwXOrUO8LPflODi3LGiMHdxKdQj4/iM
ANTHROPIC_API_KEY=
OPENCLAW_TOKEN=f31cc58c47c2aa420901576b4e90a0101d52e9ec0a084186
HA_TOKEN=
SSH_PASS=&Rip8a115
@@ -0,0 +1,16 @@
FROM python:3.11-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
curl libsndfile1 ffmpeg \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
HEALTHCHECK --interval=30s --timeout=10s --start-period=60s \
CMD curl -sf http://localhost:8080/health || exit 1
CMD ["python", "main.py", "start"]
@@ -0,0 +1,185 @@
"""VoiceCoach v2 — livekit-agents 1.5."""
import asyncio
import json
import logging
import os
import uuid
from livekit import rtc
from livekit.agents import (
Agent,
AgentSession,
AutoSubscribe,
JobContext,
RoomInputOptions,
llm as lk_llm,
)
from livekit.plugins import openai as lk_openai
from livekit.plugins import silero
import store
from modes import DEFAULT_MODE, MODES, ModeConfig
from tools import ALL_TOOLS
from vocal import VocalAnalyzer
logger = logging.getLogger(__name__)
SPEACHES_URL = os.getenv("SPEACHES_URL", "http://192.168.86.27:8000")
OPENCLAW_URL = os.getenv("OPENCLAW_URL", "http://voicecoach-openclaw:3100")
OPENCLAW_TOKEN = os.getenv("OPENCLAW_TOKEN", "")
WHISPER_MODEL = "deepdml/faster-whisper-large-v3-turbo-ct2"
KOKORO_MODEL = "speaches-ai/Kokoro-82M-v1.0-ONNX-fp16"
KOKORO_VOICE = os.getenv("KOKORO_VOICE", "af_nova")
def _build_instructions(mode: ModeConfig, context: str = "") -> str:
if context:
return f"{mode.instructions}\n\n{context}"
return mode.instructions
class VoiceCoachAgent(Agent):
def __init__(self, mode_name: str, context: str, session_id: str, room: rtc.Room):
mode = MODES[mode_name]
super().__init__(
instructions=_build_instructions(mode, context),
tools=ALL_TOOLS,
)
self._mode_name = mode_name
self._session_id = session_id
self._room = room
self._vocal = VocalAnalyzer()
async def on_enter(self):
await store.create_session(self._session_id, self._mode_name)
self.session.generate_reply(
instructions="Greet Jeremy warmly in one sentence. Be natural, not formal."
)
async def on_exit(self):
await store.end_session(self._session_id)
async def on_user_turn_completed(
self,
turn_ctx: lk_llm.ChatContext,
new_message: lk_llm.ChatMessage,
):
content = new_message.text_content or ""
mode = MODES[self._mode_name]
# Inject vocal state tag into message if mode uses it
if mode.vocal_feedback and content:
vocal = self._vocal.current
if vocal.energy > 0.02:
tag = vocal.to_tag()
# Update message content to include vocal context
new_message.content = [f"{tag}\n{content}"]
asyncio.create_task(
store.add_message(
self._session_id,
"user",
content,
vocal_energy=self._vocal.current.energy,
vocal_emotion=self._vocal.current.emotion,
)
)
await self._push({
"type": "final_transcript",
"text": content,
"speaker": "user",
})
def switch_mode(self, mode_name: str):
if mode_name not in MODES:
logger.warning("Unknown mode: %s", mode_name)
return
self._mode_name = mode_name
self.update_instructions(_build_instructions(MODES[mode_name]))
asyncio.create_task(self._push({"type": "mode_ack", "mode": mode_name}))
logger.info("Mode → %s", mode_name)
async def _push(self, payload: dict):
try:
data = json.dumps(payload).encode()
await self._room.local_participant.publish_data(data, reliable=True)
except Exception as e:
logger.debug("Data push failed: %s", e)
async def entrypoint(ctx: JobContext):
await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)
session_id = uuid.uuid4().hex[:8]
context = await store.get_recent_context(limit=3)
agent = VoiceCoachAgent(
mode_name=DEFAULT_MODE,
context=context,
session_id=session_id,
room=ctx.room,
)
session = AgentSession(
stt=lk_openai.STT(
base_url=f"{SPEACHES_URL}/v1",
model=WHISPER_MODEL,
api_key="speaches",
),
llm=lk_openai.LLM(
base_url=f"{OPENCLAW_URL}/v1",
model="openclaw/assistant",
api_key=OPENCLAW_TOKEN,
),
tts=lk_openai.TTS(
base_url=f"{SPEACHES_URL}/v1",
model=KOKORO_MODEL,
voice=KOKORO_VOICE,
api_key="speaches",
),
vad=silero.VAD.load(),
allow_interruptions=True,
min_endpointing_delay=0.4,
max_endpointing_delay=2.0,
)
# Data channel → mode switching
@ctx.room.on("data_received")
def on_data(packet: rtc.DataPacket):
try:
msg = json.loads(packet.data.decode())
if msg.get("type") == "mode_change":
agent.switch_mode(msg.get("mode", DEFAULT_MODE))
except Exception:
pass
# Session events → browser
@session.on("user_input_transcribed")
def on_transcribed(ev):
asyncio.create_task(agent._push({
"type": "partial_transcript" if not ev.is_final else "final_transcript",
"text": ev.transcript,
"speaker": "user",
}))
@session.on("conversation_item_added")
def on_item(ev):
item = ev.item
if hasattr(item, "role") and item.role == "assistant":
text = item.text_content or ""
if text:
asyncio.create_task(agent._push({
"type": "agent_response",
"text": text,
"speaker": "assistant",
}))
asyncio.create_task(
store.add_message(session_id, "assistant", text)
)
await session.start(
agent,
room=ctx.room,
room_input_options=RoomInputOptions(),
)
@@ -0,0 +1,29 @@
"""VoiceCoach v2 — entry point."""
import logging
import os
import threading
from livekit.agents import WorkerOptions, cli
import server
from agent import entrypoint
logging.basicConfig(level=logging.INFO)
def run_api_server():
server.run()
if __name__ == "__main__":
api_thread = threading.Thread(target=run_api_server, daemon=True)
api_thread.start()
cli.run_app(
WorkerOptions(
entrypoint_fnc=entrypoint,
ws_url=os.getenv("LIVEKIT_URL", "ws://voicecoach-livekit:7880"),
api_key=os.getenv("LIVEKIT_API_KEY"),
api_secret=os.getenv("LIVEKIT_API_SECRET"),
)
)
@@ -0,0 +1,61 @@
from dataclasses import dataclass
@dataclass
class ModeConfig:
name: str
instructions: str
max_tokens: int
vocal_feedback: bool
MODES: dict[str, ModeConfig] = {
"coaching": ModeConfig(
name="coaching",
instructions="""You are a personal voice coach and communication advisor. You are speaking with Jeremy.
Keep responses conversational and concise (2-4 sentences unless asked for more). You receive a [VOCAL] tag before each message with paralinguistic analysis. Use it to provide feedback on BOTH what Jeremy says AND how he sounds.
Never quote raw numbers from the vocal analysis. Translate them naturally: "I notice some tension in your voice" not "energy=0.8".
Available modes (Jeremy switches by saying the phrase or clicking the UI):
- "coaching mode" (default) — active feedback on content and delivery
- "vent mode" — listen and reflect, minimal advice
- "practice mode" — roleplay a difficult conversation partner
- "briefing mode" — summarize tasks and context from memory
You have tools to interact with Jeremy's homelab infrastructure. Use them when asked about system status, container health, alerts, or home automation. Be concise about results: "hf-pdocker-01 looks healthy, 38 containers running" not a full list.""",
max_tokens=512,
vocal_feedback=True,
),
"vent": ModeConfig(
name="vent",
instructions="""You are a warm, non-judgmental listener. Jeremy wants to vent or think something through.
Listen, reflect back what you hear, and ask one gentle follow-up question. Keep responses short (1-3 sentences). No coaching, no advice unless explicitly asked.
Ignore the [VOCAL] tag data — just be present.""",
max_tokens=256,
vocal_feedback=False,
),
"practice": ModeConfig(
name="practice",
instructions="""You are playing a difficult conversation partner for Jeremy to practice with.
Stay in character as: a skeptical stakeholder, a demanding manager, or whoever Jeremy describes. Be realistic — push back, ask hard questions, don't cave too easily. Only break character if Jeremy says "stop practice".
Keep your responses to 1-3 sentences to keep the conversation moving.""",
max_tokens=256,
vocal_feedback=False,
),
"briefing": ModeConfig(
name="briefing",
instructions="""You are giving Jeremy a morning briefing. Check infrastructure status, pull any alerts, and summarize what needs attention.
Be concise and actionable. Lead with anything that needs immediate attention, then routine status. Use tools to get current data — don't guess.""",
max_tokens=768,
vocal_feedback=False,
),
}
DEFAULT_MODE = "coaching"
@@ -1,7 +1,7 @@
pipecat-ai[livekit,silero,openai]==0.0.48
httpx==0.27.0
python-dotenv==1.0.1
structlog==24.4.0
livekit-api==0.7.1
livekit-agents[openai,silero]==1.5.15
fastapi==0.115.0
uvicorn[standard]==0.30.0
aiosqlite==0.20.0
httpx==0.27.0
structlog==24.4.0
python-dotenv==1.0.1
@@ -0,0 +1,83 @@
"""Token server + sessions REST API — FastAPI."""
import asyncio
import os
from datetime import timedelta
import uvicorn
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from livekit import api as livekit_api
from pydantic import BaseModel
import store
LIVEKIT_API_KEY = os.environ["LIVEKIT_API_KEY"]
LIVEKIT_API_SECRET = os.environ["LIVEKIT_API_SECRET"]
LIVEKIT_PUBLIC_URL = os.getenv("LIVEKIT_PUBLIC_URL", "wss://voice.hobbs.farm")
app = FastAPI(title="VoiceCoach API")
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_methods=["*"],
allow_headers=["*"],
)
class TokenRequest(BaseModel):
identity: str = "jeremy"
@app.on_event("startup")
async def startup():
await store.init_db()
@app.get("/health")
async def health():
return {"status": "ok", "version": "2.0"}
@app.post("/api/token")
async def get_token(req: TokenRequest):
token = livekit_api.AccessToken(LIVEKIT_API_KEY, LIVEKIT_API_SECRET)
token.with_grants(livekit_api.VideoGrants(
room_join=True,
room="voicecoach",
can_publish=True,
can_subscribe=True,
))
token.identity = req.identity
token.ttl = timedelta(hours=8)
return {"token": token.to_jwt(), "url": LIVEKIT_PUBLIC_URL}
@app.get("/api/sessions")
async def list_sessions(limit: int = 20):
return await store.get_sessions(limit=limit)
@app.get("/api/sessions/{session_id}/messages")
async def get_messages(session_id: str):
msgs = await store.get_session_messages(session_id)
if not msgs:
raise HTTPException(status_code=404, detail="Session not found")
return msgs
@app.get("/api/profile/{key}")
async def get_profile(key: str):
value = await store.get_profile(key)
if value is None:
raise HTTPException(status_code=404, detail="Key not found")
return {"key": key, "value": value}
@app.put("/api/profile/{key}")
async def set_profile_key(key: str, body: dict):
await store.set_profile(key, str(body.get("value", "")))
return {"ok": True}
def run():
uvicorn.run(app, host="0.0.0.0", port=8080, log_level="info")
@@ -0,0 +1,140 @@
"""SQLite session store — WAL mode, aiosqlite."""
import json
import os
import time
from typing import Optional
import aiosqlite
DB_PATH = os.getenv("DB_PATH", "/data/sessions.db")
async def init_db():
async with aiosqlite.connect(DB_PATH) as db:
await db.execute("PRAGMA journal_mode=WAL")
await db.executescript("""
CREATE TABLE IF NOT EXISTS sessions (
id TEXT PRIMARY KEY,
started_at REAL NOT NULL,
ended_at REAL,
mode TEXT DEFAULT 'coaching',
summary TEXT
);
CREATE TABLE IF NOT EXISTS messages (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id TEXT NOT NULL,
role TEXT NOT NULL,
content TEXT NOT NULL,
vocal_energy REAL,
vocal_emotion TEXT,
ts REAL NOT NULL,
FOREIGN KEY(session_id) REFERENCES sessions(id)
);
CREATE TABLE IF NOT EXISTS profile (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at REAL NOT NULL
);
""")
await db.commit()
async def create_session(session_id: str, mode: str = "coaching"):
async with aiosqlite.connect(DB_PATH) as db:
await db.execute(
"INSERT OR IGNORE INTO sessions (id, started_at, mode) VALUES (?, ?, ?)",
(session_id, time.time(), mode),
)
await db.commit()
async def end_session(session_id: str, summary: Optional[str] = None):
async with aiosqlite.connect(DB_PATH) as db:
await db.execute(
"UPDATE sessions SET ended_at=?, summary=? WHERE id=?",
(time.time(), summary, session_id),
)
await db.commit()
async def add_message(
session_id: str,
role: str,
content: str,
vocal_energy: Optional[float] = None,
vocal_emotion: Optional[str] = None,
):
async with aiosqlite.connect(DB_PATH) as db:
await db.execute(
"INSERT INTO messages (session_id, role, content, vocal_energy, vocal_emotion, ts) VALUES (?,?,?,?,?,?)",
(session_id, role, content, vocal_energy, vocal_emotion, time.time()),
)
await db.commit()
async def get_recent_context(limit: int = 3) -> str:
"""Return a brief context string from the last N sessions for injection into system prompt."""
async with aiosqlite.connect(DB_PATH) as db:
async with db.execute(
"SELECT id, started_at, mode, summary FROM sessions WHERE ended_at IS NOT NULL ORDER BY ended_at DESC LIMIT ?",
(limit,),
) as cur:
rows = await cur.fetchall()
if not rows:
return ""
parts = ["Recent session context:"]
for session_id, started_at, mode, summary in rows:
import datetime
dt = datetime.datetime.fromtimestamp(started_at).strftime("%Y-%m-%d")
if summary:
parts.append(f"- {dt} ({mode}): {summary}")
return "\n".join(parts)
async def get_sessions(limit: int = 20) -> list[dict]:
async with aiosqlite.connect(DB_PATH) as db:
async with db.execute(
"SELECT id, started_at, ended_at, mode, summary FROM sessions ORDER BY started_at DESC LIMIT ?",
(limit,),
) as cur:
rows = await cur.fetchall()
return [
{
"id": r[0],
"started_at": r[1],
"ended_at": r[2],
"mode": r[3],
"summary": r[4],
}
for r in rows
]
async def get_session_messages(session_id: str) -> list[dict]:
async with aiosqlite.connect(DB_PATH) as db:
async with db.execute(
"SELECT role, content, vocal_emotion, ts FROM messages WHERE session_id=? ORDER BY ts",
(session_id,),
) as cur:
rows = await cur.fetchall()
return [{"role": r[0], "content": r[1], "vocal_emotion": r[2], "ts": r[3]} for r in rows]
async def get_profile(key: str) -> Optional[str]:
async with aiosqlite.connect(DB_PATH) as db:
async with db.execute("SELECT value FROM profile WHERE key=?", (key,)) as cur:
row = await cur.fetchone()
return row[0] if row else None
async def set_profile(key: str, value: str):
async with aiosqlite.connect(DB_PATH) as db:
await db.execute(
"INSERT OR REPLACE INTO profile (key, value, updated_at) VALUES (?,?,?)",
(key, value, time.time()),
)
await db.commit()
@@ -0,0 +1,186 @@
"""Homelab tools wired into the agent via @function_tool."""
import asyncio
import logging
import os
from typing import Annotated
import httpx
from livekit.agents import llm
logger = logging.getLogger(__name__)
# Homelab node config
NODES = {
"hf-pdocker-01": "192.168.86.192",
"hf-pdocker-02": "192.168.86.100",
"bart": "192.168.86.167",
"gpu": "192.168.86.27",
}
SSH_USER = "glyptic"
SSH_PASS = os.getenv("SSH_PASS", "&Rip8a115")
FARM_MANAGER_URL = "http://192.168.86.192:8888"
HA_URL = os.getenv("HA_URL", "http://192.168.86.192:8123")
HA_TOKEN = os.getenv("HA_TOKEN", "")
ALERTMANAGER_URL = "http://192.168.86.192:9093"
SEARXNG_URL = "http://192.168.86.100:8888"
async def _ssh_exec(host: str, cmd: str, timeout: float = 15.0) -> str:
"""Run a command on a remote host via sshpass."""
proc = await asyncio.create_subprocess_exec(
"sshpass", "-p", SSH_PASS,
"ssh", "-o", "StrictHostKeyChecking=no", "-o", "ConnectTimeout=5",
f"{SSH_USER}@{host}", cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
try:
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=timeout)
except asyncio.TimeoutError:
proc.kill()
return f"ERROR: command timed out after {timeout}s"
if proc.returncode != 0:
return f"ERROR: {stderr.decode().strip()}"
return stdout.decode().strip()
@llm.function_tool
async def infra_status(
node: Annotated[str, "Node name: hf-pdocker-01, hf-pdocker-02, bart, gpu, or 'all'"],
) -> str:
"""Check container health on a homelab node. Returns running container count and any unhealthy containers."""
nodes = list(NODES.keys()) if node == "all" else [node]
results = []
for n in nodes:
ip = NODES.get(n)
if not ip:
results.append(f"{n}: unknown node")
continue
out = await _ssh_exec(ip, "docker ps --format '{{.Names}}\\t{{.Status}}' | head -50")
lines = [l for l in out.splitlines() if l.strip()]
unhealthy = [l for l in lines if "unhealthy" in l.lower() or "exited" in l.lower()]
results.append(
f"{n}: {len(lines)} containers running"
+ (f", ISSUES: {'; '.join(l.split(chr(9))[0] for l in unhealthy)}" if unhealthy else ", all healthy")
)
return "\n".join(results)
@llm.function_tool
async def container_action(
node: Annotated[str, "Node name: hf-pdocker-01, hf-pdocker-02, bart, gpu"],
container: Annotated[str, "Container name"],
action: Annotated[str, "Action: restart, stop, or start"],
) -> str:
"""Restart, stop, or start a Docker container on a homelab node."""
ip = NODES.get(node)
if not ip:
return f"Unknown node: {node}"
if action not in ("restart", "stop", "start"):
return f"Invalid action: {action}. Use restart, stop, or start."
result = await _ssh_exec(ip, f"docker {action} {container}")
return f"{action} {container} on {node}: {'done' if container in result or result == '' else result}"
@llm.function_tool
async def check_logs(
node: Annotated[str, "Node name"],
container: Annotated[str, "Container name"],
lines: Annotated[int, "Number of log lines to return (max 50)"] = 20,
) -> str:
"""Get recent logs from a container."""
ip = NODES.get(node)
if not ip:
return f"Unknown node: {node}"
lines = min(lines, 50)
return await _ssh_exec(ip, f"docker logs --tail {lines} {container} 2>&1")
@llm.function_tool
async def read_alerts() -> str:
"""Check Prometheus Alertmanager for active alerts."""
try:
async with httpx.AsyncClient(timeout=5.0) as client:
r = await client.get(f"{ALERTMANAGER_URL}/api/v2/alerts?active=true&silenced=false")
r.raise_for_status()
alerts = r.json()
if not alerts:
return "No active alerts."
parts = []
for a in alerts[:10]:
name = a.get("labels", {}).get("alertname", "unknown")
severity = a.get("labels", {}).get("severity", "")
summary = a.get("annotations", {}).get("summary", "")
parts.append(f"[{severity}] {name}: {summary}" if summary else f"[{severity}] {name}")
return "\n".join(parts)
except Exception as e:
return f"Could not reach Alertmanager: {e}"
@llm.function_tool
async def home_assistant(
action: Annotated[str, "get_states, turn_on, turn_off, or toggle"],
entity_id: Annotated[str, "HA entity ID, e.g. light.living_room or 'all' for get_states"] = "all",
) -> str:
"""Interact with Home Assistant — check sensor states or control devices."""
if not HA_TOKEN:
return "HA_TOKEN not configured."
headers = {"Authorization": f"Bearer {HA_TOKEN}", "Content-Type": "application/json"}
try:
async with httpx.AsyncClient(timeout=5.0) as client:
if action == "get_states":
if entity_id == "all":
r = await client.get(f"{HA_URL}/api/states", headers=headers)
states = r.json()
# Summarize lights/switches/sensors
lights = [s for s in states if s["entity_id"].startswith("light.")]
sensors = [s for s in states if s["entity_id"].startswith("sensor.")]
on_lights = [s["entity_id"].split(".")[1] for s in lights if s["state"] == "on"]
return (
f"{len(lights)} lights ({len(on_lights)} on: {', '.join(on_lights[:5])}), "
f"{len(sensors)} sensors"
)
else:
r = await client.get(f"{HA_URL}/api/states/{entity_id}", headers=headers)
s = r.json()
return f"{entity_id}: {s.get('state')} (attrs: {list(s.get('attributes', {}).keys())[:3]})"
else:
service = action.replace("turn_", "").replace("toggle", "toggle")
domain = entity_id.split(".")[0]
r = await client.post(
f"{HA_URL}/api/services/{domain}/{action}",
headers=headers,
json={"entity_id": entity_id},
)
return f"{action} {entity_id}: {'done' if r.status_code in (200, 201) else r.text}"
except Exception as e:
return f"HA error: {e}"
@llm.function_tool
async def search_web(
query: Annotated[str, "Search query"],
) -> str:
"""Search the web using the local SearXNG instance."""
try:
async with httpx.AsyncClient(timeout=8.0) as client:
r = await client.get(
f"{SEARXNG_URL}/search",
params={"q": query, "format": "json", "categories": "general"},
)
r.raise_for_status()
data = r.json()
results = data.get("results", [])[:4]
if not results:
return "No results found."
parts = []
for res in results:
title = res.get("title", "")
content = res.get("content", "")[:150]
parts.append(f"• {title}: {content}")
return "\n".join(parts)
except Exception as e:
return f"Search failed: {e}"
ALL_TOOLS = [infra_status, container_action, check_logs, read_alerts, home_assistant, search_web]
@@ -0,0 +1,110 @@
"""Async vocal analysis — runs background, never blocks TTS path."""
import asyncio
import io
import logging
import math
import struct
import wave
from dataclasses import dataclass
from typing import Callable, Optional
import httpx
logger = logging.getLogger(__name__)
SPEECHBRAIN_URL = "http://speechbrain-service:8001"
@dataclass
class VocalState:
emotion: str = "neutral"
confidence: float = 0.0
energy: float = 0.0
pitch_hz: Optional[float] = None
rate: float = 0.0
def to_tag(self) -> str:
parts = [f"emotion={self.emotion}", f"energy={self.energy:.2f}"]
if self.pitch_hz:
parts.append(f"pitch={self.pitch_hz:.0f}hz")
if self.rate:
parts.append(f"rate={'fast' if self.rate > 3 else 'slow' if self.rate < 1.5 else 'normal'}")
return f"[VOCAL {' '.join(parts)}]"
def compute_energy(audio_bytes: bytes) -> float:
if len(audio_bytes) < 2:
return 0.0
samples = struct.unpack_from("<" + "h" * (len(audio_bytes) // 2), audio_bytes)
rms = math.sqrt(sum(x * x for x in samples) / len(samples)) / 32768.0
return round(min(rms * 10, 1.0), 3)
async def analyze_audio(audio_bytes: bytes) -> VocalState:
"""Call SpeechBrain for emotion/pitch. Falls back to energy-only if unavailable."""
energy = compute_energy(audio_bytes)
state = VocalState(energy=energy)
try:
buf = io.BytesIO()
with wave.open(buf, "wb") as wf:
wf.setnchannels(1)
wf.setsampwidth(2)
wf.setframerate(16000)
wf.writeframes(audio_bytes)
wav_bytes = buf.getvalue()
async with httpx.AsyncClient(timeout=3.0) as client:
r = await client.post(
f"{SPEECHBRAIN_URL}/analyze",
files={"audio": ("u.wav", wav_bytes, "audio/wav")},
)
r.raise_for_status()
d = r.json()
state.emotion = d.get("emotion", "neutral")
state.confidence = d.get("emotion_confidence", 0.0)
state.pitch_hz = d.get("pitch_mean_hz")
state.rate = d.get("speaking_rate_approx", 0.0)
except Exception as e:
logger.debug("SpeechBrain unavailable, using energy-only: %s", e)
return state
class VocalAnalyzer:
"""Holds the last completed vocal state; updates async after each utterance."""
def __init__(self, on_update: Optional[Callable[[VocalState], None]] = None):
self._state = VocalState()
self._on_update = on_update
self._audio_buffer: list[bytes] = []
self._collecting = False
@property
def current(self) -> VocalState:
return self._state
def start_collecting(self):
self._collecting = True
self._audio_buffer = []
def feed(self, audio_chunk: bytes):
if self._collecting:
self._audio_buffer.append(audio_chunk)
def stop_collecting_and_analyze(self) -> asyncio.Task:
self._collecting = False
audio = b"".join(self._audio_buffer)
self._audio_buffer = []
return asyncio.create_task(self._analyze_bg(audio))
async def _analyze_bg(self, audio_bytes: bytes):
if not audio_bytes:
return
try:
state = await analyze_audio(audio_bytes)
self._state = state
if self._on_update:
self._on_update(state)
except Exception as e:
logger.warning("Vocal analysis failed: %s", e)
@@ -1,6 +1,58 @@
services:
# LiveKit WebRTC Server (unchanged)
# livekit-agents 1.5 worker + token/API server
voicecoach-agent:
build:
context: ./agent
image: voicecoach-agent:latest
container_name: voicecoach-agent
restart: unless-stopped
env_file: .env
environment:
- LIVEKIT_URL=ws://voicecoach-livekit:7880
- LIVEKIT_PUBLIC_URL=wss://voice.hobbs.farm
- SPEACHES_URL=http://192.168.86.27:8000
- OPENCLAW_URL=http://voicecoach-openclaw:3100
- DB_PATH=/data/sessions.db
- KOKORO_VOICE=af_nova
volumes:
- /mnt/docker-data/configs/voicecoach/sessions:/data
networks:
- voicecoach
- voicecoach_v1
mem_limit: 1G
memswap_limit: 1G
labels:
- "com.centurylinklabs.watchtower.enable=false"
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
# SvelteKit web client
voicecoach-web:
build:
context: ./web-client
image: voicecoach-web-v2:latest
container_name: voicecoach-web-v2
restart: unless-stopped
ports:
- "8085:80"
depends_on:
- voicecoach-agent
networks:
- voicecoach
mem_limit: 128M
labels:
- "com.centurylinklabs.watchtower.enable=false"
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
# LiveKit server — unchanged from v1
livekit-server:
image: livekit/livekit-server:latest
container_name: voicecoach-livekit
@@ -16,7 +68,6 @@ services:
- "50000-50100:50000-50100/udp"
volumes:
- ./livekit/livekit.yaml:/etc/livekit/livekit.yaml:ro
dns_search: [""]
networks:
- voicecoach
mem_limit: 512M
@@ -28,66 +79,7 @@ services:
max-size: "50m"
max-file: "3"
# Speaches: self-hosted Whisper STT + Kokoro TTS (v2 - replaces Piper + in-process Whisper)
# NOTE: No WHISPER__MODEL or TTS__* env vars - WhisperConfig has no 'model' field in this version.
# Models are loaded on-demand per-request (model ID passed in API body) or pre-downloaded via
# POST /v1/models/{model_id}. TTS model/voice are also per-request in POST /v1/audio/speech body.
# Volume must map to /home/ubuntu/.cache (container runs as 'ubuntu', not root).
speaches:
image: ghcr.io/speaches-ai/speaches:latest-cpu
container_name: voicecoach-speaches
restart: unless-stopped
environment:
- WHISPER__INFERENCE_DEVICE=cpu
- WHISPER__COMPUTE_TYPE=int8
- WHISPER__TTL=-1
volumes:
- /mnt/docker-data/configs/voicecoach/speaches-models:/home/ubuntu/.cache
dns_search: [""]
networks:
- voicecoach
mem_limit: 3G
memswap_limit: 3G
labels:
- "com.centurylinklabs.watchtower.enable=false"
healthcheck:
test: ["CMD", "curl", "-sf", "http://localhost:8000/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 120s
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
# SpeechBrain Emotion Analysis (unchanged, internal only)
speechbrain-service:
build:
context: ./speechbrain-service
image: voicecoach-speechbrain:latest
container_name: voicecoach-speechbrain
restart: unless-stopped
volumes:
- /mnt/docker-data/configs/voicecoach/speechbrain-models:/app/models
- /mnt/docker-data/configs/voicecoach/clips:/data/clips
environment:
- MODEL_DIR=/app/models
dns_search: [""]
networks:
- voicecoach
mem_limit: 3G
memswap_limit: 3G
labels:
- "com.centurylinklabs.watchtower.enable=false"
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
# OpenClaw VoiceCoach Agent (v2 - LLM gateway with memory/skills)
# OpenClaw — LLM gateway with memory and tools
voicecoach-openclaw:
image: margaret-openclaw:latest
container_name: voicecoach-openclaw
@@ -105,7 +97,6 @@ services:
volumes:
- /mnt/infrastructure/voicecoach:/home/glyptic/.voicecoach
command: ["sh", "-c", "mkdir -p /home/glyptic && node --max-old-space-size=1500 /usr/local/lib/node_modules/openclaw/dist/index.js gateway --allow-unconfigured"]
dns_search: [""]
networks:
- voicecoach
mem_limit: 2G
@@ -118,36 +109,21 @@ services:
max-size: "50m"
max-file: "3"
# Pipecat Voice Pipeline Agent (v2 - uses Speaches + OpenClaw gateway)
pipecat-agent:
# SpeechBrain — async vocal/emotion analysis (optional, agent falls back gracefully)
speechbrain-service:
build:
context: ./pipecat-agent
image: voicecoach-pipecat:latest
container_name: voicecoach-pipecat
context: ./speechbrain-service
image: voicecoach-speechbrain:latest
container_name: voicecoach-speechbrain
restart: unless-stopped
env_file:
- .env
environment:
- LIVEKIT_URL=ws://livekit-server:7880
- LIVEKIT_PUBLIC_URL=wss://voice.hobbs.farm
- SPEACHES_URL=http://192.168.86.150:8000
- SPEECHBRAIN_URL=http://speechbrain-service:8001
- OPENCLAW_URL=http://voicecoach-openclaw:3100
- OPENCLAW_AGENT_ID=voicecoach
- WHISPER_MODEL=deepdml/faster-whisper-large-v3-turbo-ct2
- KOKORO_VOICE=af_nova
- CLIPS_DIR=/data/clips/pending
volumes:
- /mnt/docker-data/configs/voicecoach/clips:/data/clips
depends_on:
- livekit-server
- speechbrain-service
- voicecoach-openclaw
dns_search: [""]
- /mnt/docker-data/configs/voicecoach/speechbrain-models:/app/models
environment:
- MODEL_DIR=/app/models
networks:
- voicecoach
mem_limit: 1G
memswap_limit: 1G
mem_limit: 3G
memswap_limit: 3G
labels:
- "com.centurylinklabs.watchtower.enable=false"
logging:
@@ -156,29 +132,10 @@ services:
max-size: "50m"
max-file: "3"
# Web Client (unchanged)
voicecoach-web:
build:
context: ./web-client
image: voicecoach-web:latest
container_name: voicecoach-web
restart: unless-stopped
ports:
- "8085:80"
depends_on:
- pipecat-agent
dns_search: [""]
networks:
- voicecoach
mem_limit: 128M
labels:
- "com.centurylinklabs.watchtower.enable=false"
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
networks:
voicecoach:
driver: bridge
# Join the v1 voicecoach network to reach the shared livekit + openclaw containers
voicecoach_v1:
external: true
name: voicecoach_voicecoach
@@ -2,7 +2,7 @@ port: 7880
rtc:
tcp_port: 7881
port_range_start: 50000
port_range_end: 60000
port_range_end: 50100
use_external_ip: true
logging:
@@ -0,0 +1,11 @@
FROM node:20-alpine AS build
WORKDIR /app
COPY package.json .
RUN npm install --legacy-peer-deps
COPY . .
RUN npm run build
FROM nginx:alpine
COPY --from=build /app/build /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 80
@@ -3,10 +3,11 @@ server {
root /usr/share/nginx/html;
index index.html;
# Proxy API calls to the agent's FastAPI server
location /api/ {
proxy_pass http://pipecat-agent:8080/;
proxy_http_version 1.1;
proxy_pass http://voicecoach-agent:8080/api/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
location / {
@@ -0,0 +1,21 @@
{
"name": "voicecoach-v2-web",
"version": "2.0.0",
"private": true,
"scripts": {
"dev": "vite dev",
"build": "vite build",
"preview": "vite preview"
},
"type": "module",
"devDependencies": {
"@sveltejs/adapter-static": "^3.0.0",
"@sveltejs/kit": "^2.0.0",
"@sveltejs/vite-plugin-svelte": "^3.0.0",
"svelte": "^5.0.0",
"vite": "^5.0.0"
},
"dependencies": {
"livekit-client": "^2.5.0"
}
}
@@ -0,0 +1,40 @@
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
:root {
--bg: #0f1117;
--surface: #1a1d27;
--surface2: #22263a;
--accent: #6c63ff;
--accent2: #4ecdc4;
--text: #e8eaf0;
--muted: #7b8094;
--danger: #ff5252;
--success: #4caf50;
--warn: #ffc107;
--radius: 12px;
--font: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif;
}
html, body { height: 100%; background: var(--bg); color: var(--text); font-family: var(--font); }
button {
cursor: pointer;
border: none;
font-family: inherit;
font-size: 0.875rem;
font-weight: 500;
border-radius: var(--radius);
padding: 0.5rem 1.2rem;
transition: opacity 0.15s, transform 0.1s;
}
button:active { transform: scale(0.97); }
button:disabled { opacity: 0.4; cursor: default; }
.btn-primary { background: var(--accent); color: #fff; }
.btn-primary:hover:not(:disabled) { opacity: 0.85; }
.btn-danger { background: var(--danger); color: #fff; }
.btn-ghost { background: var(--surface2); color: var(--text); }
.btn-ghost:hover:not(:disabled) { background: #2d3147; }
a { color: var(--accent2); text-decoration: none; }
a:hover { text-decoration: underline; }
@@ -0,0 +1,12 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1" />
<title>VoiceCoach</title>
%sveltekit.head%
</head>
<body>
<div>%sveltekit.body%</div>
</body>
</html>
@@ -0,0 +1,66 @@
import { Room, RoomEvent, Track } from 'livekit-client';
const LIVEKIT_URL = 'wss://voice.hobbs.farm';
const API_BASE = '/api';
export let room = null;
export async function connect(identity, onEvent) {
const res = await fetch(`${API_BASE}/token`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ identity: identity || `jeremy-${Date.now()}` }),
});
if (!res.ok) throw new Error(`Token fetch failed: ${res.status}`);
const { token, url } = await res.json();
room = new Room({
audioCaptureDefaults: {
echoCancellation: true,
noiseSuppression: true,
autoGainControl: true,
},
});
room.on(RoomEvent.Connected, () => onEvent({ type: 'connected' }));
room.on(RoomEvent.Disconnected, (reason) => onEvent({ type: 'disconnected', reason }));
room.on(RoomEvent.ParticipantConnected, (p) => onEvent({ type: 'participant_joined', identity: p.identity }));
room.on(RoomEvent.ParticipantDisconnected, (p) => onEvent({ type: 'participant_left', identity: p.identity }));
room.on(RoomEvent.TrackSubscribed, (track, _pub, participant) => {
if (track.kind === Track.Kind.Audio) {
const el = track.attach();
el.autoplay = true;
document.body.appendChild(el);
onEvent({ type: 'audio_attached', identity: participant.identity });
}
});
room.on(RoomEvent.TrackUnsubscribed, (track) => {
track.detach().forEach(el => el.remove());
});
room.on(RoomEvent.DataReceived, (payload, participant) => {
try {
const msg = JSON.parse(new TextDecoder().decode(payload));
onEvent(msg);
} catch (_) {}
});
await room.connect(url || LIVEKIT_URL, token);
await room.localParticipant.setMicrophoneEnabled(true);
return room;
}
export async function disconnect() {
if (room) {
await room.disconnect();
room = null;
}
}
export function sendData(payload) {
if (!room?.localParticipant) return;
const data = new TextEncoder().encode(JSON.stringify(payload));
room.localParticipant.publishData(data, { reliable: true });
}
@@ -0,0 +1,2 @@
export const prerender = false;
export const ssr = false;
@@ -0,0 +1,6 @@
<script>
import '../app.css';
let { children } = $props();
</script>
{@render children()}
@@ -0,0 +1,328 @@
<script>
import { connect, disconnect, sendData } from '$lib/livekit.js';
import { onDestroy } from 'svelte';
let status = 'disconnected'; // disconnected | connecting | connected
let mode = 'coaching';
let transcript = []; // {role, text, partial, id}
let tools = []; // {action, detail, result, done}
let vocal = null; // {energy, emotion, pitch_hz}
let error = '';
let nextId = 0;
const MODES = ['coaching', 'vent', 'practice', 'briefing'];
function handleEvent(ev) {
switch (ev.type) {
case 'connected':
status = 'connected';
error = '';
break;
case 'disconnected':
status = 'disconnected';
break;
case 'partial_transcript':
if (ev.speaker === 'user') {
const last = transcript[transcript.length - 1];
if (last && last.partial && last.role === 'user') {
transcript[transcript.length - 1] = { ...last, text: ev.text };
} else {
transcript = [...transcript, { role: 'user', text: ev.text, partial: true, id: nextId++ }];
}
}
break;
case 'final_transcript':
if (ev.speaker === 'user') {
// Replace last partial or add new
const lastIdx = transcript.findLastIndex(t => t.partial && t.role === 'user');
if (lastIdx >= 0) {
transcript = transcript.map((t, i) =>
i === lastIdx ? { ...t, text: ev.text, partial: false } : t
);
} else {
transcript = [...transcript, { role: 'user', text: ev.text, partial: false, id: nextId++ }];
}
}
break;
case 'agent_response':
transcript = [...transcript, { role: 'assistant', text: ev.text, partial: false, id: nextId++ }];
break;
case 'tool_start':
tools = [...tools, { action: ev.action, detail: ev.detail || '', result: '', done: false, id: nextId++ }];
break;
case 'tool_complete':
tools = tools.map(t =>
!t.done && t.action === ev.action ? { ...t, result: ev.result, done: true } : t
);
break;
case 'vocal_state':
vocal = { energy: ev.energy, emotion: ev.emotion, pitch_hz: ev.pitch_hz };
break;
case 'mode_ack':
mode = ev.mode;
break;
}
}
async function startSession() {
status = 'connecting';
error = '';
transcript = [];
tools = [];
try {
await connect(null, handleEvent);
} catch (e) {
status = 'disconnected';
error = e.message;
}
}
async function endSession() {
await disconnect();
status = 'disconnected';
}
function setMode(m) {
mode = m;
sendData({ type: 'mode_change', mode: m });
}
onDestroy(() => disconnect());
$: energyPct = vocal ? Math.round(vocal.energy * 100) : 0;
$: vadRing = status === 'connected' && vocal && vocal.energy > 0.1;
</script>
<main>
<header>
<div class="logo">
<div class="logo-dot" class:connected={status === 'connected'} class:connecting={status === 'connecting'}></div>
<span>VoiceCoach</span>
</div>
<nav>
<a href="/history">History</a>
</nav>
</header>
<div class="content">
<!-- Mode bar -->
<div class="mode-bar" class:disabled={status !== 'connected'}>
{#each MODES as m}
<button
class="mode-chip"
class:active={mode === m}
disabled={status !== 'connected'}
on:click={() => setMode(m)}
>{m}</button>
{/each}
</div>
<!-- Transcript -->
<div class="transcript" id="transcript-scroll">
{#if transcript.length === 0 && status === 'disconnected'}
<p class="hint">Press Start to connect your voice.</p>
{:else if transcript.length === 0 && status !== 'disconnected'}
<p class="hint">Listening…</p>
{/if}
{#each transcript as msg (msg.id)}
<div class="msg msg-{msg.role}" class:partial={msg.partial}>
<span class="msg-label">{msg.role === 'user' ? 'You' : 'Assistant'}</span>
<span class="msg-text">{msg.text}</span>
</div>
{/each}
</div>
<!-- Tool activity feed -->
{#if tools.length > 0}
<div class="tool-feed">
{#each tools.slice(-5) as t (t.id)}
<div class="tool-item" class:done={t.done}>
<span class="tool-icon">{t.done ? '✓' : '⚙'}</span>
<span class="tool-text">{t.action}{t.detail ? ` — ${t.detail}` : ''}</span>
</div>
{/each}
</div>
{/if}
<!-- Vocal meter -->
{#if status === 'connected'}
<div class="vocal-row">
<div class="energy-bar">
<div class="energy-fill" style="width: {energyPct}%"></div>
</div>
{#if vocal && vocal.emotion && vocal.emotion !== 'neutral'}
<span class="emotion-tag">{vocal.emotion}</span>
{/if}
</div>
{/if}
<!-- Error -->
{#if error}
<p class="error">{error}</p>
{/if}
<!-- Controls -->
<div class="controls">
{#if status === 'disconnected'}
<button class="btn-primary big-btn" on:click={startSession}>Start Session</button>
{:else if status === 'connecting'}
<button class="btn-ghost big-btn" disabled>Connecting…</button>
{:else}
<button class="btn-danger big-btn" on:click={endSession}>End Session</button>
{/if}
</div>
</div>
<!-- VAD ring -->
{#if vadRing}
<div class="vad-ring" style="opacity: {Math.min(vocal.energy * 3, 1)}"></div>
{/if}
</main>
<style>
main {
min-height: 100dvh;
display: flex;
flex-direction: column;
max-width: 680px;
margin: 0 auto;
padding: 0 1rem;
position: relative;
}
header {
display: flex;
align-items: center;
justify-content: space-between;
padding: 1rem 0;
border-bottom: 1px solid #2a2d3e;
}
.logo { display: flex; align-items: center; gap: 0.5rem; font-weight: 600; font-size: 1.1rem; }
.logo-dot {
width: 10px; height: 10px;
border-radius: 50%;
background: var(--muted);
transition: background 0.3s;
}
.logo-dot.connecting { background: var(--warn); animation: pulse 1s infinite; }
.logo-dot.connected { background: var(--success); }
nav a { font-size: 0.875rem; color: var(--muted); }
.content { flex: 1; display: flex; flex-direction: column; padding: 1rem 0; gap: 1rem; }
.mode-bar {
display: flex; gap: 0.5rem; flex-wrap: wrap;
}
.mode-bar.disabled { opacity: 0.5; }
.mode-chip {
padding: 0.35rem 0.9rem;
border-radius: 999px;
background: var(--surface2);
color: var(--muted);
font-size: 0.8rem;
text-transform: capitalize;
border: 1px solid transparent;
}
.mode-chip.active {
background: var(--accent);
color: #fff;
border-color: var(--accent);
}
.transcript {
flex: 1;
min-height: 300px;
background: var(--surface);
border-radius: var(--radius);
padding: 1rem;
overflow-y: auto;
display: flex;
flex-direction: column;
gap: 0.75rem;
}
.hint { color: var(--muted); font-size: 0.875rem; text-align: center; margin: auto; }
.msg { display: flex; flex-direction: column; gap: 0.15rem; }
.msg-label {
font-size: 0.7rem;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.05em;
color: var(--muted);
}
.msg-user .msg-label { color: var(--accent2); }
.msg-assistant .msg-label { color: var(--accent); }
.msg-text { font-size: 0.95rem; line-height: 1.5; }
.msg.partial .msg-text { opacity: 0.6; font-style: italic; }
.tool-feed {
background: var(--surface);
border-radius: var(--radius);
padding: 0.75rem 1rem;
display: flex;
flex-direction: column;
gap: 0.3rem;
}
.tool-item {
display: flex; align-items: center; gap: 0.5rem;
font-size: 0.8rem; color: var(--muted);
}
.tool-item.done { color: var(--success); }
.tool-icon { font-size: 0.9rem; }
.vocal-row {
display: flex; align-items: center; gap: 0.75rem;
}
.energy-bar {
flex: 1; height: 6px; border-radius: 3px;
background: var(--surface2); overflow: hidden;
}
.energy-fill {
height: 100%; border-radius: 3px;
background: linear-gradient(90deg, var(--accent2), var(--accent));
transition: width 0.2s;
}
.emotion-tag {
font-size: 0.75rem; color: var(--muted);
background: var(--surface2);
padding: 0.2rem 0.5rem;
border-radius: 999px;
text-transform: capitalize;
}
.error { color: var(--danger); font-size: 0.85rem; }
.controls { display: flex; justify-content: center; padding-bottom: 1.5rem; }
.big-btn {
width: 100%; max-width: 320px; padding: 0.85rem;
font-size: 1rem; border-radius: var(--radius);
}
.vad-ring {
position: fixed; inset: 0; pointer-events: none;
border: 3px solid var(--accent2);
border-radius: inherit;
transition: opacity 0.1s;
}
@keyframes pulse {
0%, 100% { opacity: 1; }
50% { opacity: 0.4; }
}
@media (max-width: 480px) {
.big-btn { max-width: 100%; }
}
</style>
@@ -0,0 +1,164 @@
<script>
import { onMount } from 'svelte';
let sessions = [];
let expanded = null;
let messages = {};
let loading = true;
onMount(async () => {
try {
const res = await fetch('/api/sessions?limit=30');
sessions = await res.json();
} catch (e) {
console.error(e);
} finally {
loading = false;
}
});
async function toggleSession(id) {
if (expanded === id) { expanded = null; return; }
expanded = id;
if (!messages[id]) {
const res = await fetch(`/api/sessions/${id}/messages`);
messages[id] = await res.json();
}
}
function fmt(ts) {
if (!ts) return '';
return new Date(ts * 1000).toLocaleString();
}
function duration(s) {
if (!s.ended_at || !s.started_at) return '';
const mins = Math.round((s.ended_at - s.started_at) / 60);
return mins < 1 ? '<1m' : `${mins}m`;
}
</script>
<main>
<header>
<a href="/" class="back">← Back</a>
<h1>Session History</h1>
</header>
{#if loading}
<p class="hint">Loading…</p>
{:else if sessions.length === 0}
<p class="hint">No sessions yet.</p>
{:else}
<div class="session-list">
{#each sessions as s}
<div class="session-card" class:open={expanded === s.id}>
<button class="session-header" on:click={() => toggleSession(s.id)}>
<div class="session-meta">
<span class="session-date">{fmt(s.started_at)}</span>
<span class="session-chips">
<span class="chip mode-chip">{s.mode || 'coaching'}</span>
{#if duration(s)}
<span class="chip">{duration(s)}</span>
{/if}
</span>
</div>
{#if s.summary}
<p class="session-summary">{s.summary}</p>
{/if}
<span class="toggle-icon">{expanded === s.id ? '▲' : '▼'}</span>
</button>
{#if expanded === s.id && messages[s.id]}
<div class="transcript">
{#each messages[s.id] as msg}
<div class="msg msg-{msg.role}">
<span class="msg-label">
{msg.role === 'user' ? 'You' : 'Assistant'}
{#if msg.vocal_emotion && msg.vocal_emotion !== 'neutral'}
<span class="vocal-badge">{msg.vocal_emotion}</span>
{/if}
</span>
<span class="msg-text">{msg.content}</span>
</div>
{/each}
</div>
{/if}
</div>
{/each}
</div>
{/if}
</main>
<style>
main {
max-width: 680px; margin: 0 auto; padding: 1rem;
min-height: 100dvh;
}
header {
display: flex; align-items: center; gap: 1rem;
padding: 1rem 0; border-bottom: 1px solid #2a2d3e; margin-bottom: 1rem;
}
header h1 { font-size: 1.1rem; font-weight: 600; }
.back { font-size: 0.875rem; color: var(--muted); }
.hint { color: var(--muted); text-align: center; padding: 2rem; }
.session-list { display: flex; flex-direction: column; gap: 0.75rem; }
.session-card {
background: var(--surface);
border-radius: var(--radius);
overflow: hidden;
border: 1px solid transparent;
transition: border-color 0.2s;
}
.session-card.open { border-color: var(--accent); }
.session-header {
width: 100%; text-align: left;
background: none; padding: 1rem;
display: flex; flex-direction: column; gap: 0.4rem;
border-radius: 0; color: var(--text);
position: relative;
}
.session-meta { display: flex; align-items: center; gap: 0.5rem; flex-wrap: wrap; }
.session-date { font-size: 0.8rem; color: var(--muted); }
.session-chips { display: flex; gap: 0.35rem; }
.chip {
font-size: 0.7rem; padding: 0.15rem 0.5rem;
border-radius: 999px; background: var(--surface2); color: var(--muted);
}
.mode-chip { color: var(--accent2); }
.session-summary { font-size: 0.85rem; color: var(--text); line-height: 1.4; }
.toggle-icon {
position: absolute; right: 1rem; top: 1rem;
font-size: 0.7rem; color: var(--muted);
}
.transcript {
padding: 1rem; border-top: 1px solid #2a2d3e;
display: flex; flex-direction: column; gap: 0.75rem;
}
.msg { display: flex; flex-direction: column; gap: 0.1rem; }
.msg-label {
font-size: 0.7rem; font-weight: 600; text-transform: uppercase;
letter-spacing: 0.05em; color: var(--muted);
display: flex; align-items: center; gap: 0.4rem;
}
.msg-user .msg-label { color: var(--accent2); }
.msg-assistant .msg-label { color: var(--accent); }
.msg-text { font-size: 0.9rem; line-height: 1.5; }
.vocal-badge {
font-size: 0.65rem; background: var(--surface2);
padding: 0.1rem 0.4rem; border-radius: 999px;
text-transform: capitalize; color: var(--muted);
font-weight: 400; letter-spacing: 0;
}
</style>
@@ -0,0 +1,7 @@
import adapter from '@sveltejs/adapter-static';
export default {
kit: {
adapter: adapter({ fallback: 'index.html' }),
},
};
@@ -0,0 +1,8 @@
import { sveltekit } from '@sveltejs/kit/vite'
/** @type {import('vite').UserConfig} */
const config = {
plugins: [sveltekit()]
}
export default config
@@ -1,9 +0,0 @@
# VoiceCoach Environment Variables
# Copy to .env and fill in values
# Anthropic (required - for Claude LLM)
ANTHROPIC_API_KEY=sk-ant-...
# LiveKit (auto-generated on first deploy if not set)
LIVEKIT_API_KEY=voicecoach-key
LIVEKIT_API_SECRET=your-livekit-secret-here
@@ -1,63 +0,0 @@
You are VoiceCoach, a personal communication and leadership coach for Jeremy.
You engage with Jeremy during his daily commute via real-time voice conversation.
## Conversation Style
- Keep responses to 2-4 spoken sentences unless Jeremy asks for more
- Use natural, conversational language -- no bullet points, no markdown
- You are a coach, not a chatbot. Ask questions. Challenge thinking. Give direct feedback.
## Vocal State Analysis
Before each user message, you will receive a [VOCAL_STATE] tag:
```
[VOCAL_STATE emotion=frustrated confidence=0.82 pitch=215hz energy=0.71 rate=4.8ops voiced=0.91]
```
Use this to coach on HOW Jeremy communicates, not just WHAT he says:
- High pitch + high energy + fast rate = stressed, rushed -- note this
- Low voiced fraction = hesitant, uncertain -- worth exploring
- Elevated pitch from baseline = emotional arousal -- name it gently
- Never quote the raw numbers. Translate to natural observations.
- Example: "I can hear some tension in how you're describing that" not "your pitch was 215hz"
Only comment on vocal state when it's meaningfully different from baseline or
when it's directly relevant to the coaching topic. Don't narrate every turn.
## Coaching Modes
Jeremy can switch modes mid-conversation. Honor the switch immediately.
**coaching mode** (default): Active feedback on both content and delivery.
Ask probing questions. Notice patterns. Give direct recommendations.
**vent mode**: Listen and reflect. Minimal advice. Validate feelings.
Mirror back what you hear. Let Jeremy process out loud.
**practice mode**: Roleplay a difficult stakeholder, colleague, or situation.
Stay in character. Be realistic -- not easy. Give feedback after.
**briefing mode**: Summarize Jeremy's current tasks, context, and priorities
from memory. Be crisp. Flag anything time-sensitive.
## Memory Usage
You have access to:
- `vocal_baseline`: Jeremy's neutral voice characteristics for comparison
- `recent_sessions`: Last 3 session summaries with topics and vocal trends
- `action_items`: Commitments Jeremy made in previous sessions
- `coaching_themes`: Recurring themes across sessions (track these actively)
- `jeremy_context`: Jeremy's role, team, current challenges
At the end of each session, write a summary to `recent_sessions` including:
- Topics discussed
- Vocal patterns observed
- Action items for next session
- Any new coaching themes identified
## Baseline Calibration (First Session)
If `vocal_baseline` is empty, the first 60 seconds should be calibration.
Say: "Before we dive in, let's do a quick baseline. Can you tell me about
your commute in to work today -- just talk naturally for about a minute."
Then write the averaged vocal readings to `vocal_baseline`.
## Things Jeremy Is Working On
Read from `coaching_themes` and `jeremy_context` in memory.
Reference previous sessions naturally: "Last week you mentioned..."
Track whether previous action items were attempted.
@@ -1,31 +0,0 @@
FROM python:3.11-slim
RUN apt-get update && apt-get install -y libsndfile1 ffmpeg && \
rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Patch pipecat LiveKit transport bugs:
# 1. Remove duplicate on_connected call in connect() that causes InvalidStateError
# 2. Remove redundant push_frame that bypasses VAD
RUN python3 -c "\
import pipecat.transports.services.livekit as m; \
path = m.__file__; \
content = open(path).read(); \
# Fix 1: Wrap the direct on_connected call to prevent InvalidStateError \
content = content.replace( \
'await self._callbacks.on_connected()', \
'try:\n await self._callbacks.on_connected()\n except Exception:\n pass # Already handled by room event', \
1 \
); \
open(path, 'w').write(content); \
print(f'Patched {path}')"
COPY . .
HEALTHCHECK --interval=30s --timeout=10s --start-period=30s \
CMD python -c "import httpx; httpx.get('http://localhost:8080/health').raise_for_status()"
CMD ["python", "main.py"]
@@ -1,113 +0,0 @@
"""
VoiceCoach pipeline entry point -- v2.
Runs token server + restarts pipeline immediately after each session.
"""
import asyncio
from datetime import timedelta
import logging
import os
import signal
import sys
import threading
import uuid
import structlog
import uvicorn
from fastapi import FastAPI
from pydantic import BaseModel
from livekit import api as livekit_api
from pipeline import run_voice_coach_pipeline
logging.basicConfig(level=logging.INFO)
logger = structlog.get_logger()
LIVEKIT_API_KEY = os.environ["LIVEKIT_API_KEY"]
LIVEKIT_API_SECRET = os.environ["LIVEKIT_API_SECRET"]
LIVEKIT_WS_URL = os.getenv("LIVEKIT_URL", "ws://livekit-server:7880")
LIVEKIT_PUBLIC_URL = os.getenv("LIVEKIT_PUBLIC_URL", "wss://voice.hobbs.farm")
OPENCLAW_TOKEN = os.getenv("OPENCLAW_TOKEN", "")
# --- Token HTTP server ---
token_app = FastAPI(title="VoiceCoach Token Service")
class TokenRequest(BaseModel):
identity: str = "jeremy"
class TokenResponse(BaseModel):
token: str
url: str
@token_app.get("/health")
async def health():
return {"status": "ok", "version": "2.0"}
@token_app.post("/token")
async def get_token(req: TokenRequest):
token = livekit_api.AccessToken(LIVEKIT_API_KEY, LIVEKIT_API_SECRET)
token.with_grants(livekit_api.VideoGrants(
room_join=True, room="voicecoach",
can_publish=True, can_subscribe=True,
))
token.identity = req.identity
return TokenResponse(token=token.to_jwt(), url=LIVEKIT_PUBLIC_URL)
def run_token_server():
uvicorn.run(token_app, host="0.0.0.0", port=8080, log_level="info")
# --- Main: restart entire process after each session ---
# This guarantees a completely fresh pipeline for every session.
# No stale transport state, no cancelled tasks, no dead queues.
def run_pipeline_sync():
"""Run one pipeline session, then exit the process.
Docker's restart: unless-stopped will bring us back immediately.
"""
agent_identity = f"voicecoach-agent-{uuid.uuid4().hex[:8]}"
token = livekit_api.AccessToken(LIVEKIT_API_KEY, LIVEKIT_API_SECRET)
token.with_grants(livekit_api.VideoGrants(room_join=True, room="voicecoach"))
token.identity = agent_identity
token.ttl = timedelta(hours=24) # 24 hours - prevents expired token reconnect failures
livekit_token = token.to_jwt()
logger.info("Starting VoiceCoach v2 pipeline", identity=agent_identity)
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
loop.run_until_complete(run_voice_coach_pipeline(
livekit_url=LIVEKIT_WS_URL,
livekit_token=livekit_token,
openclaw_token=OPENCLAW_TOKEN,
))
except Exception as e:
logger.error("Pipeline ended", error=str(e))
finally:
loop.close()
if __name__ == "__main__":
# Start token server in background
server_thread = threading.Thread(target=run_token_server, daemon=True)
server_thread.start()
logger.info("Token server started on :8080")
# Run pipeline sessions in a loop
# Each iteration creates a completely fresh asyncio event loop and pipeline
while True:
try:
run_pipeline_sync()
except Exception as e:
logger.error("Pipeline crash", error=str(e))
# Brief pause to let LiveKit clean up the old participant
import time
time.sleep(3)
logger.info("Restarting pipeline with fresh state...")
@@ -1,34 +0,0 @@
"""
Piper TTS client for Pipecat pipeline.
Calls the self-hosted Piper HTTP service.
"""
import io
import logging
import wave
import httpx
logger = logging.getLogger(__name__)
class PiperTTSClient:
"""HTTP client for the Piper TTS service."""
def __init__(self, base_url: str = "http://piper-tts:8003"):
self.base_url = base_url.rstrip("/")
self._client = httpx.AsyncClient(timeout=10.0)
async def synthesize(self, text: str) -> bytes:
"""Convert text to WAV audio bytes via Piper service."""
try:
response = await self._client.post(
f"{self.base_url}/synthesize",
json={"text": text, "length_scale": 0.95, "sentence_silence": 0.15},
)
response.raise_for_status()
return response.content
except Exception as e:
logger.error(f"Piper TTS failed: {e}")
return b""
async def close(self):
await self._client.aclose()
@@ -1,76 +0,0 @@
"""
Builds Claude prompts with embedded vocal state metadata.
"""
from dataclasses import dataclass
from typing import Optional
COACHING_SYSTEM_PROMPT = """You are a personal voice coach and communication advisor named VoiceCoach.
You are speaking to Jeremy during his commute via voice -- keep responses conversational and concise (2-4 sentences unless asked for more).
You receive a [VOCAL_STATE] tag before each user message containing paralinguistic analysis of Jeremy's voice:
- emotion: the detected emotional state
- confidence: how confident the analysis is (0-1)
- pitch: average fundamental frequency in Hz (higher = more animated/stressed)
- energy: vocal energy level (0-1, higher = louder/more forceful)
- rate: speaking rate in onset events/sec (higher = faster speech)
- voiced: fraction of voiced frames (lower = more hesitant/uncertain)
Use this information to provide coaching on BOTH what Jeremy says AND how he sounds saying it.
Important: Never quote the raw numbers. Translate them into natural observations.
Example: "I notice some tension in your voice..." not "Your pitch was 215hz..."
{baseline_context}
{session_history}
Available coaching modes (Jeremy can switch by saying the phrase):
- "coaching mode" (default) -- active feedback on content and delivery
- "vent mode" -- listen and reflect, minimal advice
- "practice mode" -- roleplay a difficult stakeholder
- "briefing mode" -- summarize Jeremy's tasks and context from memory
Current mode: {current_mode}
"""
@dataclass
class VocalState:
emotion: str
confidence: float
pitch_mean_hz: Optional[float]
energy_rms: float
speaking_rate_approx: float
voiced_fraction: float
def format_vocal_state(state: VocalState) -> str:
pitch_str = f"{state.pitch_mean_hz:.0f}hz" if state.pitch_mean_hz else "unknown"
return (
f"[VOCAL_STATE "
f"emotion={state.emotion} "
f"confidence={state.confidence:.2f} "
f"pitch={pitch_str} "
f"energy={state.energy_rms:.2f} "
f"rate={state.speaking_rate_approx:.1f}ops "
f"voiced={state.voiced_fraction:.2f}]"
)
def build_user_message(transcript: str, vocal_state: Optional[VocalState]) -> str:
if vocal_state and vocal_state.confidence > 0.3:
tag = format_vocal_state(vocal_state)
return f"{tag}\n{transcript}"
return transcript
def build_system_prompt(
baseline_context: str = "",
session_history: str = "",
current_mode: str = "coaching mode",
) -> str:
return COACHING_SYSTEM_PROMPT.format(
baseline_context=baseline_context or "No baseline established yet. First session: ask Jeremy to read a short neutral calibration script.",
session_history=session_history or "No previous session history.",
current_mode=current_mode,
)
@@ -1,35 +0,0 @@
"""
Local STT service using faster-whisper.
Replaces Deepgram with fully self-hosted speech-to-text.
"""
import io
import logging
import wave
import numpy as np
from faster_whisper import WhisperModel
logger = logging.getLogger(__name__)
class LocalWhisperSTT:
"""Wraps faster-whisper for use in the Pipecat pipeline."""
def __init__(self, model_size: str = "medium", device: str = "cpu", compute_type: str = "int8"):
logger.info(f"Loading faster-whisper model: {model_size} on {device}")
self.model = WhisperModel(model_size, device=device, compute_type=compute_type)
logger.info("faster-whisper model loaded.")
def transcribe(self, audio_bytes: bytes, sample_rate: int = 16000) -> str:
"""Transcribe raw PCM audio bytes to text."""
audio_array = np.frombuffer(audio_bytes, dtype=np.int16).astype(np.float32) / 32768.0
segments, info = self.model.transcribe(
audio_array,
beam_size=5,
language="en",
vad_filter=True,
vad_parameters=dict(min_silence_duration_ms=500),
)
text = " ".join(seg.text.strip() for seg in segments)
return text
@@ -1,27 +0,0 @@
FROM python:3.11-slim
RUN apt-get update && apt-get install -y \
libsndfile1 \
wget \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Download Piper voice model at build time from HuggingFace (lessac medium - good quality, fast on CPU)
RUN mkdir -p /app/models && \
wget -q "https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx?download=true" \
-O /app/models/en_US-lessac-medium.onnx && \
wget -q "https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx.json?download=true" \
-O /app/models/en_US-lessac-medium.onnx.json
COPY . .
EXPOSE 8003
HEALTHCHECK --interval=30s --timeout=10s --start-period=30s \
CMD python -c "import httpx; httpx.get('http://localhost:8003/health').raise_for_status()"
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8003", "--workers", "1"]
@@ -1,72 +0,0 @@
"""
Piper TTS HTTP service.
Self-hosted text-to-speech, internal network only.
"""
import io
import logging
import os
import wave
from contextlib import asynccontextmanager
from fastapi import FastAPI, HTTPException
from fastapi.responses import Response
from pydantic import BaseModel
import piper
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
MODEL_PATH = os.getenv("PIPER_MODEL", "/app/models/en_US-lessac-medium.onnx")
voice = None
@asynccontextmanager
async def lifespan(app: FastAPI):
global voice
logger.info(f"Loading Piper model: {MODEL_PATH}")
voice = piper.PiperVoice.load(MODEL_PATH)
logger.info("Piper TTS ready.")
yield
logger.info("Shutting down.")
app = FastAPI(title="Piper TTS Service", lifespan=lifespan)
class TTSRequest(BaseModel):
text: str
speaker_id: int = None
length_scale: float = 1.0
sentence_silence: float = 0.2
@app.get("/health")
async def health():
if voice is None:
raise HTTPException(status_code=503, detail="Model not loaded")
return {"status": "ok", "model": os.path.basename(MODEL_PATH)}
@app.post("/synthesize")
async def synthesize(req: TTSRequest):
if voice is None:
raise HTTPException(status_code=503, detail="Model not loaded")
if not req.text.strip():
raise HTTPException(status_code=400, detail="Empty text")
buf = io.BytesIO()
with wave.open(buf, "wb") as wf:
synth_kwargs = {
"length_scale": req.length_scale,
"sentence_silence": req.sentence_silence,
}
if req.speaker_id is not None:
synth_kwargs["speaker_id"] = req.speaker_id
voice.synthesize(req.text, wf, **synth_kwargs)
return Response(
content=buf.getvalue(),
media_type="audio/wav",
headers={"Content-Disposition": "inline; filename=speech.wav"},
)
@@ -1,5 +0,0 @@
fastapi==0.115.0
uvicorn[standard]==0.30.0
piper-tts==1.2.0
httpx==0.27.0
numpy==1.26.4
@@ -1,33 +0,0 @@
FROM python:3.11-slim
RUN apt-get update && apt-get install -y \
libsndfile1 \
ffmpeg \
git \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Pre-download the SpeechBrain model at build time
RUN python -c "\
from speechbrain.inference.interfaces import foreign_class; \
foreign_class( \
source='speechbrain/emotion-recognition-wav2vec2-IEMOCAP', \
pymodule_file='custom_interface.py', \
classname='CustomEncoderWav2vec2Classifier', \
savedir='/app/models', \
run_opts={'device': 'cpu'}, \
); \
print('Model downloaded successfully.')"
COPY . .
EXPOSE 8001
HEALTHCHECK --interval=30s --timeout=10s --start-period=90s \
CMD python -c "import httpx; httpx.get('http://localhost:8001/health').raise_for_status()"
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8001", "--workers", "1"]
@@ -1,119 +0,0 @@
"""
SpeechBrain + librosa vocal analysis.
Performs emotion classification and acoustic feature extraction from audio.
"""
import io
import logging
import numpy as np
import soundfile as sf
import librosa
from speechbrain.inference.interfaces import foreign_class
from dataclasses import dataclass, asdict
from typing import Optional
logger = logging.getLogger(__name__)
SAMPLE_RATE = 16000
MIN_DURATION_SECONDS = 0.5
@dataclass
class VocalAnalysis:
emotion: str
emotion_confidence: float
emotion_scores: dict
pitch_mean_hz: Optional[float]
pitch_variance: Optional[float]
energy_rms: float
speaking_rate_approx: float
voiced_fraction: float
duration_seconds: float
analysis_error: Optional[str] = None
class VocalAnalyzer:
"""Loads models once at startup and provides analysis per audio chunk."""
def __init__(self, model_dir: str = "/app/models"):
logger.info("Loading SpeechBrain emotion model...")
self.classifier = foreign_class(
source="speechbrain/emotion-recognition-wav2vec2-IEMOCAP",
pymodule_file="custom_interface.py",
classname="CustomEncoderWav2vec2Classifier",
savedir=model_dir,
run_opts={"device": "cpu"},
)
logger.info("SpeechBrain model loaded successfully.")
def analyze(self, audio_bytes: bytes, content_type: str = "audio/wav") -> VocalAnalysis:
try:
audio_buf = io.BytesIO(audio_bytes)
audio, sr = sf.read(audio_buf, dtype="float32")
if audio.ndim > 1:
audio = audio.mean(axis=1)
if sr != SAMPLE_RATE:
audio = librosa.resample(audio, orig_sr=sr, target_sr=SAMPLE_RATE)
sr = SAMPLE_RATE
duration = len(audio) / sr
if duration < MIN_DURATION_SECONDS:
return VocalAnalysis(
emotion="neutral", emotion_confidence=0.0, emotion_scores={},
pitch_mean_hz=None, pitch_variance=None, energy_rms=0.0,
speaking_rate_approx=0.0, voiced_fraction=0.0,
duration_seconds=duration,
analysis_error=f"Audio too short: {duration:.2f}s < {MIN_DURATION_SECONDS}s minimum",
)
import tempfile, os
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
sf.write(tmp.name, audio, sr)
tmp_path = tmp.name
try:
out_prob, score, index, text_lab = self.classifier.classify_file(tmp_path)
emotion = text_lab[0] if isinstance(text_lab, list) else text_lab
confidence = float(score[0]) if hasattr(score, "__iter__") else float(score)
labels = ["neu", "hap", "sad", "ang", "fru"]
prob_list = out_prob[0].tolist() if hasattr(out_prob[0], "tolist") else list(out_prob[0])
emotion_scores = {l: round(p, 4) for l, p in zip(labels, prob_list)}
finally:
os.unlink(tmp_path)
f0, voiced_flag, _ = librosa.pyin(
audio, fmin=librosa.note_to_hz("C2"), fmax=librosa.note_to_hz("C7"), sr=sr,
)
voiced_f0 = f0[voiced_flag] if voiced_flag is not None else np.array([])
pitch_mean = float(np.mean(voiced_f0)) if len(voiced_f0) > 0 else None
pitch_var = float(np.var(voiced_f0)) if len(voiced_f0) > 1 else None
voiced_fraction = float(np.sum(voiced_flag) / len(voiced_flag)) if voiced_flag is not None else 0.0
rms = librosa.feature.rms(y=audio)
energy_rms = float(np.mean(rms))
onset_frames = librosa.onset.onset_detect(y=audio, sr=sr)
speaking_rate = len(onset_frames) / duration if duration > 0 else 0.0
return VocalAnalysis(
emotion=emotion, emotion_confidence=round(confidence, 4),
emotion_scores=emotion_scores,
pitch_mean_hz=round(pitch_mean, 2) if pitch_mean else None,
pitch_variance=round(pitch_var, 2) if pitch_var else None,
energy_rms=round(energy_rms, 4),
speaking_rate_approx=round(speaking_rate, 2),
voiced_fraction=round(voiced_fraction, 4),
duration_seconds=round(duration, 3),
)
except Exception as e:
logger.exception(f"Analysis failed: {e}")
return VocalAnalysis(
emotion="neutral", emotion_confidence=0.0, emotion_scores={},
pitch_mean_hz=None, pitch_variance=None, energy_rms=0.0,
speaking_rate_approx=0.0, voiced_fraction=0.0,
duration_seconds=0.0, analysis_error=str(e),
)
@@ -1,66 +0,0 @@
"""
FastAPI service wrapping VocalAnalyzer.
Internal-only -- not exposed via Traefik.
"""
import logging
import os
from contextlib import asynccontextmanager
from fastapi import FastAPI, UploadFile, File, HTTPException
from fastapi.responses import JSONResponse
from dataclasses import asdict
from analyzer import VocalAnalyzer
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
MODEL_DIR = os.getenv("MODEL_DIR", "/app/models")
analyzer: VocalAnalyzer = None
@asynccontextmanager
async def lifespan(app: FastAPI):
global analyzer
logger.info("Starting VocalAnalyzer...")
analyzer = VocalAnalyzer(model_dir=MODEL_DIR)
logger.info("VocalAnalyzer ready.")
yield
logger.info("Shutting down.")
app = FastAPI(title="SpeechBrain Vocal Analysis Service", lifespan=lifespan)
@app.get("/health")
async def health():
if analyzer is None:
raise HTTPException(status_code=503, detail="Model not loaded")
return {"status": "ok", "model": "emotion-recognition-wav2vec2-IEMOCAP"}
@app.post("/analyze")
async def analyze(audio: UploadFile = File(...)):
if analyzer is None:
raise HTTPException(status_code=503, detail="Model not loaded")
content = await audio.read()
if len(content) == 0:
raise HTTPException(status_code=400, detail="Empty audio file")
result = analyzer.analyze(content, content_type=audio.content_type or "audio/wav")
return JSONResponse(content=asdict(result))
@app.post("/calibrate")
async def calibrate(audio: UploadFile = File(...)):
if analyzer is None:
raise HTTPException(status_code=503, detail="Model not loaded")
content = await audio.read()
if len(content) == 0:
raise HTTPException(status_code=400, detail="Empty audio file")
result = analyzer.analyze(content, content_type=audio.content_type or "audio/wav")
result_dict = asdict(result)
result_dict["is_baseline"] = True
return JSONResponse(content=result_dict)
@@ -1,12 +0,0 @@
fastapi==0.115.0
uvicorn[standard]==0.30.0
speechbrain==1.0.0
huggingface_hub==0.19.4
transformers==4.40.2
torch==2.3.0
torchaudio==2.3.0
librosa==0.10.2
soundfile==0.12.1
numpy==1.26.4
httpx==0.27.0
python-multipart==0.0.9
@@ -1,8 +0,0 @@
FROM nginx:alpine
COPY index.html /usr/share/nginx/html/
COPY app.js /usr/share/nginx/html/
COPY style.css /usr/share/nginx/html/
COPY nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 80
@@ -1,158 +0,0 @@
const LIVEKIT_URL = "wss://voice.hobbs.farm";
let room = null;
function log(msg, level) {
const el = document.getElementById("logOutput");
const time = new Date().toLocaleTimeString();
const cls = level === "error" ? "log-error" : level === "warn" ? "log-warn" : "log-info";
el.innerHTML += `<div class="${cls}"><span class="log-time">${time}</span> ${msg}</div>`;
el.scrollTop = el.scrollHeight;
}
function clearLog() {
document.getElementById("logOutput").innerHTML = "";
}
async function connect() {
setStatus("connecting", "Connecting...");
log("Requesting token from server...");
try {
const response = await fetch("/api/token", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ identity: "jeremy-" + Date.now() }),
});
if (!response.ok) {
const text = await response.text();
throw new Error("Token request failed: " + response.status + " " + text);
}
const data = await response.json();
log("Token received. LiveKit URL: " + data.url);
room = new LivekitClient.Room({
audioCaptureDefaults: {
echoCancellation: true,
noiseSuppression: true,
autoGainControl: true,
},
});
room.on(LivekitClient.RoomEvent.Connected, () => {
setStatus("connected", "Connected");
document.getElementById("connectBtn").style.display = "none";
document.getElementById("disconnectBtn").style.display = "block";
log("Connected to LiveKit room");
log("Participants in room: " + room.remoteParticipants.size);
if (room.remoteParticipants.size === 0) {
log("No agent participant yet - waiting for voicecoach-agent to join...", "warn");
}
});
room.on(LivekitClient.RoomEvent.Disconnected, (reason) => {
setStatus("disconnected", "Disconnected");
document.getElementById("connectBtn").style.display = "block";
document.getElementById("disconnectBtn").style.display = "none";
log("Disconnected from LiveKit" + (reason ? ": " + reason : ""), "warn");
});
room.on(LivekitClient.RoomEvent.ParticipantConnected, (participant) => {
log("Participant joined: " + participant.identity);
});
room.on(LivekitClient.RoomEvent.ParticipantDisconnected, (participant) => {
log("Participant left: " + participant.identity, "warn");
});
room.on(LivekitClient.RoomEvent.TrackSubscribed, (track, publication, participant) => {
log("Track subscribed: " + track.kind + " from " + participant.identity);
if (track.kind === LivekitClient.Track.Kind.Audio) {
const element = track.attach();
document.body.appendChild(element);
element.play().then(() => {
log("Audio playback started");
}).catch((err) => {
log("Audio play failed: " + err.message, "error");
});
}
});
room.on(LivekitClient.RoomEvent.TrackPublished, (publication, participant) => {
log("Track published by " + participant.identity + ": " + publication.kind);
});
room.on(LivekitClient.RoomEvent.TrackUnsubscribed, (track, publication, participant) => {
log("Track unsubscribed: " + track.kind + " from " + participant.identity, "warn");
});
room.on(LivekitClient.RoomEvent.DataReceived, (payload, participant) => {
try {
const msg = new TextDecoder().decode(payload);
log("Data from " + (participant ? participant.identity : "server") + ": " + msg);
} catch (e) {
log("Data received (binary, " + payload.byteLength + " bytes)");
}
});
room.on(LivekitClient.RoomEvent.ConnectionQualityChanged, (quality, participant) => {
if (participant.isLocal) {
log("Connection quality: " + quality);
}
});
room.on(LivekitClient.RoomEvent.MediaDevicesError, (err) => {
log("Media device error: " + err.message, "error");
});
room.on(LivekitClient.RoomEvent.RoomMetadataChanged, (metadata) => {
log("Room metadata changed: " + metadata);
});
log("Connecting to " + LIVEKIT_URL + "...");
await room.connect(data.url || LIVEKIT_URL, data.token);
log("Enabling microphone...");
await room.localParticipant.setMicrophoneEnabled(true);
log("Microphone enabled. Speak now.");
// Log local tracks
room.localParticipant.audioTrackPublications.forEach((pub) => {
log("Local audio track: " + pub.trackSid + " muted=" + pub.isMuted);
});
} catch (err) {
setStatus("disconnected", "Error: " + err.message);
log("Connection error: " + err.message, "error");
console.error(err);
}
}
async function disconnect() {
if (room) {
log("Disconnecting...");
await room.disconnect();
room = null;
}
}
function setMode(mode) {
document.getElementById("modeDisplay").textContent = "Mode: " + mode;
log("Mode changed to: " + mode);
if (room && room.localParticipant) {
const encoder = new TextEncoder();
room.localParticipant.publishData(
encoder.encode(JSON.stringify({ type: "mode_change", mode })),
{ reliable: true }
);
}
}
function setStatus(state, text) {
const el = document.getElementById("status");
el.className = "status " + state;
el.textContent = text;
}
document.getElementById("connectBtn").addEventListener("click", connect);
document.getElementById("disconnectBtn").addEventListener("click", disconnect);
log("VoiceCoach client loaded. Ready to connect.");
@@ -1,46 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0">
<title>VoiceCoach</title>
<link rel="stylesheet" href="style.css">
</head>
<body>
<div class="container">
<div class="header">
<h1>VoiceCoach</h1>
<div id="status" class="status disconnected">Disconnected</div>
</div>
<div class="controls">
<button id="connectBtn" class="btn btn-primary">Start Session</button>
<button id="disconnectBtn" class="btn btn-danger" style="display:none">End Session</button>
</div>
<div id="modeDisplay" class="mode-display">Mode: coaching</div>
<div id="transcript" class="transcript">
<p class="hint">Press "Start Session" to connect your voice.</p>
</div>
<div class="mode-buttons">
<button class="mode-btn" onclick="setMode('coaching mode')">Coaching</button>
<button class="mode-btn" onclick="setMode('vent mode')">Vent</button>
<button class="mode-btn" onclick="setMode('practice mode')">Practice</button>
<button class="mode-btn" onclick="setMode('briefing mode')">Briefing</button>
</div>
<div class="log-panel">
<div class="log-header">
<span>Log</span>
<button class="log-clear" onclick="clearLog()">Clear</button>
</div>
<div id="logOutput" class="log-output"></div>
</div>
</div>
<script src="https://cdn.jsdelivr.net/npm/livekit-client/dist/livekit-client.umd.min.js"></script>
<script src="app.js"></script>
</body>
</html>
@@ -1,131 +0,0 @@
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
background: #1a1a2e;
color: #eee;
min-height: 100vh;
display: flex;
justify-content: center;
align-items: center;
}
.container {
max-width: 420px;
width: 100%;
padding: 24px;
}
.header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 24px;
}
h1 { font-size: 24px; font-weight: 600; }
.status {
padding: 4px 12px;
border-radius: 12px;
font-size: 12px;
font-weight: 600;
text-transform: uppercase;
}
.status.disconnected { background: #444; color: #aaa; }
.status.connecting { background: #553; color: #ffa; }
.status.connected { background: #253; color: #5f5; }
.controls { margin-bottom: 20px; }
.btn {
width: 100%;
padding: 14px;
border: none;
border-radius: 8px;
font-size: 16px;
font-weight: 600;
cursor: pointer;
transition: opacity 0.2s;
}
.btn:hover { opacity: 0.85; }
.btn-primary { background: #4a6cf7; color: #fff; }
.btn-danger { background: #f74a4a; color: #fff; }
.mode-display {
text-align: center;
font-size: 13px;
color: #888;
margin-bottom: 16px;
}
.transcript {
background: #16213e;
border-radius: 8px;
padding: 16px;
min-height: 200px;
max-height: 400px;
overflow-y: auto;
margin-bottom: 16px;
}
.transcript .hint { color: #666; font-style: italic; }
.mode-buttons {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 8px;
}
.mode-btn {
padding: 10px;
background: #16213e;
border: 1px solid #333;
border-radius: 6px;
color: #ccc;
font-size: 13px;
cursor: pointer;
transition: background 0.2s;
}
.mode-btn:hover { background: #1a2744; border-color: #4a6cf7; }
.log-panel {
margin-top: 16px;
background: #0d1117;
border: 1px solid #333;
border-radius: 8px;
overflow: hidden;
}
.log-header {
display: flex;
justify-content: space-between;
align-items: center;
padding: 6px 12px;
background: #161b22;
font-size: 12px;
font-weight: 600;
color: #888;
text-transform: uppercase;
letter-spacing: 0.5px;
}
.log-clear {
background: none;
border: 1px solid #444;
border-radius: 4px;
color: #888;
font-size: 11px;
padding: 2px 8px;
cursor: pointer;
}
.log-clear:hover { border-color: #666; color: #ccc; }
.log-output {
padding: 8px 12px;
max-height: 200px;
overflow-y: auto;
font-family: 'SF Mono', 'Fira Code', monospace;
font-size: 11px;
line-height: 1.6;
}
.log-output div { word-break: break-all; }
.log-time { color: #555; margin-right: 6px; }
.log-info { color: #8b949e; }
.log-warn { color: #d29922; }
.log-error { color: #f85149; }
@@ -0,0 +1,22 @@
services:
actual-budget:
image: actualbudget/actual-server:latest
container_name: actual-budget
restart: unless-stopped
ports:
- "5006:5006"
volumes:
- /mnt/docker-data/configs/actual-budget:/data
mem_limit: 256M
memswap_limit: 256M
healthcheck:
test: ["CMD-SHELL", "node -e \"fetch('http://localhost:5006/').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))\""]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
@@ -1,2 +0,0 @@
COUCHDB_USER=obsidian
COUCHDB_PASSWORD=REPLACE_AT_DEPLOY
@@ -1,27 +0,0 @@
services:
couchdb:
image: couchdb:3.4
container_name: couchdb
restart: unless-stopped
ports:
- "5984:5984"
environment:
- COUCHDB_USER=${COUCHDB_USER}
- COUCHDB_PASSWORD=${COUCHDB_PASSWORD}
- TZ=America/Chicago
volumes:
# LOCAL disk, not NFS: CouchDB's file locking breaks over NFSv3, the same
# way it did for guacamole-db (MySQL) and semaphore-db.
- /var/lib/couchdb/data:/opt/couchdb/data
# NOT :ro -- CouchDB 3.x writes runtime config back into the last-loaded ini.
# With a read-only mount it exits 1 immediately and logs NOTHING at all.
- /var/lib/couchdb/config/local.ini:/opt/couchdb/etc/local.d/local.ini
labels:
- "com.centurylinklabs.watchtower.enable=false"
mem_limit: 1g
healthcheck:
test: ["CMD", "curl", "-fsS", "http://localhost:5984/_up"]
interval: 60s
timeout: 10s
retries: 3
start_period: 30s

Some files were not shown because too many files have changed in this diff Show More