From 6a32d080b5eba2f5261c228bebdb77d35c81a0cb Mon Sep 17 00:00:00 2001 From: jeremy Date: Wed, 27 May 2026 22:01:09 -0500 Subject: [PATCH] Initial commit: LinkedStorm README --- README.md | 70 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..a88a28d --- /dev/null +++ b/README.md @@ -0,0 +1,70 @@ +# LinkedStorm + +Whole-home audio assistant. Deploy browser tabs as microphones throughout the house — LinkedStorm captures voice activity, transcribes speech with Whisper, identifies who is speaking, and maintains a searchable daily log of all household conversations. + +All processing is local. No cloud services required. + +## What It Does + +- **Always-on VAD capture** — Open `capture.html?room=Kitchen` on any phone, tablet, or laptop. Silero VAD detects speech and streams only the active segments (no silence stored). +- **Whisper transcription** — GPU-accelerated Whisper large-v3-turbo via the existing Speaches service. +- **Speaker identification** — resemblyzer voice embeddings + cosine similarity, same pattern as frigate-notify. Review unknowns and tag them once; the system recognizes that voice automatically going forward. +- **Daily log** — Timeline view of all utterances, color-coded by speaker and room. +- **Full-text search** — Search across all transcripts (SQLite FTS5). +- **Ask AI** — Natural language queries against the transcript archive via Ollama (e.g. "What did we decide about the vacation?"). +- **Live monitoring** — Dashboard joins LiveKit rooms to monitor any room or replay clips. +- **Daily summaries** — Ollama generates a plain-English summary of each day's activity at midnight. + +## Container Architecture + +| Container | Port | Purpose | +|---|---|---| +| `linkedstorm-pipeline` | 8300 | WebSocket ingestion, Silero VAD, Whisper STT, speaker embedding, cosine matching, SQLite writes, LiveKit publish | +| `linkedstorm-api` | 8310 | FastAPI dashboard + REST API + static UI | +| `linkedstorm-cron` | — | Nightly Ollama summary, speaker clustering, WAV clip purge | + +## External Dependencies (all pre-deployed in homelab) + +| Service | Host | Purpose | +|---|---|---| +| Speaches/Whisper | 192.168.86.150:8000 | GPU speech-to-text | +| LiveKit | 192.168.86.150:7880 | WebRTC for live monitoring + replay | +| Ollama (qwen2.5:14b) | 192.168.86.150:11434 | RAG queries + daily summaries | + +## Adding a Room + +1. Open `https://linkedstorm.hobbs.farm/capture?room=YourRoomName` on any device with a microphone +2. Allow microphone access +3. The device appears in the Rooms tab of the dashboard within seconds +4. Use the mute toggle on the capture page to pause recording without closing the tab + +## Speaker Labeling + +Unknown utterances appear in the Speakers tab review queue with a Play Clip and Tag Speaker button. Tag a speaker once — their voice embedding is stored and all future utterances from that voice resolve automatically. Mirrors the frigate-notify person-labeling workflow exactly. + +## Storage + +- **Database:** SQLite at `/mnt/docker-data/configs/linkedstorm/linkedstorm.db` +- **Audio clips:** `/mnt/docker-data/configs/linkedstorm/clips/` — 30-day retention (configurable via `CLIP_RETENTION_DAYS`). Transcripts and embeddings are kept permanently. +- **NAS:** All data on Synology NAS via NFS at 192.168.86.30 + +## Dashboard + +`https://linkedstorm.hobbs.farm` — 5-tab interface: +- **Daily Log** — timeline of utterances, filter by room/speaker +- **Search** — full-text search with keyword highlighting +- **Speakers** — known speakers + unknown review queue +- **Ask AI** — RAG chat against transcript archive +- **Rooms** — live capture device status + mute controls + +## Deployment + +```bash +# On hf-pdocker-01 +cd /mnt/docker-data/compose/hf-pdocker-01/linkedstorm +docker compose up -d +``` + +Compose file: `compose/hf-pdocker-01/linkedstorm/docker-compose.yml` in the [docker repo](http://192.168.86.100:3003/jeremy/docker). + +Full design spec: [docs/superpowers/specs/2026-05-27-linkedstorm-design.md](http://192.168.86.100:3003/jeremy/docker/src/branch/master/docs/superpowers/specs/2026-05-27-linkedstorm-design.md)