71 lines
3.7 KiB
Markdown
71 lines
3.7 KiB
Markdown
# LinkedStorm
|
|
|
|
Whole-home audio assistant. Deploy browser tabs as microphones throughout the house — LinkedStorm captures voice activity, transcribes speech with Whisper, identifies who is speaking, and maintains a searchable daily log of all household conversations.
|
|
|
|
All processing is local. No cloud services required.
|
|
|
|
## What It Does
|
|
|
|
- **Always-on VAD capture** — Open `capture.html?room=Kitchen` on any phone, tablet, or laptop. Silero VAD detects speech and streams only the active segments (no silence stored).
|
|
- **Whisper transcription** — GPU-accelerated Whisper large-v3-turbo via the existing Speaches service.
|
|
- **Speaker identification** — resemblyzer voice embeddings + cosine similarity, same pattern as frigate-notify. Review unknowns and tag them once; the system recognizes that voice automatically going forward.
|
|
- **Daily log** — Timeline view of all utterances, color-coded by speaker and room.
|
|
- **Full-text search** — Search across all transcripts (SQLite FTS5).
|
|
- **Ask AI** — Natural language queries against the transcript archive via Ollama (e.g. "What did we decide about the vacation?").
|
|
- **Live monitoring** — Dashboard joins LiveKit rooms to monitor any room or replay clips.
|
|
- **Daily summaries** — Ollama generates a plain-English summary of each day's activity at midnight.
|
|
|
|
## Container Architecture
|
|
|
|
| Container | Port | Purpose |
|
|
|---|---|---|
|
|
| `linkedstorm-pipeline` | 8300 | WebSocket ingestion, Silero VAD, Whisper STT, speaker embedding, cosine matching, SQLite writes, LiveKit publish |
|
|
| `linkedstorm-api` | 8310 | FastAPI dashboard + REST API + static UI |
|
|
| `linkedstorm-cron` | — | Nightly Ollama summary, speaker clustering, WAV clip purge |
|
|
|
|
## External Dependencies (all pre-deployed in homelab)
|
|
|
|
| Service | Host | Purpose |
|
|
|---|---|---|
|
|
| Speaches/Whisper | 192.168.86.150:8000 | GPU speech-to-text |
|
|
| LiveKit | 192.168.86.150:7880 | WebRTC for live monitoring + replay |
|
|
| Ollama (qwen2.5:14b) | 192.168.86.150:11434 | RAG queries + daily summaries |
|
|
|
|
## Adding a Room
|
|
|
|
1. Open `https://linkedstorm.hobbs.farm/capture?room=YourRoomName` on any device with a microphone
|
|
2. Allow microphone access
|
|
3. The device appears in the Rooms tab of the dashboard within seconds
|
|
4. Use the mute toggle on the capture page to pause recording without closing the tab
|
|
|
|
## Speaker Labeling
|
|
|
|
Unknown utterances appear in the Speakers tab review queue with a Play Clip and Tag Speaker button. Tag a speaker once — their voice embedding is stored and all future utterances from that voice resolve automatically. Mirrors the frigate-notify person-labeling workflow exactly.
|
|
|
|
## Storage
|
|
|
|
- **Database:** SQLite at `/mnt/docker-data/configs/linkedstorm/linkedstorm.db`
|
|
- **Audio clips:** `/mnt/docker-data/configs/linkedstorm/clips/` — 30-day retention (configurable via `CLIP_RETENTION_DAYS`). Transcripts and embeddings are kept permanently.
|
|
- **NAS:** All data on Synology NAS via NFS at 192.168.86.30
|
|
|
|
## Dashboard
|
|
|
|
`https://linkedstorm.hobbs.farm` — 5-tab interface:
|
|
- **Daily Log** — timeline of utterances, filter by room/speaker
|
|
- **Search** — full-text search with keyword highlighting
|
|
- **Speakers** — known speakers + unknown review queue
|
|
- **Ask AI** — RAG chat against transcript archive
|
|
- **Rooms** — live capture device status + mute controls
|
|
|
|
## Deployment
|
|
|
|
```bash
|
|
# On hf-pdocker-01
|
|
cd /mnt/docker-data/compose/hf-pdocker-01/linkedstorm
|
|
docker compose up -d
|
|
```
|
|
|
|
Compose file: `compose/hf-pdocker-01/linkedstorm/docker-compose.yml` in the [docker repo](http://192.168.86.100:3003/jeremy/docker).
|
|
|
|
Full design spec: [docs/superpowers/specs/2026-05-27-linkedstorm-design.md](http://192.168.86.100:3003/jeremy/docker/src/branch/master/docs/superpowers/specs/2026-05-27-linkedstorm-design.md)
|