jeremy
de1d9e1d93
Replaces initial Nomad/Consul/Terraform skeleton with current Docker-based infrastructure across all three nodes (hf-pdocker-01, hf-pdocker-02, bart). Key changes: - CLAUDE.md: updated service distribution reflecting hf-pdocker-01 restored - docs/plans/2026-03-08-hf-pdocker-01-rebuild.md: new rebuild plan doc - compose/bart/: ai-agents, guacamole, matrix, media, openvas, smarthome, watchtower - compose/hf-pdocker-02/: traefik, monitoring, adguard, gitea, semaphore, wikijs, unifi, vaultwarden, actual-budget, mcpo, netclaw, trivy, watchtower, n8n - compose/hf-pdocker-01/: frigate, frigate-notify, farm-agent - stacks/: all Swarm stack definitions (monitoring, traefik, smarthome, matrix, guacamole, media, ai-agents, frigate, gitea, semaphore, wikijs, unifi, etc.) - .gitignore: exclude .env credential files and acme.json certs - .env.example: placeholder credential files for all services - .claude/agents/: gitea-backup and other infrastructure agent configs - scripts/: frigate media cleanup and NAS sync scripts Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
113 lines
3.6 KiB
Markdown
113 lines
3.6 KiB
Markdown
# Frigate Video Alerts via Matrix
|
|
|
|
## Summary
|
|
|
|
A lightweight Python container that listens for Frigate object detection events via MQTT, analyzes snapshots with Claude Vision (Haiku), and posts rich notifications to a dedicated Matrix room with inline images and AI-generated descriptions.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
Frigate NVR (hf-pdocker-01)
|
|
| MQTT: frigate/events
|
|
v
|
|
Mosquitto Broker (:1883)
|
|
|
|
|
v
|
|
frigate-notify (Python container, hf-pdocker-01, host network)
|
|
|-- Frigate API: GET /api/events/{id}/snapshot.jpg
|
|
|-- Claude Vision API: analyze snapshot (Haiku)
|
|
|-- Matrix API: upload image + send message
|
|
v
|
|
#video-alerts:hobbs.farm (@frigate:hobbs.farm bot)
|
|
```
|
|
|
|
## Event Flow & Filtering
|
|
|
|
**MQTT topic:** `frigate/events`
|
|
|
|
**Filtering:**
|
|
1. Only `type == "end"` events (complete detection, best snapshot)
|
|
2. Only `label` in `["person", "car", "dog", "cat", "bird"]`
|
|
3. Only `score >= 0.7`
|
|
4. Only if `has_snapshot == true`
|
|
5. Deduplication: skip duplicate event IDs within 30-second window per camera
|
|
6. Rate limit: max 1 notification per camera per 60 seconds
|
|
|
|
## AI Vision Analysis
|
|
|
|
- Model: `claude-haiku-4-5-20251001` (~$0.001/event, ~$1.50/month at 50 events/day)
|
|
- Prompt: "Briefly describe what's happening in this security camera image in 1-2 sentences. Focus on the people, vehicles, or animals and their activity. Be concise."
|
|
- Max tokens: 150
|
|
- Fallback: if Claude API fails, use structured description ("Person detected on {camera}")
|
|
|
|
## Matrix Message Format
|
|
|
|
Two messages per event:
|
|
|
|
1. **Image message** (`m.image`): snapshot uploaded to Matrix content repository
|
|
2. **Text message** (`m.text` with HTML):
|
|
```
|
|
[camera_emoji] camera_name - Object - score%
|
|
AI-generated description of the scene.
|
|
|
|
Link: http://192.168.86.192:5000/review?id={event_id}
|
|
Timestamp
|
|
```
|
|
|
|
**Bot:** `@frigate:hobbs.farm` (new dedicated account)
|
|
**Room:** `#video-alerts:hobbs.farm` (new room, @jeremy:hobbs.farm invited)
|
|
|
|
## Container & Deployment
|
|
|
|
**Location:** `compose/hf-pdocker-01/frigate-notify/`
|
|
|
|
```
|
|
compose/hf-pdocker-01/frigate-notify/
|
|
docker-compose.yml
|
|
.env
|
|
Dockerfile
|
|
app/
|
|
main.py (~200 lines - MQTT listener, event processor)
|
|
vision.py (~40 lines - Claude Vision API wrapper)
|
|
matrix_client.py (~60 lines - image upload + message send)
|
|
requirements.txt
|
|
```
|
|
|
|
**Container config:**
|
|
- Image: custom, from `python:3.12-slim`
|
|
- Network: host
|
|
- Memory: 256M (memswap_limit: 256M)
|
|
- Stateless (no persistent storage)
|
|
- Health endpoint: port 5199, returns JSON status
|
|
|
|
**Environment variables (.env):**
|
|
- `ANTHROPIC_API_KEY`
|
|
- `MATRIX_ACCESS_TOKEN`
|
|
- `MATRIX_ROOM_ID`
|
|
- `FRIGATE_URL=http://192.168.86.192:5000`
|
|
- `MQTT_HOST=192.168.86.192`
|
|
- `MQTT_PORT=1883`
|
|
|
|
## Error Handling
|
|
|
|
- **Claude API failure:** fall back to structured message, never drop alert
|
|
- **Matrix API failure:** retry 3x with exponential backoff (1s, 2s, 4s)
|
|
- **Snapshot fetch failure:** send text-only notification
|
|
- **MQTT disconnect:** paho-mqtt auto-reconnects
|
|
- **Invalid payload:** log and skip
|
|
|
|
## Observability
|
|
|
|
- Structured JSON logging to stdout
|
|
- Health endpoint at `:5199/health` with MQTT status, event count, last event time
|
|
- Added to Prometheus blackbox probes (http-pdocker01 job)
|
|
- ContainerDown alert covers it automatically
|
|
|
|
## Decisions
|
|
|
|
- **Haiku over Sonnet:** 10x cheaper, fast enough for security descriptions
|
|
- **host network:** Frigate, Mosquitto, Synapse all on LAN IPs, simplest routing
|
|
- **aiohttp over matrix-nio:** lighter, no crypto dependencies needed for bot usage
|
|
- **Two messages (image + text):** image renders natively across all Matrix clients
|
|
- **End events only:** best snapshot quality, avoids duplicate notifications
|