infra: repurpose Spirit as autonomous infrastructure ops agent

Spirit now monitors #notifications Matrix room for Prometheus alerts,
auto-investigates issues, takes corrective action (container restarts,
disk cleanup, etc.), and documents incidents in Wiki.js.

Changes:
- Custom spirit-ops Docker image (Docker CLI + SSH + diagnostic tools)
- Docker socket mount for local container management
- SSH keys for remote access to hf-pdocker-02 and bart
- Wiki.js API integration for incident documentation
- Prometheus probe for Spirit gateway health
- Safety guardrails: protected services, rate limiting, escalation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-03-19 10:36:20 -05:00
parent 59f529e3a7
commit 9943fd4410
5 changed files with 1045 additions and 2 deletions
+3 -2
View File
@@ -30,7 +30,7 @@ hf-pdocker-01 (Manager, 23GB RAM, i7-7700T, NVMe, HD 630 GPU) -- RESTORED (2026-
├── [COMPOSE] Mosquitto (:1883, :8883)
├── [COMPOSE] Synapse/Matrix (:8008) + synapse-db (postgres) + wellknown (:8010)
├── [COMPOSE] Guacamole (:8199) + guacd + guacamole-db (mysql, LOCAL /var/lib/guacamole-db)
├── [COMPOSE] AI Agents (margaret:18790, spirit:18791, clippy:18792, bob:18800)
├── [COMPOSE] AI Agents (margaret:18790, spirit:18795 [infra ops], clippy:18800, bob:18810, travis:18815)
├── [COMPOSE] Traefik (reverse proxy, ports 80/443/8090)
├── [COMPOSE] AdGuard Home (:3000, :53)
├── [COMPOSE] Monitoring (Grafana :3001, Prometheus :9095, Alertmanager :9093, InfluxDB :8086, Blackbox :9115, matrix-receiver :9094)
@@ -170,7 +170,7 @@ compose/ # Standalone Compose definitions (deployed per-node)
- InfluxDB: :8086
- Blackbox Exporter: :9115
- matrix-alertmanager-receiver: :9094
- AI Agents (margaret :18790, spirit :18791, clippy :18792, bob :18800) -- loopback only
- AI Agents (margaret :18790, spirit :18795 [infra ops — monitors #notifications, auto-remediates alerts], clippy :18800, bob :18810, travis :18815) -- LAN-accessible (bind=lan, 0.0.0.0)
- Farm Manager: :8888 (dashboard + API)
- Farm Agent: :8889
@@ -209,6 +209,7 @@ Port conflicts already resolved:
- Port 3000: AdGuard Home (Gitea moved to 3003)
- Port 3001: Grafana (Semaphore moved to 3004)
- Port 5000: Frigate uses host networking (registry container stopped)
- AI Agent ports re-spaced (2026-03-15): openclaw 2026.3.13 uses gateway+2 and gateway+3 for browser/chrome-relay. Agents now spaced 5 apart: margaret=18790, spirit=18795, clippy=18800, bob=18810, travis=18815
## Critical Configuration Preservation