chore: decommission clip-service (.192) and Actual Budget (.100)

clip-service: container+volumes+compose removed, ~2GB RAM freed; frigate-notify
config never referenced it. Actual Budget: container+compose removed, data kept
at /mnt/docker-data/configs/actual-budget. Prometheus probes for both removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-13 20:24:29 -05:00
parent ed5f330d2c
commit d35af1ddad
3 changed files with 4 additions and 46 deletions
+4 -5
View File
@@ -23,7 +23,7 @@ This repository manages a **homelab infrastructure** with self-hosted services r
hf-pdocker-01 (23GB RAM, i7-7700T, NVMe, HD 630 GPU) -- RESTORED (2026-03-08, hardware repaired)
├── [COMPOSE] Frigate NVR (:5000) -- Coral USB TPU (edgetpu), ~11ms inference, 13 cameras active
├── [COMPOSE] Frigate-Notify v2 (:5199) -- MQTT->AI->Matrix notifications
├── [COMPOSE] clip-service (:5200) -- CLIP embeddings (ViT-L-14, CPU)
├── [REMOVED] clip-service -- decommissioned 2026-07-13 (CLIP embeddings; ~2GB RAM freed, no config references from frigate-notify existed)
├── [COMPOSE] Home Assistant (:8123)
├── [COMPOSE] ESPHome (:6052)
├── [COMPOSE] Mosquitto (:1883, :8883)
@@ -59,7 +59,8 @@ hf-pdocker-02 (7.6GB RAM, i7-6700T, HDD)
├── [COMPOSE] Gitea + gitea-db (PostgreSQL) -- LevelDB queues on LOCAL /var/lib/gitea-queues (2026-07-13, stale NFS lock caused 7,972-restart loop)
├── [COMPOSE] Semaphore + semaphore-db (MySQL, LOCAL /var/lib/semaphore-db since 2026-07-13)
├── [COMPOSE] Wiki.js + wikijs-db (PostgreSQL)
├── [COMPOSE] Vaultwarden, Actual Budget, MCPO, Trivy
├── [COMPOSE] Vaultwarden, MCPO, Trivy
├── [REMOVED] Actual Budget -- decommissioned 2026-07-13 (data kept at /mnt/docker-data/configs/actual-budget)
├── [COMPOSE] n8n + n8n-db (PostgreSQL) -- workflow automation
├── [COMPOSE] CouchDB (:5984) -- Obsidian LiveSync backend for the Second Brain vault. LAN-only. LOCAL disk (NOT NFS).
├── [COMPOSE] Farm Agent (:8889)
@@ -153,7 +154,6 @@ compose/ # Standalone Compose definitions (deployed per-node)
**hf-pdocker-01 (192.168.86.192):**
- Frigate: :5000
- Frigate Alerts (v2): :5199 (web UI + REST API)
- clip-service: :5200 (CLIP embeddings, ViT-L-14, CPU)
- Home Assistant: :8123
- ESPHome: :6052
- Mosquitto: :1883
@@ -181,7 +181,6 @@ compose/ # Standalone Compose definitions (deployed per-node)
- Semaphore: :3004
- Wiki.js: :3002
- Vaultwarden: :26226
- Actual Budget: :5006
- MCPO: :8001
- Trivy: :4954
- CouchDB: :5984 -- Obsidian LiveSync backend. DB `secondbrain`; admin creds in Vaultwarden ("CouchDB - Obsidian LiveSync"). LAN-only (no Traefik route). Data on LOCAL disk `/var/lib/couchdb` (NFS breaks CouchDB file locking). **`local.ini` must be mounted READ-WRITE** -- with `:ro` CouchDB exits 1 and logs nothing. Auth required on everything except `/_up` (left open for the blackbox probe). Vault repo: git.hobbs.farm/jeremy/second-brain (private); vault lives at `C:\Users\Jeremy\second-brain` on the Windows box.
@@ -300,7 +299,7 @@ All persistent data on NAS at `/mnt/docker-data/`:
9. **Guacamole schema**: App image pinned to `guacamole/guacamole:1.6.0` + `guacd:1.6.0` with Watchtower disabled. DB schema was reset to fresh 1.6.0 on 2026-05-20 (old schema was non-standard). Recreate connections after any DB reset.
10. **Gitea image**: Pinned to `gitea/gitea:1.26.1` with Watchtower disabled (2026-05-20). Watchtower previously pulled broken `:latest` images causing missing git binary errors.
11. **Home Assistant reverse proxy**: Config at `/mnt/docker-data/configs/homeassistant/configuration.yaml` (no hyphen). Must have `http: use_x_forwarded_for: true` + `trusted_proxies: [172.16.0.0/12, 127.0.0.1]` or proxied requests return 400.
12. **hf-pdocker-01 memory pressure (2026-07-05, retrimmed 2026-07-11)**: Node runs hot (~15-18/23GB, chronic swap). Mitigations applied: `vm.swappiness` lowered 60→10 (persisted in `/etc/sysctl.d/99-swappiness.conf`), and VoiceCoach (v1+v2, ~10 containers) fully removed (replaced by FRIDAY Voice on .150). Frigate is the main consumer. **2026-07-11:** Frigate's built-in AI features (`semantic_search: large`, `face_recognition`, `lpr`, `classification.bird`) were found *enabled* (contradicting the intended baseline) and re-disabled — dropped the embeddings CLIP model ~1.2GB→168MB and cut CPU inference; host mem 18→~9GB, swap 3.9→1.3GB right after. Config `/mnt/docker-data/configs/frigate/config/config.yml` (backup `.bak.20260711-aitrim`). Recognition is instead handled externally by clip-service + frigate-notify v2. **2026-07-12 — WHOLE-NODE CRASH:** .192 froze hard (~14:15, sshd/HA/DNS all dead, required hard reboot). Cause: Frigate `memswap_limit` had drifted to **16G** vs `mem_limit` 12G, letting it spill ~4GB into host swap; with the ffmpeg leak pinning it at 12G all day (cgroup OOM every ~45min), the host swap-thrashed to death. **Fix: node compose `memswap_limit: 12G` (= mem_limit) so Frigate can never use host swap** — a hit now trims its own ffmpeg (contained cgroup OOM), not the host. Repo already had 12G; node had drifted (never redeployed). NOTE: recording ffmpeg still leaks ~0.8GB/h atop ~6GB baseline (fills 12G in ~7.5h); twice-daily 4am+4pm restart-frigate cron mitigates but doesn't fully beat it — consider every-6h restart + fixing broken cameras (gate_camera, garage_2). Always keep `memswap_limit == mem_limit` on this node.
12. **hf-pdocker-01 memory pressure (2026-07-05, retrimmed 2026-07-11)**: Node runs hot (~15-18/23GB, chronic swap). Mitigations applied: `vm.swappiness` lowered 60→10 (persisted in `/etc/sysctl.d/99-swappiness.conf`), and VoiceCoach (v1+v2, ~10 containers) fully removed (replaced by FRIDAY Voice on .150). Frigate is the main consumer. **2026-07-11:** Frigate's built-in AI features (`semantic_search: large`, `face_recognition`, `lpr`, `classification.bird`) were found *enabled* (contradicting the intended baseline) and re-disabled — dropped the embeddings CLIP model ~1.2GB→168MB and cut CPU inference; host mem 18→~9GB, swap 3.9→1.3GB right after. Config `/mnt/docker-data/configs/frigate/config/config.yml` (backup `.bak.20260711-aitrim`). Recognition is instead handled externally by frigate-notify v2 (clip-service was removed 2026-07-13 — frigate-notify's config never referenced it). **2026-07-12 — WHOLE-NODE CRASH:** .192 froze hard (~14:15, sshd/HA/DNS all dead, required hard reboot). Cause: Frigate `memswap_limit` had drifted to **16G** vs `mem_limit` 12G, letting it spill ~4GB into host swap; with the ffmpeg leak pinning it at 12G all day (cgroup OOM every ~45min), the host swap-thrashed to death. **Fix: node compose `memswap_limit: 12G` (= mem_limit) so Frigate can never use host swap** — a hit now trims its own ffmpeg (contained cgroup OOM), not the host. Repo already had 12G; node had drifted (never redeployed). NOTE: recording ffmpeg still leaks ~0.8GB/h atop ~6GB baseline (fills 12G in ~7.5h); twice-daily 4am+4pm restart-frigate cron mitigates but doesn't fully beat it — consider every-6h restart + fixing broken cameras (gate_camera, garage_2). Always keep `memswap_limit == mem_limit` on this node.
13. **Docker Desktop vs UniFi OS Server pipe conflict (Windows box .150, 2026-07-05)**: Both fight over `\\.\pipe\docker_engine`; Docker Desktop's backend nil-panics ("backend exited before becoming ready") if Podman wins. Fix: let Docker Desktop (auto-starts at login) claim the pipe first, launch UniFi OS Server manually after. Full detail in memory `docker-desktop-unifi-podman-conflict`.
14. **MySQL InnoDB corruption on NFS**: `semaphore-db` (mysql:8.0 on NFS) hit system-tablespace page corruption and crash-looped 3789× (2026-07-11), same class as the Guacamole-on-NFS issue (#9). Recovered in-place via `--innodb-force-recovery=1` → mysqldump → fresh volume → reimport (DB was empty, no data lost); admin recreated with `semaphore user add`. Any MySQL DB on NFS is at risk — prefer LOCAL storage like guacamole-db. See memory `semaphore-db-nfs-corruption`.
15. **litellm hangs after DB blip**: litellm does NOT self-recover from a transient `litellm-db` disconnect — it loops on "Prisma DB reconnect failed" forever; needs manual `docker restart litellm`. Its healthcheck was fixed from `curl` (absent from image, permanent false-unhealthy) to a python urllib check (2026-07-11).
@@ -1,19 +0,0 @@
services:
clip-service:
image: clip-service:latest
container_name: clip-service
restart: unless-stopped
network_mode: host
mem_limit: 3G
memswap_limit: 3G
healthcheck:
test: ["CMD-SHELL", "curl -sf http://localhost:5200/health || exit 1"]
interval: 30s
timeout: 10s
retries: 3
start_period: 180s
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"
@@ -1,22 +0,0 @@
services:
actual-budget:
image: actualbudget/actual-server:latest
container_name: actual-budget
restart: unless-stopped
ports:
- "5006:5006"
volumes:
- /mnt/docker-data/configs/actual-budget:/data
mem_limit: 256M
memswap_limit: 256M
healthcheck:
test: ["CMD-SHELL", "node -e \"fetch('http://localhost:5006/').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))\""]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
logging:
driver: "json-file"
options:
max-size: "50m"
max-file: "3"