fix: agent timeouts hid hf-pdocker-01; refresh links and groups

- agent: stats one_shot=True + 20 workers (.192 went 12s -> ~2s);
  /health ignores Dead containers
- server: /containers fetch timeout 10s -> 30s (was silently returning
  [] for the busiest node)
- links page: drop FRIDAY Voice, add Onyx, Guacamole, Synapse Admin,
  n8n, CouchDB; MCPO -> /docs, Trivy -> /healthz, Jellyseerr -> Seerr
- config.json: drop dead gpu-workstation (192.168.86.27)
- groups.json: rebuilt for current container names (14 groups)
- includes earlier uncommitted work (models, styles, ports sorting)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NQhgEUqcaV4WM2S4hfXgHD

Dockerfile.server

@@ -0,0 +1,15 @@
+FROM python:3.12-slim
+
+WORKDIR /app
+
+RUN apt-get update && apt-get install -y --no-install-recommends curl && rm -rf /var/lib/apt/lists/*
+
+COPY server/requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+COPY server/ ./server/
+COPY config.json .
+
+EXPOSE 8888
+
+CMD ["uvicorn", "server.main:app", "--host", "0.0.0.0", "--port", "8888"]

README.md

@@ -1,3 +1,87 @@
-# farm-manager
+# Farm Manager
 
-Web dashboard for managing Docker services across the homelab cluster
+Web dashboard for managing Docker services across the homelab cluster. Provides full control (start/stop/restart/logs/pull) with service grouping for bulk operations.
+
+## Architecture
+
+```
+Browser --> API Server (hf-pdocker-01:8888)
+                |
+         +------+------+
+         |      |      |
+       Agent  Agent  Agent
+     (01:8889)(02:8889)(bart:8889)
+         |      |      |
+      Docker Docker  Docker
+      Socket Socket  Socket
+```
+
+- **API Server** — FastAPI on :8888. Serves the dashboard, proxies commands to node agents, manages service groups.
+- **Node Agent** — FastAPI on :8889 per node. Mounts Docker socket, auto-detects Swarm vs Compose containers.
+- **Dashboard** — Vanilla HTML/CSS/JS. Dark theme, service cards, group management, log viewer.
+
+## Quick Start
+
+```bash
+# Build and push images
+docker build -f agent/Dockerfile -t 127.0.0.1:5050/farm-agent:latest agent/
+docker build -f Dockerfile.server -t 127.0.0.1:5050/farm-manager:latest .
+docker push 127.0.0.1:5050/farm-agent:latest
+docker push 127.0.0.1:5050/farm-manager:latest
+
+# Deploy agent on each node
+cd /mnt/docker-data/compose/{node}/farm-agent && docker compose up -d
+
+# Deploy server on hf-pdocker-01
+cd /mnt/docker-data/compose/hf-pdocker-01/farm-manager && docker compose up -d
+```
+
+## Configuration
+
+`config.json` — Node definitions (name, host, agent_port)
+
+`groups.json` — Service groups for bulk operations (10 default groups included)
+
+Both stored at `/mnt/docker-data/configs/farm-manager/`.
+
+## API
+
+### Server (`:8888`)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/nodes` | List nodes with health |
+| GET | `/api/services` | All containers across nodes |
+| POST | `/api/services/{node}/{container}/start` | Start container |
+| POST | `/api/services/{node}/{container}/stop` | Stop container |
+| POST | `/api/services/{node}/{container}/restart` | Restart container |
+| GET | `/api/services/{node}/{container}/logs` | Get logs |
+| POST | `/api/services/{node}/{container}/pull` | Pull image |
+| GET/POST/PUT/DELETE | `/api/groups[/{id}]` | Group CRUD |
+| POST | `/api/groups/{id}/start\|stop\|restart` | Bulk group actions |
+
+### Agent (`:8889`)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/health` | Agent health check |
+| GET | `/containers` | List all containers |
+| POST | `/containers/{id}/start\|stop\|restart` | Container actions |
+| GET | `/containers/{id}/logs` | Get logs |
+| POST | `/containers/{id}/pull` | Pull image |
+
+## Testing
+
+```bash
+pip install -r agent/requirements.txt -r server/requirements.txt pytest pytest-asyncio httpx
+pytest -v
+```
+
+52 tests covering agent Docker operations, agent API, server proxy routes, and group CRUD.
+
+## Swarm Handling
+
+The agent auto-detects Swarm containers via the `com.docker.swarm.service.name` label:
+- **Start**: Scales service to 1 replica
+- **Stop**: Scales service to 0 replicas
+- **Restart**: Force-updates the service

agent/Dockerfile

@@ -0,0 +1,14 @@
+FROM python:3.12-slim
+
+WORKDIR /app
+
+RUN apt-get update && apt-get install -y --no-install-recommends curl && rm -rf /var/lib/apt/lists/*
+
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+COPY . /app/agent/
+
+EXPOSE 8889
+
+CMD ["uvicorn", "agent.main:app", "--host", "0.0.0.0", "--port", "8889"]

agent/docker_ops.py

@@ -1,6 +1,7 @@
 """Docker operations wrapper for managing Swarm and Compose containers."""
 
 import socket
+from concurrent.futures import ThreadPoolExecutor
 from datetime import datetime, timezone
 
 import docker
@@ -77,6 +78,51 @@ class DockerOps:
         except Exception:
             return "unknown"
 
+    @staticmethod
+    def _extract_ports(container) -> list[str]:
+        """Extract exposed host ports from a container's network settings.
+
+        Returns a sorted, deduplicated list of strings like '8080', '443'.
+        Only includes ports bound to the host.  For host-network containers,
+        reads the exposed ports from the image config instead.
+        """
+        ports = set()
+        network_mode = container.attrs.get("HostConfig", {}).get("NetworkMode", "")
+
+        if network_mode == "host":
+            # Host networking: use exposed ports from the image/container config
+            exposed = container.attrs.get("Config", {}).get("ExposedPorts", {})
+            for port_proto in exposed:
+                port = port_proto.split("/")[0]
+                ports.add(port)
+        else:
+            # Bridge/overlay: use published port bindings
+            port_bindings = container.attrs.get("NetworkSettings", {}).get("Ports", {}) or {}
+            for port_proto, bindings in port_bindings.items():
+                if bindings:
+                    for binding in bindings:
+                        host_port = binding.get("HostPort")
+                        if host_port:
+                            ports.add(host_port)
+
+        return sorted(ports, key=lambda p: int(p) if p.isdigit() else 0)
+
+    def _get_container_memory(self, container_id: str) -> tuple[str, int, int]:
+        """Fetch memory usage and limit for a single running container.
+
+        Returns (container_id, usage_bytes, limit_bytes).
+        """
+        try:
+            stats = self.client.api.stats(container_id, stream=False, one_shot=True)
+            mem = stats.get("memory_stats", {})
+            usage = mem.get("usage", 0)
+            limit = mem.get("limit", 0)
+            # Subtract inactive file cache for accurate usage
+            cache = mem.get("stats", {}).get("inactive_file", 0)
+            return (container_id, max(usage - cache, 0), limit)
+        except Exception:
+            return (container_id, 0, 0)
+
     # ------------------------------------------------------------------
     # Public API
     # ------------------------------------------------------------------
@@ -84,17 +130,35 @@ class DockerOps:
     def list_containers(self) -> list[dict]:
         """List all containers (running + stopped) with metadata.
 
+        Also includes Swarm services scaled to 0 replicas (which have no
+        containers) so they remain visible in the dashboard.
+
         Returns a list of dicts, each containing:
             id, name, status, image, created, uptime, is_swarm,
-            swarm_service
+            swarm_service, memory_usage, memory_limit
+
+        Uses the low-level API to get container IDs to avoid NotFound errors
+        from containers that disappear between list and inspect (e.g. Swarm
+        task containers in the Dead state).
         """
-        containers = self.client.containers.list(all=True)
+        raw_ids = [r["Id"] for r in self.client.api.containers(all=True)]
         result = []
-        for c in containers:
+        swarm_services_with_containers = set()
+        running_ids = {}  # full_id -> short_id for memory stats lookup
+
+        for cid in raw_ids:
+            try:
+                c = self.client.containers.get(cid)
+            except NotFound:
+                continue
             image_tags = c.image.tags
             image_name = image_tags[0] if image_tags else "<none>"
             swarm_service = self._swarm_service_name(c)
+            if swarm_service:
+                swarm_services_with_containers.add(swarm_service)
             started_at = c.attrs.get("State", {}).get("StartedAt", "")
+            if c.status == "running":
+                running_ids[cid] = c.short_id
             result.append(
                 {
                     "id": c.short_id,
@@ -105,12 +169,84 @@ class DockerOps:
                     "uptime": self._format_uptime(started_at) if c.status == "running" else None,
                     "is_swarm": self._is_swarm(c),
                     "swarm_service": swarm_service,
+                    "ports": self._extract_ports(c),
+                    "memory_usage": 0,
+                    "memory_limit": 0,
                 }
             )
+
+        # Fetch memory stats for running containers in parallel
+        if running_ids:
+            mem_stats = {}
+            with ThreadPoolExecutor(max_workers=20) as pool:
+                futures = {
+                    pool.submit(self._get_container_memory, full_id): short_id
+                    for full_id, short_id in running_ids.items()
+                }
+                for future in futures:
+                    full_id, usage, limit = future.result()
+                    short_id = futures[future]
+                    mem_stats[short_id] = (usage, limit)
+            for entry in result:
+                if entry["id"] in mem_stats:
+                    entry["memory_usage"], entry["memory_limit"] = mem_stats[entry["id"]]
+
+        # Include Swarm services with 0 replicas (only works on manager nodes)
+        try:
+            for svc in self.client.services.list():
+                if svc.name not in swarm_services_with_containers:
+                    image = (
+                        svc.attrs.get("Spec", {})
+                        .get("TaskTemplate", {})
+                        .get("ContainerSpec", {})
+                        .get("Image", "<none>")
+                    )
+                    if "@" in image:
+                        image = image.split("@")[0]
+                    # Extract published ports from Swarm service endpoint spec
+                    svc_ports = []
+                    endpoint_spec = svc.attrs.get("Spec", {}).get("EndpointSpec", {})
+                    for p in endpoint_spec.get("Ports", []):
+                        published = p.get("PublishedPort")
+                        if published:
+                            svc_ports.append(str(published))
+                    svc_ports.sort(key=lambda x: int(x) if x.isdigit() else 0)
+
+                    result.append(
+                        {
+                            "id": f"swarm:{svc.name}",
+                            "name": svc.name,
+                            "status": "stopped",
+                            "image": image,
+                            "created": svc.attrs.get("CreatedAt", ""),
+                            "uptime": None,
+                            "is_swarm": True,
+                            "swarm_service": svc.name,
+                            "ports": svc_ports,
+                        }
+                    )
+        except Exception:
+            pass  # Not a manager node or Swarm not active
+
         return result
 
     def start_container(self, container_id: str) -> dict:
         """Start a container.  For Swarm services, scale to 1 replica."""
+        # Handle 0-replica Swarm services (synthetic swarm: IDs)
+        if container_id.startswith("swarm:"):
+            svc_name = container_id[6:]
+            try:
+                service = self._get_service(svc_name)
+                if service:
+                    service.scale(1)
+                    return {
+                        "success": True,
+                        "message": f"Scaled Swarm service {svc_name} to 1 replica",
+                    }
+                return {"success": False, "message": f"Swarm service {svc_name} not found"}
+            except Exception as exc:
+                return {"success": False, "message": str(exc)}
+
         try:
             container = self.client.containers.get(container_id)
         except NotFound:
@@ -135,6 +271,20 @@ class DockerOps:
 
     def stop_container(self, container_id: str) -> dict:
         """Stop a container.  For Swarm services, scale to 0 replicas."""
+        if container_id.startswith("swarm:"):
+            svc_name = container_id[6:]
+            try:
+                service = self._get_service(svc_name)
+                if service:
+                    service.scale(0)
+                    return {
+                        "success": True,
+                        "message": f"Scaled Swarm service {svc_name} to 0 replicas",
+                    }
+                return {"success": False, "message": f"Swarm service {svc_name} not found"}
+            except Exception as exc:
+                return {"success": False, "message": str(exc)}
+
         try:
             container = self.client.containers.get(container_id)
         except NotFound:
@@ -159,6 +309,20 @@ class DockerOps:
 
     def restart_container(self, container_id: str) -> dict:
         """Restart a container.  For Swarm services, force-update."""
+        if container_id.startswith("swarm:"):
+            svc_name = container_id[6:]
+            try:
+                service = self._get_service(svc_name)
+                if service:
+                    service.force_update()
+                    return {
+                        "success": True,
+                        "message": f"Force-updated Swarm service {svc_name}",
+                    }
+                return {"success": False, "message": f"Swarm service {svc_name} not found"}
+            except Exception as exc:
+                return {"success": False, "message": str(exc)}
+
         try:
             container = self.client.containers.get(container_id)
         except NotFound:
@@ -187,6 +351,13 @@ class DockerOps:
         Returns:
             dict with keys ``container`` (name) and ``logs`` (string).
         """
+        if container_id.startswith("swarm:"):
+            svc_name = container_id[6:]
+            return {
+                "container": svc_name,
+                "logs": "Service is scaled to 0 replicas — no logs available.\nStart the service to view logs.",
+            }
+
         try:
             container = self.client.containers.get(container_id)
         except NotFound:
@@ -200,6 +371,27 @@ class DockerOps:
 
     def pull_image(self, container_id: str) -> dict:
         """Pull the latest version of the container's image."""
+        if container_id.startswith("swarm:"):
+            svc_name = container_id[6:]
+            service = self._get_service(svc_name)
+            if not service:
+                return {"success": False, "message": f"Swarm service {svc_name} not found"}
+            image = (
+                service.attrs.get("Spec", {})
+                .get("TaskTemplate", {})
+                .get("ContainerSpec", {})
+                .get("Image", "")
+            )
+            if "@" in image:
+                image = image.split("@")[0]
+            if not image:
+                return {"success": False, "message": f"No image found for service {svc_name}"}
+            try:
+                self.client.images.pull(image)
+                return {"success": True, "message": f"Pulled image {image}"}
+            except Exception as exc:
+                return {"success": False, "message": str(exc)}
+
         try:
             container = self.client.containers.get(container_id)
         except NotFound:
@@ -217,8 +409,15 @@ class DockerOps:
             return {"success": False, "message": str(exc)}
 
     def get_health(self) -> dict:
-        """Return node health info: hostname and container count."""
-        containers = self.client.containers.list(all=True)
+        """Return node health info: hostname and container count.
+
+        Uses the low-level API to avoid NotFound errors from containers
+        stuck in Docker's 'Dead' state.
+        """
+        containers = [
+            c for c in self.client.api.containers(all=True)
+            if c.get("State") != "dead"
+        ]
         return {
             "status": "healthy",
             "hostname": socket.gethostname(),

agent/main.py

@@ -0,0 +1,48 @@
+from fastapi import FastAPI, Query
+from agent.docker_ops import DockerOps
+
+app = FastAPI(title="Farm Manager Agent")
+
+try:
+    ops = DockerOps()
+except Exception:
+    ops = None  # Will be mocked in tests
+
+@app.get("/health")
+def health():
+    return ops.get_health()
+
+
+@app.get("/containers")
+def list_containers():
+    return ops.list_containers()
+
+
+@app.post("/containers/{container_id}/start")
+def start_container(container_id: str):
+    return ops.start_container(container_id)
+
+
+@app.post("/containers/{container_id}/stop")
+def stop_container(container_id: str):
+    return ops.stop_container(container_id)
+
+
+@app.post("/containers/{container_id}/restart")
+def restart_container(container_id: str):
+    return ops.restart_container(container_id)
+
+
+@app.get("/containers/{container_id}/logs")
+def get_logs(container_id: str, tail: int = Query(default=200)):
+    return ops.get_logs(container_id, tail=tail)
+
+
+@app.post("/containers/{container_id}/pull")
+def pull_image(container_id: str):
+    return ops.pull_image(container_id)
+
+
+if __name__ == "__main__":
+    import uvicorn
+    uvicorn.run(app, host="0.0.0.0", port=8889)

agent/models.py

@@ -10,6 +10,9 @@ class ContainerInfo(BaseModel):
     uptime: str | None = None
     is_swarm: bool = False
     swarm_service: str | None = None
+    ports: list[str] = []
+    memory_usage: int = 0
+    memory_limit: int = 0
 
 
 class ContainerAction(BaseModel):

config.json

@@ -1,7 +1,19 @@
 {
   "nodes": [
-    {"name": "hf-pdocker-01", "host": "192.168.86.192", "agent_port": 8889},
-    {"name": "hf-pdocker-02", "host": "192.168.86.100", "agent_port": 8889},
-    {"name": "bart", "host": "192.168.86.167", "agent_port": 8889}
+    {
+      "name": "hf-pdocker-01",
+      "host": "192.168.86.192",
+      "agent_port": 8889
+    },
+    {
+      "name": "hf-pdocker-02",
+      "host": "192.168.86.100",
+      "agent_port": 8889
+    },
+    {
+      "name": "bart",
+      "host": "192.168.86.167",
+      "agent_port": 8889
+    }
   ]
-}
+}

deploy/agent/docker-compose.yml

@@ -0,0 +1,22 @@
+services:
+  farm-agent:
+    image: 127.0.0.1:5050/farm-agent:latest
+    container_name: farm-agent
+    restart: unless-stopped
+    ports:
+      - "8889:8889"
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock
+    mem_limit: 128M
+    memswap_limit: 128M
+    healthcheck:
+      test: ["CMD-SHELL", "curl -sf http://localhost:8889/health || exit 1"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 15s
+    logging:
+      driver: "json-file"
+      options:
+        max-size: "50m"
+        max-file: "3"

deploy/server/docker-compose.yml

@@ -0,0 +1,25 @@
+services:
+  farm-manager:
+    image: 127.0.0.1:5050/farm-manager:latest
+    container_name: farm-manager
+    restart: unless-stopped
+    ports:
+      - "8888:8888"
+    environment:
+      - CONFIG_PATH=/app/config.json
+      - GROUPS_PATH=/data/groups.json
+    volumes:
+      - /mnt/docker-data/configs/farm-manager:/data
+    mem_limit: 256M
+    memswap_limit: 256M
+    healthcheck:
+      test: ["CMD-SHELL", "curl -sf http://localhost:8888/health || exit 1"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 15s
+    logging:
+      driver: "json-file"
+      options:
+        max-size: "50m"
+        max-file: "3"

groups.json

@@ -1,103 +1,340 @@
 {
-  "groups": [
-    {
-      "id": "ai-agents",
-      "name": "AI Agents",
-      "services": [
-        {"node": "hf-pdocker-01", "container": "ai-agents_margaret"},
-        {"node": "hf-pdocker-01", "container": "ai-agents_spirit"},
-        {"node": "hf-pdocker-01", "container": "ai-agents_bob"},
-        {"node": "hf-pdocker-01", "container": "ai-agents_clippy"}
-      ]
-    },
-    {
-      "id": "smart-home",
-      "name": "Smart Home",
-      "services": [
-        {"node": "hf-pdocker-01", "container": "smarthome_homeassistant"},
-        {"node": "hf-pdocker-01", "container": "smarthome_esphome"},
-        {"node": "hf-pdocker-01", "container": "smarthome_mosquitto"}
-      ]
-    },
-    {
-      "id": "monitoring",
-      "name": "Monitoring",
-      "services": [
-        {"node": "hf-pdocker-01", "container": "monitoring_grafana"},
-        {"node": "hf-pdocker-01", "container": "monitoring_prometheus"},
-        {"node": "hf-pdocker-01", "container": "monitoring_alertmanager"},
-        {"node": "hf-pdocker-01", "container": "monitoring_influxdb"},
-        {"node": "hf-pdocker-01", "container": "monitoring_blackbox-exporter"}
-      ]
-    },
-    {
-      "id": "media",
-      "name": "Media",
-      "services": [
-        {"node": "bart", "container": "jellyfin"},
-        {"node": "bart", "container": "sonarr"},
-        {"node": "bart", "container": "radarr"},
-        {"node": "bart", "container": "sabnzbd"},
-        {"node": "bart", "container": "nzbhydra2"},
-        {"node": "bart", "container": "jellyseerr"}
-      ]
-    },
-    {
-      "id": "matrix",
-      "name": "Matrix",
-      "services": [
-        {"node": "hf-pdocker-01", "container": "matrix_synapse"},
-        {"node": "hf-pdocker-01", "container": "matrix_synapse-db"},
-        {"node": "hf-pdocker-01", "container": "matrix_wellknown"}
-      ]
-    },
-    {
-      "id": "guacamole",
-      "name": "Guacamole",
-      "services": [
-        {"node": "hf-pdocker-01", "container": "guacamole_guacamole"},
-        {"node": "hf-pdocker-01", "container": "guacamole_guacd"},
-        {"node": "hf-pdocker-01", "container": "guacamole_guacamole-db"}
-      ]
-    },
-    {
-      "id": "dev-tools",
-      "name": "Dev Tools",
-      "services": [
-        {"node": "hf-pdocker-02", "container": "gitea"},
-        {"node": "hf-pdocker-02", "container": "gitea-db"},
-        {"node": "hf-pdocker-02", "container": "semaphore"},
-        {"node": "hf-pdocker-02", "container": "semaphore-db"},
-        {"node": "hf-pdocker-02", "container": "wikijs"},
-        {"node": "hf-pdocker-02", "container": "wikijs-db"}
-      ]
-    },
-    {
-      "id": "security",
-      "name": "Security",
-      "services": [
-        {"node": "hf-pdocker-02", "container": "vaultwarden"},
-        {"node": "hf-pdocker-02", "container": "trivy"},
-        {"node": "bart", "container": "openvas"}
-      ]
-    },
-    {
-      "id": "network",
-      "name": "Network",
-      "services": [
-        {"node": "hf-pdocker-01", "container": "traefik_traefik"},
-        {"node": "hf-pdocker-01", "container": "adguard_adguardhome"},
-        {"node": "hf-pdocker-02", "container": "unifi"},
-        {"node": "hf-pdocker-02", "container": "unifi-db"}
-      ]
-    },
-    {
-      "id": "frigate",
-      "name": "Frigate",
-      "services": [
-        {"node": "hf-pdocker-01", "container": "frigate"},
-        {"node": "hf-pdocker-01", "container": "frigate-notify"}
-      ]
-    }
-  ]
-}
+    "groups": [
+        {
+            "id": "ai-agents",
+            "name": "AI Agents",
+            "services": [
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "margaret"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "spirit"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "hermes"
+                }
+            ]
+        },
+        {
+            "id": "smart-home",
+            "name": "Smart Home",
+            "services": [
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "homeassistant"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "esphome"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "mosquitto"
+                }
+            ]
+        },
+        {
+            "id": "monitoring",
+            "name": "Monitoring",
+            "services": [
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "monitoring-grafana-1"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "monitoring-prometheus-1"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "monitoring-alertmanager-1"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "monitoring-blackbox-exporter-1"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "monitoring-matrix-alertmanager-receiver-1"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "monitoring-cadvisor-1"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "influxdb"
+                }
+            ]
+        },
+        {
+            "id": "media",
+            "name": "Media",
+            "services": [
+                {
+                    "node": "bart",
+                    "container": "jellyfin"
+                },
+                {
+                    "node": "bart",
+                    "container": "sonarr"
+                },
+                {
+                    "node": "bart",
+                    "container": "radarr"
+                },
+                {
+                    "node": "bart",
+                    "container": "sabnzbd"
+                },
+                {
+                    "node": "bart",
+                    "container": "nzbhydra2"
+                },
+                {
+                    "node": "bart",
+                    "container": "seerr"
+                }
+            ]
+        },
+        {
+            "id": "matrix",
+            "name": "Matrix",
+            "services": [
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "synapse"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "synapse-db"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "matrix-wellknown"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "mas"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "mas-db"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "synapse-admin"
+                }
+            ]
+        },
+        {
+            "id": "guacamole",
+            "name": "Guacamole",
+            "services": [
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "guacamole"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "guacd"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "guacamole-db"
+                }
+            ]
+        },
+        {
+            "id": "dev-tools",
+            "name": "Dev Tools",
+            "services": [
+                {
+                    "node": "hf-pdocker-02",
+                    "container": "gitea"
+                },
+                {
+                    "node": "hf-pdocker-02",
+                    "container": "gitea-db"
+                },
+                {
+                    "node": "hf-pdocker-02",
+                    "container": "semaphore"
+                },
+                {
+                    "node": "hf-pdocker-02",
+                    "container": "semaphore-db"
+                },
+                {
+                    "node": "hf-pdocker-02",
+                    "container": "wikijs"
+                },
+                {
+                    "node": "hf-pdocker-02",
+                    "container": "wikijs-db"
+                }
+            ]
+        },
+        {
+            "id": "security",
+            "name": "Security",
+            "services": [
+                {
+                    "node": "hf-pdocker-02",
+                    "container": "vaultwarden"
+                },
+                {
+                    "node": "hf-pdocker-02",
+                    "container": "trivy"
+                }
+            ]
+        },
+        {
+            "id": "network",
+            "name": "Network",
+            "services": [
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "traefik"
+                }
+            ]
+        },
+        {
+            "id": "frigate",
+            "name": "Frigate",
+            "services": [
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "frigate"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "frigate-notify"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "clip-service"
+                }
+            ]
+        },
+        {
+            "id": "onyx",
+            "name": "Onyx",
+            "services": [
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "onyx-api_server-1"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "onyx-background-1"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "onyx-cache-1"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "onyx-index-1"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "onyx-indexing_model_server-1"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "onyx-inference_model_server-1"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "onyx-nginx-1"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "onyx-relational_db-1"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "onyx-web_server-1"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "onyx-opensearch-1"
+                }
+            ]
+        },
+        {
+            "id": "llm",
+            "name": "LLM",
+            "services": [
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "litellm"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "litellm-db"
+                }
+            ]
+        },
+        {
+            "id": "automation",
+            "name": "Automation & Utilities",
+            "services": [
+                {
+                    "node": "hf-pdocker-02",
+                    "container": "n8n"
+                },
+                {
+                    "node": "hf-pdocker-02",
+                    "container": "mcpo"
+                },
+                {
+                    "node": "hf-pdocker-02",
+                    "container": "searxng"
+                },
+                {
+                    "node": "hf-pdocker-02",
+                    "container": "unstructured-api"
+                },
+                {
+                    "node": "hf-pdocker-02",
+                    "container": "couchdb"
+                }
+            ]
+        },
+        {
+            "id": "home-apps",
+            "name": "Home Apps",
+            "services": [
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "linkedstorm-api"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "linkedstorm-pipeline"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "linkedstorm-cron"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "rongorongo-web"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "rongorongo-api"
+                },
+                {
+                    "node": "hf-pdocker-01",
+                    "container": "farm-manager"
+                }
+            ]
+        }
+    ]
+}

server/config.py

@@ -0,0 +1,33 @@
+"""Configuration loader for the Farm Manager server.
+
+Reads node definitions from a JSON config file. The path defaults to
+``/app/config.json`` but can be overridden via the ``CONFIG_PATH``
+environment variable.
+"""
+
+import json
+import os
+
+_config = None
+
+
+def _load_config():
+    global _config
+    if _config is None:
+        config_path = os.environ.get("CONFIG_PATH", "/app/config.json")
+        with open(config_path) as f:
+            _config = json.load(f)
+    return _config
+
+
+def get_nodes() -> list[dict]:
+    """Return the list of node definitions from the config file."""
+    return _load_config()["nodes"]
+
+
+def get_node_url(node_name: str) -> str | None:
+    """Return the agent base URL for *node_name*, or ``None`` if unknown."""
+    for node in get_nodes():
+        if node["name"] == node_name:
+            return f"http://{node['host']}:{node['agent_port']}"
+    return None

server/group_store.py

@@ -0,0 +1,101 @@
+"""Singleton store for service groups with JSON file persistence.
+
+Groups are persisted to a JSON file specified by the ``GROUPS_PATH``
+environment variable (default ``/app/groups.json``).
+"""
+
+import json
+import os
+import threading
+
+from server.models import Group, GroupsData
+
+
+class GroupStore:
+    """Thread-safe singleton that manages groups and persists to disk."""
+
+    _instance: "GroupStore | None" = None
+    _lock = threading.Lock()
+
+    def __new__(cls) -> "GroupStore":
+        with cls._lock:
+            if cls._instance is None:
+                cls._instance = super().__new__(cls)
+                cls._instance._initialized = False
+            return cls._instance
+
+    def __init__(self) -> None:
+        if self._initialized:
+            return
+        self._initialized = True
+        self._data_lock = threading.Lock()
+        self._path = os.environ.get("GROUPS_PATH", "/app/groups.json")
+        self._groups: list[Group] = []
+        self._load()
+
+    # ------------------------------------------------------------------
+    # Persistence
+    # ------------------------------------------------------------------
+
+    def _load(self) -> None:
+        """Load groups from the JSON file on disk."""
+        try:
+            with open(self._path) as f:
+                raw = json.load(f)
+            data = GroupsData(**raw)
+            self._groups = data.groups
+        except (FileNotFoundError, json.JSONDecodeError):
+            self._groups = []
+
+    def _save(self) -> None:
+        """Persist the current groups list to the JSON file."""
+        data = GroupsData(groups=self._groups)
+        with open(self._path, "w") as f:
+            json.dump(data.model_dump(), f, indent=2)
+
+    # ------------------------------------------------------------------
+    # CRUD operations
+    # ------------------------------------------------------------------
+
+    def list_groups(self) -> list[Group]:
+        """Return all groups."""
+        with self._data_lock:
+            return list(self._groups)
+
+    def get_group(self, group_id: str) -> Group | None:
+        """Return a single group by id, or None."""
+        with self._data_lock:
+            for g in self._groups:
+                if g.id == group_id:
+                    return g
+            return None
+
+    def create_group(self, group: Group) -> Group:
+        """Add a new group. Raises ValueError if duplicate id."""
+        with self._data_lock:
+            for g in self._groups:
+                if g.id == group.id:
+                    raise ValueError(f"Group '{group.id}' already exists")
+            self._groups.append(group)
+            self._save()
+            return group
+
+    def update_group(self, group_id: str, group: Group) -> Group | None:
+        """Replace a group by id. Returns updated group or None."""
+        with self._data_lock:
+            for i, g in enumerate(self._groups):
+                if g.id == group_id:
+                    self._groups[i] = group
+                    self._save()
+                    return group
+            return None
+
+    def delete_group(self, group_id: str) -> bool:
+        """Remove a group by id. Returns True if found and deleted."""
+        with self._data_lock:
+            for i, g in enumerate(self._groups):
+                if g.id == group_id:
+                    self._groups.pop(i)
+                    self._save()
+                    return True
+            return False

server/main.py

@@ -0,0 +1,29 @@
+"""Farm Manager API server — main FastAPI application."""
+
+import os
+
+from fastapi import FastAPI
+from fastapi.staticfiles import StaticFiles
+
+from server.routes.groups import router as groups_router
+from server.routes.nodes import router as nodes_router
+from server.routes.services import router as services_router
+
+app = FastAPI(title="Farm Manager")
+app.include_router(groups_router)
+app.include_router(nodes_router)
+app.include_router(services_router)
+
+
+@app.get("/health")
+def health():
+    return {"status": "ok"}
+
+
+static_dir = os.path.join(os.path.dirname(__file__), "static")
+if os.path.isdir(static_dir):
+    app.mount("/", StaticFiles(directory=static_dir, html=True), name="static")
+
+if __name__ == "__main__":
+    import uvicorn
+    uvicorn.run(app, host="0.0.0.0", port=8888)

server/models.py

@@ -40,3 +40,6 @@ class ServiceInfo(BaseModel):
     uptime: str | None = None
     is_swarm: bool = False
     swarm_service: str | None = None
+    ports: list[str] = []
+    memory_usage: int = 0
+    memory_limit: int = 0

server/routes/groups.py

@@ -0,0 +1,109 @@
+"""Group CRUD and bulk action endpoints for the Farm Manager API server."""
+
+import asyncio
+
+import httpx
+from fastapi import APIRouter, HTTPException
+
+from server.config import get_node_url
+from server.group_store import GroupStore
+from server.models import Group
+
+router = APIRouter()
+
+
+def _get_store() -> GroupStore:
+    return GroupStore()
+
+
+def _require_group(group_id: str) -> Group:
+    """Return the group or raise 404."""
+    group = _get_store().get_group(group_id)
+    if group is None:
+        raise HTTPException(status_code=404, detail=f"Group not found: {group_id}")
+    return group
+
+
+# ---------------------------------------------------------------------------
+# CRUD
+# ---------------------------------------------------------------------------
+
+@router.get("/api/groups")
+def list_groups():
+    """Return all groups."""
+    return [g.model_dump() for g in _get_store().list_groups()]
+
+
+@router.post("/api/groups")
+def create_group(group: Group):
+    """Create a new group. Returns 409 if duplicate id."""
+    try:
+        created = _get_store().create_group(group)
+        return created.model_dump()
+    except ValueError:
+        raise HTTPException(status_code=409, detail=f"Group '{group.id}' already exists")
+
+
+@router.put("/api/groups/{group_id}")
+def update_group(group_id: str, group: Group):
+    """Update an existing group by id. Returns 404 if not found."""
+    updated = _get_store().update_group(group_id, group)
+    if updated is None:
+        raise HTTPException(status_code=404, detail=f"Group not found: {group_id}")
+    return updated.model_dump()
+
+
+@router.delete("/api/groups/{group_id}")
+def delete_group(group_id: str):
+    """Delete a group by id. Returns 404 if not found."""
+    if not _get_store().delete_group(group_id):
+        raise HTTPException(status_code=404, detail=f"Group not found: {group_id}")
+    return {"success": True}
+
+
+# ---------------------------------------------------------------------------
+# Bulk actions — start/stop/restart all services in a group
+# ---------------------------------------------------------------------------
+
+async def _send_action(
+    http_client: httpx.AsyncClient, node: str, container: str, action: str
+) -> dict:
+    """Send a start/stop/restart action to a single container via its node agent."""
+    url = get_node_url(node)
+    if url is None:
+        return {"node": node, "container": container, "success": False, "error": f"Unknown node: {node}"}
+    try:
+        resp = await http_client.post(f"{url}/containers/{container}/{action}", timeout=30.0)
+        return {"node": node, "container": container, **resp.json()}
+    except Exception as exc:
+        return {"node": node, "container": container, "success": False, "error": str(exc)}
+
+
+async def _bulk_action(group_id: str, action: str) -> list[dict]:
+    """Execute *action* on every service in the group."""
+    group = _require_group(group_id)
+    async with httpx.AsyncClient() as client:
+        tasks = [
+            _send_action(client, svc.node, svc.container, action)
+            for svc in group.services
+        ]
+        results = await asyncio.gather(*tasks)
+    return list(results)
+
+
+@router.post("/api/groups/{group_id}/start")
+async def start_group(group_id: str):
+    """Start all services in a group."""
+    return await _bulk_action(group_id, "start")
+
+
+@router.post("/api/groups/{group_id}/stop")
+async def stop_group(group_id: str):
+    """Stop all services in a group."""
+    return await _bulk_action(group_id, "stop")
+
+
+@router.post("/api/groups/{group_id}/restart")
+async def restart_group(group_id: str):
+    """Restart all services in a group."""
+    return await _bulk_action(group_id, "restart")

server/routes/nodes.py

@@ -0,0 +1,44 @@
+"""Node health endpoints for the Farm Manager API server."""
+
+import asyncio
+
+import httpx
+from fastapi import APIRouter
+
+from server.config import get_nodes
+
+router = APIRouter()
+
+
+async def _check_node_health(http_client: httpx.AsyncClient, node: dict) -> dict:
+    """Query a single agent's /health endpoint and return a NodeStatus dict."""
+    url = f"http://{node['host']}:{node['agent_port']}/health"
+    try:
+        resp = await http_client.get(url, timeout=5.0)
+        resp.raise_for_status()
+        data = resp.json()
+        return {
+            "name": node["name"],
+            "host": node["host"],
+            "healthy": True,
+            "containers_total": data.get("containers_total", 0),
+            "error": None,
+        }
+    except Exception as exc:
+        return {
+            "name": node["name"],
+            "host": node["host"],
+            "healthy": False,
+            "containers_total": 0,
+            "error": str(exc),
+        }
+
+
+@router.get("/api/nodes")
+async def list_nodes():
+    """Return all configured nodes with their current health status."""
+    nodes = get_nodes()
+    async with httpx.AsyncClient() as client:
+        tasks = [_check_node_health(client, node) for node in nodes]
+        results = await asyncio.gather(*tasks)
+    return list(results)

server/routes/services.py

@@ -0,0 +1,107 @@
+"""Service proxy endpoints for the Farm Manager API server.
+
+Each endpoint proxies to the appropriate node agent based on the
+``{node}`` path parameter.
+"""
+
+import asyncio
+
+import httpx
+from fastapi import APIRouter, HTTPException, Query
+
+from server.config import get_nodes, get_node_url
+
+router = APIRouter()
+
+
+def _require_node_url(node: str) -> str:
+    """Return the agent URL for *node* or raise 404."""
+    url = get_node_url(node)
+    if url is None:
+        raise HTTPException(status_code=404, detail=f"Unknown node: {node}")
+    return url
+
+
+# ---------------------------------------------------------------------------
+# GET /api/services — aggregate containers from all nodes
+# ---------------------------------------------------------------------------
+
+async def _fetch_containers(http_client: httpx.AsyncClient, node: dict) -> list[dict]:
+    """Fetch containers from a single node agent, adding the node field."""
+    url = f"http://{node['host']}:{node['agent_port']}/containers"
+    try:
+        resp = await http_client.get(url, timeout=30.0)
+        resp.raise_for_status()
+        containers = resp.json()
+        for c in containers:
+            c["node"] = node["name"]
+        return containers
+    except Exception:
+        return []
+
+
+@router.get("/api/services")
+async def list_services():
+    """Return containers from all nodes with a ``node`` field added."""
+    nodes = get_nodes()
+    async with httpx.AsyncClient() as client:
+        tasks = [_fetch_containers(client, node) for node in nodes]
+        results = await asyncio.gather(*tasks)
+    # Flatten list of lists
+    all_containers = []
+    for node_containers in results:
+        all_containers.extend(node_containers)
+    return all_containers
+
+
+# ---------------------------------------------------------------------------
+# Container actions — proxy to the correct agent
+# ---------------------------------------------------------------------------
+
+@router.post("/api/services/{node}/{container_id}/start")
+async def start_service(node: str, container_id: str):
+    """Proxy start request to the agent on *node*."""
+    base = _require_node_url(node)
+    async with httpx.AsyncClient() as client:
+        resp = await client.post(f"{base}/containers/{container_id}/start", timeout=30.0)
+        return resp.json()
+
+
+@router.post("/api/services/{node}/{container_id}/stop")
+async def stop_service(node: str, container_id: str):
+    """Proxy stop request to the agent on *node*."""
+    base = _require_node_url(node)
+    async with httpx.AsyncClient() as client:
+        resp = await client.post(f"{base}/containers/{container_id}/stop", timeout=30.0)
+        return resp.json()
+
+
+@router.post("/api/services/{node}/{container_id}/restart")
+async def restart_service(node: str, container_id: str):
+    """Proxy restart request to the agent on *node*."""
+    base = _require_node_url(node)
+    async with httpx.AsyncClient() as client:
+        resp = await client.post(f"{base}/containers/{container_id}/restart", timeout=30.0)
+        return resp.json()
+
+
+@router.get("/api/services/{node}/{container_id}/logs")
+async def get_service_logs(node: str, container_id: str, tail: int = Query(default=200)):
+    """Proxy logs request to the agent on *node*."""
+    base = _require_node_url(node)
+    async with httpx.AsyncClient() as client:
+        resp = await client.get(
+            f"{base}/containers/{container_id}/logs",
+            params={"tail": tail},
+            timeout=10.0,
+        )
+        return resp.json()
+
+
+@router.post("/api/services/{node}/{container_id}/pull")
+async def pull_service(node: str, container_id: str):
+    """Proxy pull request to the agent on *node*."""
+    base = _require_node_url(node)
+    async with httpx.AsyncClient() as client:
+        resp = await client.post(f"{base}/containers/{container_id}/pull", timeout=60.0)
+        return resp.json()

server/static/index.html

@@ -0,0 +1,139 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Farm Manager</title>
+    <link rel="stylesheet" href="/style.css">
+</head>
+<body>
+
+<!-- ===== Header ===== -->
+<header id="header">
+    <h1>Farm Manager</h1>
+    <div class="header-right">
+        <span id="refresh-indicator" class="refresh-indicator" title="Auto-refreshing every 15s">
+            <span class="refresh-dot"></span> Auto-refresh
+        </span>
+        <span id="last-updated" class="last-updated"></span>
+    </div>
+</header>
+
+<!-- ===== Sidebar ===== -->
+<nav id="sidebar">
+    <div class="sidebar-section">
+        <a href="#" class="sidebar-item active" id="nav-all" data-view="all">
+            <span class="sidebar-icon">&#9776;</span> All Services
+        </a>
+        <a href="#" class="sidebar-item" id="nav-links" data-view="links">
+            <span class="sidebar-icon">&#128279;</span> Server Links
+        </a>
+    </div>
+
+    <div class="sidebar-section">
+        <div class="sidebar-heading">Nodes</div>
+        <div id="node-list"></div>
+    </div>
+
+    <div class="sidebar-section">
+        <div class="sidebar-heading">
+            Groups
+            <button class="btn-icon" id="btn-add-group" title="Create group">&#10010;</button>
+        </div>
+        <div id="group-list"></div>
+    </div>
+</nav>
+
+<!-- ===== Main Content ===== -->
+<main id="main">
+    <!-- Toolbar -->
+    <div id="toolbar" class="toolbar">
+        <div class="toolbar-left">
+            <h2 id="view-title">All Services</h2>
+            <span id="service-count" class="service-count"></span>
+        </div>
+        <div class="toolbar-center" id="group-actions" style="display:none;">
+            <button class="btn btn-success btn-sm" id="btn-group-start" title="Start all in group">&#9654; Start All</button>
+            <button class="btn btn-danger btn-sm" id="btn-group-stop" title="Stop all in group">&#9632; Stop All</button>
+            <button class="btn btn-warning btn-sm" id="btn-group-restart" title="Restart all in group">&#8635; Restart All</button>
+        </div>
+        <div class="toolbar-filters" id="status-filters">
+            <button class="filter-pill active" data-filter="all">All</button>
+            <button class="filter-pill" data-filter="running">Running <span id="count-running" class="filter-count"></span></button>
+            <button class="filter-pill" data-filter="stopped">Stopped <span id="count-stopped" class="filter-count"></span></button>
+        </div>
+        <div class="toolbar-right">
+            <input type="text" id="search-input" class="search-input" placeholder="Search services...">
+        </div>
+    </div>
+
+    <!-- Service Grid -->
+    <div id="service-grid" class="service-grid"></div>
+
+    <!-- Empty state -->
+    <div id="empty-state" class="empty-state" style="display:none;">
+        <span class="empty-icon">&#128230;</span>
+        <p>No services found</p>
+    </div>
+
+    <!-- Links view -->
+    <div id="links-view" style="display:none;"></div>
+</main>
+
+<!-- ===== Log Viewer (slide-out panel) ===== -->
+<div id="log-overlay" class="log-overlay" style="display:none;"></div>
+<div id="log-panel" class="log-panel">
+    <div class="log-header">
+        <h3 id="log-title">Container Logs</h3>
+        <button class="btn-icon log-close" id="btn-close-logs" title="Close">&#10005;</button>
+    </div>
+    <pre id="log-content" class="log-content"></pre>
+</div>
+
+<!-- ===== Group Editor Modal ===== -->
+<div id="group-modal" class="modal-overlay" style="display:none;">
+    <div class="modal">
+        <div class="modal-header">
+            <h3 id="group-modal-title">Create Group</h3>
+            <button class="btn-icon" id="btn-close-group-modal" title="Close">&#10005;</button>
+        </div>
+        <div class="modal-body">
+            <div class="form-group">
+                <label for="group-name-input">Group Name</label>
+                <input type="text" id="group-name-input" class="form-input" placeholder="e.g. Media Stack">
+            </div>
+            <div class="form-group">
+                <label>Services</label>
+                <input type="text" id="group-service-search" class="form-input" placeholder="Filter services...">
+                <div id="group-service-list" class="group-service-list"></div>
+            </div>
+        </div>
+        <div class="modal-footer">
+            <button class="btn btn-secondary" id="btn-cancel-group">Cancel</button>
+            <button class="btn btn-primary" id="btn-save-group">Save</button>
+        </div>
+    </div>
+</div>
+
+<!-- ===== Confirmation Modal ===== -->
+<div id="confirm-modal" class="modal-overlay" style="display:none;">
+    <div class="modal modal-sm">
+        <div class="modal-header">
+            <h3>Confirm Action</h3>
+        </div>
+        <div class="modal-body">
+            <p id="confirm-message"></p>
+        </div>
+        <div class="modal-footer">
+            <button class="btn btn-secondary" id="btn-confirm-cancel">Cancel</button>
+            <button class="btn btn-danger" id="btn-confirm-ok">Confirm</button>
+        </div>
+    </div>
+</div>
+
+<!-- ===== Toast Container ===== -->
+<div id="toast-container" class="toast-container"></div>
+
+<script src="/app.js"></script>
+</body>
+</html>

tests/test_agent_api.py

@@ -0,0 +1,71 @@
+import pytest
+from unittest.mock import patch, MagicMock
+from fastapi.testclient import TestClient
+
+
+@pytest.fixture
+def mock_ops():
+    with patch("agent.main.ops") as mock:
+        yield mock
+
+
+@pytest.fixture
+def client(mock_ops):
+    from agent.main import app
+    return TestClient(app)
+
+
+class TestHealthEndpoint:
+    def test_health_returns_ok(self, client, mock_ops):
+        mock_ops.get_health.return_value = {
+            "status": "ok", "hostname": "test-node", "containers_total": 5
+        }
+        resp = client.get("/health")
+        assert resp.status_code == 200
+        assert resp.json()["status"] == "ok"
+
+
+class TestListContainers:
+    def test_list_containers(self, client, mock_ops):
+        mock_ops.list_containers.return_value = [{
+            "id": "abc123", "name": "test-app", "status": "running",
+            "image": "nginx:latest", "created": "2026-03-05T00:00:00Z",
+            "uptime": "2h 30m", "is_swarm": False, "swarm_service": None,
+        }]
+        resp = client.get("/containers")
+        assert resp.status_code == 200
+        assert len(resp.json()) == 1
+        assert resp.json()[0]["name"] == "test-app"
+
+
+class TestContainerActions:
+    def test_start(self, client, mock_ops):
+        mock_ops.start_container.return_value = {"success": True, "message": "Started"}
+        assert client.post("/containers/abc123/start").json()["success"] is True
+
+    def test_stop(self, client, mock_ops):
+        mock_ops.stop_container.return_value = {"success": True, "message": "Stopped"}
+        assert client.post("/containers/abc123/stop").json()["success"] is True
+
+    def test_restart(self, client, mock_ops):
+        mock_ops.restart_container.return_value = {"success": True, "message": "Restarted"}
+        assert client.post("/containers/abc123/restart").json()["success"] is True
+
+
+class TestLogs:
+    def test_get_logs(self, client, mock_ops):
+        mock_ops.get_logs.return_value = {"container": "test-app", "logs": "line1\nline2"}
+        resp = client.get("/containers/abc123/logs?tail=50")
+        assert resp.status_code == 200
+        assert "line1" in resp.json()["logs"]
+
+    def test_get_logs_default_tail(self, client, mock_ops):
+        mock_ops.get_logs.return_value = {"container": "test-app", "logs": "logs"}
+        client.get("/containers/abc123/logs")
+        mock_ops.get_logs.assert_called_once_with("abc123", tail=200)
+
+
+class TestPull:
+    def test_pull(self, client, mock_ops):
+        mock_ops.pull_image.return_value = {"success": True, "message": "Pulled nginx:latest"}
+        assert client.post("/containers/abc123/pull").json()["success"] is True

tests/test_server_groups.py

@@ -0,0 +1,110 @@
+"""Tests for Group CRUD and bulk action endpoints.
+
+Follows TDD: tests written first, then implementation.
+"""
+
+import json
+import os
+import tempfile
+import pytest
+from fastapi.testclient import TestClient
+
+
+@pytest.fixture(autouse=True)
+def reset_config():
+    os.environ["CONFIG_PATH"] = os.path.join(os.path.dirname(__file__), "..", "config.json")
+    from server.config import _load_config
+    import server.config
+    server.config._config = None
+    yield
+
+
+@pytest.fixture
+def groups_file():
+    f = tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False)
+    json.dump({"groups": []}, f)
+    f.close()
+    os.environ["GROUPS_PATH"] = f.name
+    yield f.name
+    os.unlink(f.name)
+
+
+@pytest.fixture
+def client(groups_file):
+    from server.group_store import GroupStore
+    GroupStore._instance = None
+    from server.main import app
+    return TestClient(app)
+
+
+class TestListGroups:
+    def test_list_empty(self, client):
+        resp = client.get("/api/groups")
+        assert resp.status_code == 200
+        assert resp.json() == []
+
+
+class TestCreateGroup:
+    def test_create(self, client):
+        resp = client.post("/api/groups", json={
+            "id": "test", "name": "Test", "services": [{"node": "hf-pdocker-01", "container": "test"}]
+        })
+        assert resp.status_code == 200
+        assert resp.json()["id"] == "test"
+
+    def test_create_then_list(self, client):
+        client.post("/api/groups", json={"id": "g1", "name": "G1", "services": []})
+        resp = client.get("/api/groups")
+        assert len(resp.json()) == 1
+        assert resp.json()[0]["name"] == "G1"
+
+    def test_create_duplicate_409(self, client):
+        client.post("/api/groups", json={"id": "g1", "name": "G1", "services": []})
+        resp = client.post("/api/groups", json={"id": "g1", "name": "Dup", "services": []})
+        assert resp.status_code == 409
+
+
+class TestUpdateGroup:
+    def test_update(self, client):
+        client.post("/api/groups", json={"id": "g1", "name": "G1", "services": []})
+        resp = client.put("/api/groups/g1", json={
+            "id": "g1", "name": "Updated", "services": [{"node": "bart", "container": "jellyfin"}]
+        })
+        assert resp.status_code == 200
+        assert resp.json()["name"] == "Updated"
+        assert len(resp.json()["services"]) == 1
+
+    def test_update_nonexistent_404(self, client):
+        resp = client.put("/api/groups/nope", json={"id": "nope", "name": "N", "services": []})
+        assert resp.status_code == 404
+
+
+class TestDeleteGroup:
+    def test_delete(self, client):
+        client.post("/api/groups", json={"id": "g1", "name": "G1", "services": []})
+        resp = client.delete("/api/groups/g1")
+        assert resp.status_code == 200
+        assert len(client.get("/api/groups").json()) == 0
+
+    def test_delete_nonexistent_404(self, client):
+        resp = client.delete("/api/groups/nope")
+        assert resp.status_code == 404
+
+
+class TestPersistence:
+    def test_persists_to_file(self, client, groups_file):
+        client.post("/api/groups", json={"id": "g1", "name": "G1", "services": []})
+        with open(groups_file) as f:
+            data = json.load(f)
+        assert len(data["groups"]) == 1
+
+
+class TestBulkActions:
+    def test_start_nonexistent_group_404(self, client):
+        assert client.post("/api/groups/nope/start").status_code == 404
+
+    def test_stop_nonexistent_group_404(self, client):
+        assert client.post("/api/groups/nope/stop").status_code == 404
+
+    def test_restart_nonexistent_group_404(self, client):
+        assert client.post("/api/groups/nope/restart").status_code == 404

tests/test_server_proxy.py

@@ -0,0 +1,347 @@
+"""Tests for server API proxy routes — nodes and services.
+
+Mocks httpx.AsyncClient to avoid real network calls to agents.
+"""
+
+import os
+import pytest
+from unittest.mock import AsyncMock, patch, MagicMock
+
+os.environ.setdefault("CONFIG_PATH", os.path.join(
+    os.path.dirname(__file__), "..", "config.json"
+))
+
+from fastapi.testclient import TestClient
+
+
+@pytest.fixture(autouse=True)
+def _reset_config():
+    """Reset the config module cache between tests."""
+    import server.config as cfg
+    cfg._config = None
+    yield
+    cfg._config = None
+
+
+@pytest.fixture
+def client():
+    from server.main import app
+    return TestClient(app)
+
+
+# ---------------------------------------------------------------------------
+# Mock helpers
+# ---------------------------------------------------------------------------
+
+def _mock_response(json_data, status_code=200):
+    """Build a MagicMock httpx.Response with sync .json() method."""
+    resp = MagicMock()
+    resp.status_code = status_code
+    resp.json.return_value = json_data
+    resp.raise_for_status = MagicMock()
+    return resp
+
+
+def _make_mock_client(**side_effects):
+    """Create a mock httpx.AsyncClient context manager.
+
+    *side_effects* maps method names ('get', 'post') to lists of
+    return values or Exception instances for side_effect.
+    """
+    mock_instance = AsyncMock()
+    for method, values in side_effects.items():
+        getattr(mock_instance, method).side_effect = values
+    return mock_instance
+
+
+def _patch_httpx(module_path, mock_instance):
+    """Return a context-manager patch for httpx.AsyncClient in *module_path*."""
+    patcher = patch(f"{module_path}.httpx.AsyncClient")
+    MockClient = patcher.start()
+    MockClient.return_value.__aenter__ = AsyncMock(return_value=mock_instance)
+    MockClient.return_value.__aexit__ = AsyncMock(return_value=False)
+    return patcher
+
+
+# ---------------------------------------------------------------------------
+# GET /api/nodes
+# ---------------------------------------------------------------------------
+
+class TestGetNodes:
+
+    def test_returns_all_three_nodes(self, client):
+        """GET /api/nodes returns a list of 3 nodes with health status."""
+        mock = _make_mock_client(get=[
+            _mock_response({"status": "healthy", "hostname": "hf-pdocker-01", "containers_total": 10}),
+            _mock_response({"status": "healthy", "hostname": "hf-pdocker-02", "containers_total": 5}),
+            _mock_response({"status": "healthy", "hostname": "bart", "containers_total": 8}),
+        ])
+
+        p = _patch_httpx("server.routes.nodes", mock)
+        try:
+            resp = client.get("/api/nodes")
+        finally:
+            p.stop()
+
+        assert resp.status_code == 200
+        data = resp.json()
+        assert len(data) == 3
+        names = [n["name"] for n in data]
+        assert "hf-pdocker-01" in names
+        assert "hf-pdocker-02" in names
+        assert "bart" in names
+
+    def test_node_contains_health_fields(self, client):
+        """Each node has name, host, healthy, and containers_total."""
+        mock = _make_mock_client(get=[
+            _mock_response({"status": "healthy", "hostname": "hf-pdocker-01", "containers_total": 10}),
+            _mock_response({"status": "healthy", "hostname": "hf-pdocker-02", "containers_total": 5}),
+            _mock_response({"status": "healthy", "hostname": "bart", "containers_total": 8}),
+        ])
+
+        p = _patch_httpx("server.routes.nodes", mock)
+        try:
+            resp = client.get("/api/nodes")
+        finally:
+            p.stop()
+
+        data = resp.json()
+        node = data[0]
+        assert "name" in node
+        assert "host" in node
+        assert "healthy" in node
+        assert "containers_total" in node
+
+    def test_unhealthy_node_on_agent_error(self, client):
+        """If an agent is unreachable, the node shows healthy=False."""
+        mock = _make_mock_client(get=[
+            _mock_response({"status": "healthy", "hostname": "hf-pdocker-01", "containers_total": 10}),
+            Exception("Connection refused"),
+            _mock_response({"status": "healthy", "hostname": "bart", "containers_total": 8}),
+        ])
+
+        p = _patch_httpx("server.routes.nodes", mock)
+        try:
+            resp = client.get("/api/nodes")
+        finally:
+            p.stop()
+
+        data = resp.json()
+        assert resp.status_code == 200
+        unhealthy = [n for n in data if n["name"] == "hf-pdocker-02"]
+        assert len(unhealthy) == 1
+        assert unhealthy[0]["healthy"] is False
+        assert unhealthy[0]["error"] is not None
+
+
+# ---------------------------------------------------------------------------
+# GET /api/services
+# ---------------------------------------------------------------------------
+
+class TestGetServices:
+
+    def test_returns_containers_from_all_nodes(self, client):
+        """GET /api/services aggregates containers from all nodes, adding node field."""
+        containers_node1 = [
+            {"id": "abc1", "name": "app1", "status": "running",
+             "image": "nginx:latest", "created": "2026-03-05T00:00:00Z",
+             "uptime": "1h", "is_swarm": False, "swarm_service": None},
+        ]
+        containers_node2 = [
+            {"id": "abc2", "name": "app2", "status": "running",
+             "image": "redis:latest", "created": "2026-03-05T00:00:00Z",
+             "uptime": "2h", "is_swarm": False, "swarm_service": None},
+        ]
+        containers_node3 = [
+            {"id": "abc3", "name": "app3", "status": "exited",
+             "image": "postgres:15", "created": "2026-03-05T00:00:00Z",
+             "uptime": None, "is_swarm": False, "swarm_service": None},
+        ]
+
+        mock = _make_mock_client(get=[
+            _mock_response(containers_node1),
+            _mock_response(containers_node2),
+            _mock_response(containers_node3),
+        ])
+
+        p = _patch_httpx("server.routes.services", mock)
+        try:
+            resp = client.get("/api/services")
+        finally:
+            p.stop()
+
+        assert resp.status_code == 200
+        data = resp.json()
+        assert len(data) == 3
+        nodes = [c["node"] for c in data]
+        assert "hf-pdocker-01" in nodes
+        assert "hf-pdocker-02" in nodes
+        assert "bart" in nodes
+
+    def test_skips_unreachable_node(self, client):
+        """If a node agent is down, its containers are omitted."""
+        containers_node1 = [
+            {"id": "abc1", "name": "app1", "status": "running",
+             "image": "nginx:latest", "created": "2026-03-05T00:00:00Z",
+             "uptime": "1h", "is_swarm": False, "swarm_service": None},
+        ]
+
+        mock = _make_mock_client(get=[
+            _mock_response(containers_node1),
+            Exception("Connection refused"),
+            _mock_response([]),
+        ])
+
+        p = _patch_httpx("server.routes.services", mock)
+        try:
+            resp = client.get("/api/services")
+        finally:
+            p.stop()
+
+        assert resp.status_code == 200
+        data = resp.json()
+        assert len(data) == 1
+
+
+# ---------------------------------------------------------------------------
+# POST /api/services/{node}/{container_id}/start
+# ---------------------------------------------------------------------------
+
+class TestStartService:
+
+    def test_start_proxies_to_agent(self, client):
+        """POST start sends request to correct agent and returns result."""
+        mock = _make_mock_client(post=[
+            _mock_response({"success": True, "message": "Started container app1"}),
+        ])
+
+        p = _patch_httpx("server.routes.services", mock)
+        try:
+            resp = client.post("/api/services/hf-pdocker-01/abc123/start")
+        finally:
+            p.stop()
+
+        assert resp.status_code == 200
+        assert resp.json()["success"] is True
+
+    def test_start_invalid_node_returns_404(self, client):
+        """POST start with unknown node returns 404."""
+        resp = client.post("/api/services/nonexistent-node/abc123/start")
+        assert resp.status_code == 404
+
+
+# ---------------------------------------------------------------------------
+# POST /api/services/{node}/{container_id}/stop
+# ---------------------------------------------------------------------------
+
+class TestStopService:
+
+    def test_stop_proxies_to_agent(self, client):
+        """POST stop sends request to correct agent."""
+        mock = _make_mock_client(post=[
+            _mock_response({"success": True, "message": "Stopped container app1"}),
+        ])
+
+        p = _patch_httpx("server.routes.services", mock)
+        try:
+            resp = client.post("/api/services/bart/abc123/stop")
+        finally:
+            p.stop()
+
+        assert resp.status_code == 200
+        assert resp.json()["success"] is True
+
+    def test_stop_invalid_node_returns_404(self, client):
+        resp = client.post("/api/services/nonexistent-node/abc123/stop")
+        assert resp.status_code == 404
+
+
+# ---------------------------------------------------------------------------
+# POST /api/services/{node}/{container_id}/restart
+# ---------------------------------------------------------------------------
+
+class TestRestartService:
+
+    def test_restart_proxies_to_agent(self, client):
+        mock = _make_mock_client(post=[
+            _mock_response({"success": True, "message": "Restarted container app1"}),
+        ])
+
+        p = _patch_httpx("server.routes.services", mock)
+        try:
+            resp = client.post("/api/services/hf-pdocker-02/abc123/restart")
+        finally:
+            p.stop()
+
+        assert resp.status_code == 200
+        assert resp.json()["success"] is True
+
+    def test_restart_invalid_node_returns_404(self, client):
+        resp = client.post("/api/services/nonexistent-node/abc123/restart")
+        assert resp.status_code == 404
+
+
+# ---------------------------------------------------------------------------
+# GET /api/services/{node}/{container_id}/logs
+# ---------------------------------------------------------------------------
+
+class TestGetServiceLogs:
+
+    def test_logs_proxies_to_agent(self, client):
+        mock = _make_mock_client(get=[
+            _mock_response({"container": "test-app", "logs": "log line 1\nlog line 2"}),
+        ])
+
+        p = _patch_httpx("server.routes.services", mock)
+        try:
+            resp = client.get("/api/services/hf-pdocker-01/abc123/logs")
+        finally:
+            p.stop()
+
+        assert resp.status_code == 200
+        assert "log line 1" in resp.json()["logs"]
+
+    def test_logs_passes_tail_param(self, client):
+        mock = _make_mock_client(get=[
+            _mock_response({"container": "test-app", "logs": "log data"}),
+        ])
+
+        p = _patch_httpx("server.routes.services", mock)
+        try:
+            resp = client.get("/api/services/hf-pdocker-01/abc123/logs?tail=50")
+        finally:
+            p.stop()
+
+        assert resp.status_code == 200
+        # Verify tail param was passed through to the agent
+        call_args = mock.get.call_args
+        assert "tail" in str(call_args)
+
+    def test_logs_invalid_node_returns_404(self, client):
+        resp = client.get("/api/services/nonexistent-node/abc123/logs")
+        assert resp.status_code == 404
+
+
+# ---------------------------------------------------------------------------
+# POST /api/services/{node}/{container_id}/pull
+# ---------------------------------------------------------------------------
+
+class TestPullService:
+
+    def test_pull_proxies_to_agent(self, client):
+        mock = _make_mock_client(post=[
+            _mock_response({"success": True, "message": "Pulled nginx:latest"}),
+        ])
+
+        p = _patch_httpx("server.routes.services", mock)
+        try:
+            resp = client.post("/api/services/bart/abc123/pull")
+        finally:
+            p.stop()
+
+        assert resp.status_code == 200
+        assert resp.json()["success"] is True
+
+    def test_pull_invalid_node_returns_404(self, client):
+        resp = client.post("/api/services/nonexistent-node/abc123/pull")
+        assert resp.status_code == 404