From 8dffba2c521c5d63b8c3bf766756115b79fae254 Mon Sep 17 00:00:00 2001 From: CTO Date: Sun, 16 Aug 2026 20:15:01 +0000 Subject: [PATCH] fix(forgejo-runner): pin to nodes with docker installed (DEV-498) Rolling OS updates removed docker.io from all workers, breaking the runner's hostPath mount of /var/run/docker.sock (type=Socket check). - Install docker.io on all 5 worker nodes and enable the systemd unit. - Label the nodes basicstack.de/docker=true so the dependency is explicit. - Add nodeSelector to the Deployment so scheduling fails fast on non-labeled nodes instead of hanging in ContainerCreating. - Document the node prerequisite in the README. Co-Authored-By: Paperclip --- apps/forgejo-runner/README.md | 35 +++++++++++++++---- .../forgejo-runner-deployment.yaml | 2 ++ 2 files changed, 31 insertions(+), 6 deletions(-) diff --git a/apps/forgejo-runner/README.md b/apps/forgejo-runner/README.md index ca52deb..b5d6abf 100644 --- a/apps/forgejo-runner/README.md +++ b/apps/forgejo-runner/README.md @@ -50,17 +50,40 @@ kubectl apply -f apps/app-forgejo-runner.yaml ## Runner Configuration -The runner is deployed as a StatefulSet with Docker-in-Docker (dind) sidecar for proper isolation and state management. +The runner is deployed as a Deployment (single replica) that bind-mounts the +host's `/var/run/docker.sock` to execute job containers. This requires the +Docker Engine (package `docker.io`) to be installed and running on the target +node — see **Node prerequisites** below. Configuration: -- **Deployment type**: StatefulSet (stable pod identity, persistent storage) -- **Docker execution**: Docker-in-Docker sidecar (privileged init container) +- **Deployment type**: Deployment (replicas=1) +- **Docker execution**: Host Docker socket (`/var/run/docker.sock`) - **Concurrent jobs**: 2 (configurable via config.yaml) - **Labels**: ubuntu-latest:docker://node:24-bookworm, ubuntu-22.04:docker://node:24-bookworm - **Forgejo URL**: https://forgejo.basicstack.de (external URL for proper webhook/API access) -- **Persistent volumes**: - - runner-data (1Gi): Runner registration and config - - docker-data (20Gi): Docker image cache +- **Node selector**: `basicstack.de/docker=true` — schedules only on nodes with the + Docker Engine installed. See **Node prerequisites** below. + +## Node prerequisites (required) + +The runner uses the host's Docker daemon. Every worker node that should be +eligible to run the runner MUST have `docker.io` installed, the `docker` +systemd unit enabled, and be labeled `basicstack.de/docker=true`. Bootstrap a +worker with: + +```bash +ssh root@ +DEBIAN_FRONTEND=noninteractive apt-get update +DEBIAN_FRONTEND=noninteractive apt-get install -y docker.io +systemctl enable --now docker +# from the control plane: +kubectl label node basicstack.de/docker=true --overwrite +``` + +Rolling OS updates that reimage a node will remove Docker. Re-run the bootstrap +after any node reimage. Without Docker, the runner pod fails to mount +`/var/run/docker.sock` (hostPath type check for `Socket` fails); the +nodeSelector prevents that scheduling mistake by pinning to labeled nodes. ## Troubleshooting diff --git a/apps/forgejo-runner/forgejo-runner-deployment.yaml b/apps/forgejo-runner/forgejo-runner-deployment.yaml index 52f0ee8..6425e95 100644 --- a/apps/forgejo-runner/forgejo-runner-deployment.yaml +++ b/apps/forgejo-runner/forgejo-runner-deployment.yaml @@ -14,6 +14,8 @@ spec: app: forgejo-runner spec: serviceAccountName: forgejo-runner + nodeSelector: + basicstack.de/docker: "true" containers: - name: runner image: code.forgejo.org/forgejo/runner:4.0.1