The weekly rolling OS-update cycle has been silently stripping docker.io from worker nodes (DEV-498), breaking the Forgejo runner whose hostPath mount for /var/run/docker.sock requires the socket to exist. Fix in two layers: - Defensive pin: update-node.sh now runs `apt-mark manual docker.io` in the apt phase (whenever it is installed) so `apt-get autoremove --purge` cannot silently drop it during subsequent upgrades. - Post-reboot reconciliation: new `ensure-node-docker.sh` installs docker.io if missing, enables + starts the systemd unit, waits for /var/run/docker.sock, and re-applies the `basicstack.de/docker=true` label. Wired into update-node.sh between kubelet-Ready and uncordon. No-op on nodes without the label (safe for cp-1 and the update runner). Verified idempotent against all 5 labeled workers; `apt-mark manual docker.io` now set on every worker (survived across reboots by design). Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|---|---|---|
| apps | ||
| docs | ||
| infrastructure | ||
| .gitignore | ||
| add-user-andreas.ldif | ||
| DEV-349-progress.md | ||
| README.md | ||
stack.basicstack.de
CD/CI deployment manifests and configurations for the basicstack.de Kubernetes cluster.
Repository Structure
stack.basicstack.de/
├── apps/ # Application deployments
│ ├── stalwart/ # Stalwart mail server (example)
│ └── forgejo/ # Forgejo Git service (placeholder)
├── infrastructure/ # Infrastructure-level configurations
│ ├── networking/ # Network policies, ingress, DNS
│ └── monitoring/ # Monitoring, logging, observability
└── docs/ # Documentation and guides
Purpose
This repository serves as the central source of truth for all deployment configurations targeting the basicstack.de Kubernetes cluster. It follows GitOps principles where infrastructure and application state is declaratively defined and version-controlled.
Directory Details
apps/
Contains deployment configurations for individual applications and services running on the cluster. Each application should have its own subdirectory with:
- Kubernetes manifests (Deployments, StatefulSets, Services, etc.)
- Helm values files
- Configuration files
- Application-specific documentation
Example: The stalwart/ directory contains the complete deployment configuration for the Stalwart mail server, including multiple deployment variants, monitoring setup, and operational guides.
infrastructure/
Contains cluster-wide infrastructure configurations:
- networking/: Ingress controllers, network policies, DNS configurations, load balancers
- monitoring/: Prometheus, Grafana, logging infrastructure, observability tools
docs/
General documentation including:
- Deployment procedures
- Cluster architecture
- Troubleshooting guides
- Best practices
GitOps with Argo CD
This repository is managed via Argo CD, the GitOps deployment platform for the cluster.
- Argo CD UI: https://argo.basicstack.de
- Authentication: Pocket ID SSO (https://auth.basicstack.de)
- Documentation: docs/argocd.md
All changes pushed to the main branch are automatically synchronized to the cluster. Applications are defined in apps/app-*.yaml files and reference subdirectories for their manifests.
For details on managing applications, repository credentials, troubleshooting, and emergency procedures, see the Argo CD documentation.
Getting Started
- Clone this repository
- Review the example Stalwart deployment in
apps/stalwart/ - Follow the pattern for new application deployments
- Ensure all manifests are tested before committing
- Argo CD will automatically sync changes to the cluster (or use manual sync for critical changes)
Contributing
All changes should be:
- Committed with clear, descriptive messages
- Tested in a development environment when possible
- Documented appropriately
- Reviewed before deployment to production
Cluster Information
- Cluster: basicstack.de
- Platform: K3s on Hetzner Cloud
- Namespace Strategy: One namespace per application (recommended)
- Ingress: Traefik (default K3s ingress controller)