0c1c05fc50
The restic pipeline (loki/grafana/k8s-resources) is proven end-to-end after the DEV-488 restore drill, so the legacy rsync/tar pipeline into the 100 Gi local-path `backup-storage` PVC is removed. - Delete `apps/monitoring/backup-volumes-cronjob.yaml`. - Remove the DEV-483 bridge `nodeSelector: k3s-worker-2` from `apps/monitoring/loki-deployment.yaml`. Loki's data protection now runs via `backup-loki-restic`, which follows the pod via podAffinity regardless of which node the RWO CSI volume attaches on. The `Recreate` rollout strategy stays — it is unrelated (avoids the attach-deadlock during a rollout). Resolves the RWO/nodeSelector attach race that was blocking DEV-478 weekly OS updates. - Update `apps/monitoring/README.md` to drop the `backup-volumes` section, link the restic restore runbook, and record the pin removal. - Clean stale coexistence comments in the restic/prometheus CronJob manifests now that the legacy job is gone. Cluster-side (already applied out-of-band, since these manifests are `kubectl apply`-based, not Argo-managed): - `kubectl -n monitoring delete cronjob backup-volumes` -> NotFound. - `kubectl -n monitoring delete pvc backup-storage` -> gone; local-path PV `pvc-d0db0ba9-8f89-4f66-9e65-d573ebe1085a` reclaimed automatically (Delete policy). Two stale pre-DEV-487 `backup-k8s-resources` job pods that still referenced the PVC were deleted to release the `pvc-protection` finalizer. - `kubectl -n monitoring apply -f loki-deployment.yaml` -> Recreate rollout, new pod Ready in ~60s, no nodeSelector on the new spec. - No `VolumeAttachment` for the retired PV. - Restic CronJobs (`backup-loki-restic`, `backup-grafana-restic`, `backup-k8s-resources`, `prometheus-backup`) intact. Pre-delete snapshots retained on k3s-cp-1 under `/root/dev489-snapshots-20260816T155411Z/` for post-mortem. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|---|---|---|
| .. | ||
| argocd | ||
| basicstack-org | ||
| bookstack | ||
| directus | ||
| dozzle | ||
| forgejo | ||
| forgejo-runner | ||
| harbor | ||
| headlamp | ||
| monitoring | ||
| opencloud | ||
| pangolin | ||
| paperclip | ||
| passbolt | ||
| pocket-id | ||
| stalwart | ||
| app-argocd.yaml | ||
| app-basicstack-org.yaml | ||
| app-bookstack.yaml | ||
| app-directus.yaml | ||
| app-dozzle.yaml | ||
| app-forgejo-runner.yaml | ||
| app-forgejo.yaml | ||
| app-harbor.yaml | ||
| app-headlamp.yaml | ||
| app-opencloud.yaml | ||
| app-pangolin.yaml | ||
| app-paperclip.yaml | ||
| app-passbolt.yaml | ||
| app-pocket-id.yaml | ||
| app-stalwart.yaml | ||
| ARGOCD-MIGRATION.md | ||
| README.md | ||
Applications
This directory contains deployment configurations for all applications running on the basicstack.de cluster.
ArgoCD Application Management
Each application has two types of files:
app-<name>.yaml: ArgoCD Application manifest that tells ArgoCD to sync the app subdirectory<name>/: Application-specific Kubernetes manifests and configuration
The app-*.yaml files are synced by ArgoCD and create/manage the corresponding Application resources.
Each application's manifests in its subdirectory are then synced by its Application resource.
Structure
Each application should have its own subdirectory containing:
- Kubernetes manifests: Deployment, StatefulSet, Service, ConfigMap, Secret definitions
- Helm values: If using Helm charts, include values.yaml files
- Configuration files: Application-specific configs (TOML, JSON, YAML)
- Documentation: README or guide specific to the application deployment
- Patches: Any kubectl patches or modifications needed
Example: Stalwart
The stalwart/ directory serves as a reference implementation, containing:
- Multiple deployment variants (basic, with OIDC, etc.)
- Helm values files
- Monitoring dashboard configurations
- Backup/restore procedures
- Operational documentation
Adding a New Application
- Create a new directory:
apps/<application-name>/ - Add your Kubernetes manifests
- Include a README.md explaining:
- What the application does
- How to deploy it
- Configuration options
- Troubleshooting steps
- Test the deployment in a dev environment
- Commit with a descriptive message
Naming Conventions
- Directory names: lowercase, hyphen-separated (e.g.,
my-app) - Manifest files: descriptive names indicating resource type (e.g.,
deployment.yaml,service.yaml) - Use consistent naming across applications