basicstack/stack.basicstack.de
2
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions
stack.basicstack.de/apps/monitoring
History
CTO Agent 95de27437b feat(monitoring): seal restic-password into monitoring-s3-backup (DEV-484) 
Add key restic-password to SealedSecret monitoring-s3-backup so the
upcoming restic CronJobs (DEV-485/486/487) can mount it as
RESTIC_PASSWORD without any further secret rotation. Value is a fresh
32-byte openssl-random base64 password, sealed strict-scope against
the running sealed-secrets-controller. Server-side dry-run of the
new SealedSecret reports "configured".

Bucket write probe against s3://basicstack-backup/ succeeded from an
in-cluster amazon/aws-cli pod using the existing access-key/secret-key
in the same SealedSecret (ls, put, stat, delete).

Recovery-key copy for Passbolt is held root-only on the control plane
at /root/dev-484/restic-password.txt pending the human-side Passbolt
paste; that step tracks as follow-up.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-08-16 13:22:10 +00:00
..
backup-k8s-resources-cronjob.yaml fix(backups): Repair three broken CronJobs blocking weekly OS updates (DEV-464) 2026-08-09 16:51:30 +00:00
backup-volumes-cronjob.yaml fix(backups): Repair three broken CronJobs blocking weekly OS updates (DEV-464) 2026-08-09 16:51:30 +00:00
loki-deployment.yaml fix(monitoring): Loki uses Recreate strategy for RWO PVC rollouts (DEV-483) 2026-08-16 12:54:26 +00:00
prometheus-backup-cronjob.yaml fix(monitoring): prune Prometheus S3 backups by prefix-date, not file mtime (DEV-465) 2026-08-09 17:40:50 +00:00
prometheus-backup-sealed.yaml feat(monitoring): seal restic-password into monitoring-s3-backup (DEV-484) 2026-08-16 13:22:10 +00:00
README.md Add dedicated Prometheus data backup CronJob (DEV-465) 2026-08-09 17:18:21 +00:00

README.md

monitoring — backup CronJobs

Manifests recording the cluster-side monitoring backup CronJobs that were previously applied out-of-band. These files are the authoritative source (kubectl apply -f apps/monitoring/). See DEV-464 for the repair context.

  • backup-k8s-resources-cronjob.yaml — daily dump of Kubernetes resources into backup-storage PVC.
  • backup-volumes-cronjob.yaml — daily rsync/tar of Grafana + Loki PVCs into backup-storage. Prometheus data is NOT included here — it lives on a different node (see below).
  • prometheus-backup-cronjob.yaml + prometheus-backup-sealed.yaml — dedicated Prometheus data backup that streams prometheus-data-encrypted to Hetzner S3 via rclone. Co-schedules with the Prometheus pod via podAffinity so the RWO PVC attaches on the same node (DEV-465).

The backup-storage PVC (100Gi, local-path, bound to k3s-worker-2) is the shared destination for backup-k8s-resources and backup-volumes.

backup-k8s-resources and backup-volumes pin themselves to k3s-worker-2 via nodeSelector because that is the node that holds all destination + source PVCs used there. prometheus-backup follows the Prometheus pod via podAffinity, writing to Hetzner S3 (hetzner-s3:basicstack-backup/prometheus/) so it stays independent of backup-storage.