Public DNS for paperclip.home.imicros.de returns the LAN IP 10.0.0.2, so the
drill's webhook POST was landing with HTTP 000 from Hetzner. The home reverse
proxy at 46.225.155.185 already serves that vhost publicly, so pinning the
hostname to the home public IP in the CronJob's hostAliases lets the webhook
fire the Paperclip routine without any home-network or SealedSecret changes.
Verified end-to-end: bearer POST via --resolve returns HTTP 202 with
status=issue_created (run issue b07fc035 created under DEV-578).
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- CronJob at 0 4 1 * * restores the latest openwebui-manifests backup into
a transient restore-test-<ts> namespace, diffs core resource-kind counts
against the live openwebui namespace, and deletes the target namespace.
- ServiceAccount + ClusterRole scoped to velero.io + namespace + configmap
ops. Script enforces the 'restore-test-' name prefix before deleting a
namespace. Cluster-wide 'view' bound for the diff.
- Failure surfaces via kube_job_status_failed (DEV-574 alert stack) and via
ConfigMap velero-restore-drill-last-result written on every run.
- Optional Paperclip webhook Secret is sealed but unreachable from Hetzner
right now (home Paperclip is LAN-only); wired as optional secretKeyRef so
the drill runs regardless. Connectivity is a follow-up.
- Argo Application velero-restore-drill points at velero/restore-drill/.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Adds Prometheus scrape of Velero's :8085 metrics and four alerts
(VeleroBackupFailed, VeleroBackupPartial, VeleroBackupMissed,
VeleroStorageLocationUnavailable) labeled team=infra severity=warning.
VeleroStorageLocationUnavailable substitutes for the spec's
VeleroRepoUnhealthy since Velero v1.18.1 does not expose
velero_backup_repository_available (see comment in the rule).
Wired via a new ArgoCD Application (velero-monitoring) that syncs
only servicemonitor.yaml + prometheusrule.yaml from basicstack-repo/velero/.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- velero/schedules/: cluster-all @ 03:00 (30d TTL, manifests only),
per-namespace <ns>-manifests @ 03:15+n min (30 in-scope namespaces,
30d TTL, no fs backup), and per-namespace <ns>-data @ 03:45+n min
for the whitelist (pocket-id, vaultwarden, vikunja, passbolt;
14d TTL, defaultVolumesToFsBackup=true).
- All schedules exclude velero, kube-system, kube-node-lease,
kube-public so the Kopia password Secret never lands in S3.
- apps/app-velero-schedules.yaml: sibling Argo app syncing that
directory into namespace velero with prune+selfHeal.
- opencloud, harbor(registry+trivy): pod-template annotation
backup.velero.io/backup-volumes-excludes as a safety net so blob-
heavy PVCs are never fs-backed-up even under a manual defaultVolumes
run. nextcloud/monitoring workloads are not Argo-tracked yet — they
were patched live in the same commit but need a follow-up to source-
control them.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
