fix(monitoring): Prometheus deployment source-of-truth + Recreate strategy (DEV-583)

Adds apps/monitoring/prometheus-deployment.yaml capturing the live PVC/Service/
Deployment for monitoring/prometheus. Two behaviour changes vs the previous
live state:

- strategy: RollingUpdate -> Recreate. prometheus-data-encrypted is RWO on
  hcloud-volumes-encrypted; a rolling update deadlocks because the surge pod
  cannot attach the PVC while the old pod holds it. This has been jamming the
  weekly OS-update preflight (see DEV-582). Same fix pattern as loki
  (loki-deployment.yaml) and opencloud (DEV-538).
- pod-template annotation backup.velero.io/backup-volumes-excludes:
  prometheus-storage. Restores the annotation that rev 6 tried to add before
  the RollingUpdate deadlock forced a rollback in DEV-582.

Applied and verified: rollout succeeded, /-/ready and /-/healthy both green,
pod running on k3s-worker-6 after PVC re-attach.

Co-Authored-By: Paperclip <noreply@paperclip.ing>

apps/monitoring/prometheus-deployment.yaml

@@ -0,0 +1,110 @@
+---
+# Prometheus data volume — RWO on hcloud-volumes-encrypted (fsn1). Because the
+# volume is RWO, the Deployment below must use Recreate; a rolling update would
+# spin up a surge pod that cannot mount the volume while the old pod holds it
+# and the rollout would stall at Init:0/1. See DEV-583 / mirrors DEV-538.
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: prometheus-data-encrypted
+  namespace: monitoring
+spec:
+  accessModes:
+    - ReadWriteOnce
+  storageClassName: hcloud-volumes-encrypted
+  resources:
+    requests:
+      storage: 50Gi
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: prometheus
+  namespace: monitoring
+  labels:
+    app: prometheus
+spec:
+  type: ClusterIP
+  ports:
+    - port: 9090
+      targetPort: 9090
+      name: web
+  selector:
+    app: prometheus
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: prometheus
+  namespace: monitoring
+  labels:
+    app: prometheus
+spec:
+  replicas: 1
+  # prometheus-data-encrypted is RWO on hcloud-volumes-encrypted, so a rolling
+  # update deadlocks (new pod cannot attach the PVC while the old pod holds
+  # it). Recreate drops the old pod first so the CSI detaches the volume
+  # before the new pod tries to attach it. Same pattern as loki/opencloud.
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app: prometheus
+  template:
+    metadata:
+      labels:
+        app: prometheus
+      annotations:
+        # Exclude the prometheus TSDB volume from Velero fs-backup — TSDB is
+        # backed up separately by prometheus-backup-cronjob and streaming it
+        # through Kopia would double-copy tens of GiB on every backup.
+        backup.velero.io/backup-volumes-excludes: prometheus-storage
+    spec:
+      serviceAccountName: prometheus
+      initContainers:
+        - name: fix-permissions
+          image: busybox:latest
+          command:
+            - sh
+            - -c
+            - chown -R 65534:65534 /prometheus && chmod -R 755 /prometheus
+          volumeMounts:
+            - name: prometheus-storage
+              mountPath: /prometheus
+      containers:
+        - name: prometheus
+          image: prom/prometheus:v2.47.0
+          args:
+            - '--config.file=/etc/prometheus/prometheus.yml'
+            - '--storage.tsdb.path=/prometheus'
+            - '--storage.tsdb.retention.time=30d'
+            - '--web.console.libraries=/usr/share/prometheus/console_libraries'
+            - '--web.console.templates=/usr/share/prometheus/consoles'
+            - '--web.enable-lifecycle'
+          ports:
+            - containerPort: 9090
+              name: web
+          volumeMounts:
+            - name: prometheus-config
+              mountPath: /etc/prometheus
+            - name: prometheus-rules
+              mountPath: /etc/prometheus/rules
+            - name: prometheus-storage
+              mountPath: /prometheus
+          resources:
+            requests:
+              cpu: 500m
+              memory: 1Gi
+            limits:
+              cpu: 1000m
+              memory: 2Gi
+      volumes:
+        - name: prometheus-config
+          configMap:
+            name: prometheus-config
+        - name: prometheus-rules
+          configMap:
+            name: prometheus-rules
+        - name: prometheus-storage
+          persistentVolumeClaim:
+            claimName: prometheus-data-encrypted