basicstack/stack.basicstack.de
2
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions
CD/CI deployment manifests and configurations for basicstack.de cluster
286 commits 7 branches 0 tags 2.5 MiB
Find a file
CTO Agent 0c1c05fc50 feat(monitoring): retire legacy backup-volumes CronJob + backup-storage PVC (DEV-489) 
The restic pipeline (loki/grafana/k8s-resources) is proven end-to-end
after the DEV-488 restore drill, so the legacy rsync/tar pipeline into
the 100 Gi local-path `backup-storage` PVC is removed.

- Delete `apps/monitoring/backup-volumes-cronjob.yaml`.
- Remove the DEV-483 bridge `nodeSelector: k3s-worker-2` from
  `apps/monitoring/loki-deployment.yaml`. Loki's data protection now
  runs via `backup-loki-restic`, which follows the pod via podAffinity
  regardless of which node the RWO CSI volume attaches on. The
  `Recreate` rollout strategy stays — it is unrelated (avoids the
  attach-deadlock during a rollout). Resolves the RWO/nodeSelector
  attach race that was blocking DEV-478 weekly OS updates.
- Update `apps/monitoring/README.md` to drop the `backup-volumes`
  section, link the restic restore runbook, and record the pin
  removal.
- Clean stale coexistence comments in the restic/prometheus CronJob
  manifests now that the legacy job is gone.

Cluster-side (already applied out-of-band, since these manifests are
`kubectl apply`-based, not Argo-managed):
- `kubectl -n monitoring delete cronjob backup-volumes` -> NotFound.
- `kubectl -n monitoring delete pvc backup-storage` -> gone; local-path
  PV `pvc-d0db0ba9-8f89-4f66-9e65-d573ebe1085a` reclaimed automatically
  (Delete policy). Two stale pre-DEV-487 `backup-k8s-resources` job
  pods that still referenced the PVC were deleted to release the
  `pvc-protection` finalizer.
- `kubectl -n monitoring apply -f loki-deployment.yaml` -> Recreate
  rollout, new pod Ready in ~60s, no nodeSelector on the new spec.
- No `VolumeAttachment` for the retired PV.
- Restic CronJobs (`backup-loki-restic`, `backup-grafana-restic`,
  `backup-k8s-resources`, `prometheus-backup`) intact.

Pre-delete snapshots retained on k3s-cp-1 under
`/root/dev489-snapshots-20260816T155411Z/` for post-mortem.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-08-16 15:58:43 +00:00
apps feat(monitoring): retire legacy backup-volumes CronJob + backup-storage PVC (DEV-489) 2026-08-16 15:58:43 +00:00
docs docs(monitoring): add restic restore runbook + first drill log (DEV-488) 2026-08-16 15:47:45 +00:00
infrastructure Add weekly rolling OS-update procedure for k3s nodes (DEV-462) 2026-08-09 15:55:39 +00:00
.gitignore Convert all secrets to SealedSecrets for enhanced security 2026-07-01 18:38:27 +00:00
add-user-andreas.ldif OpenCloud: Use native bash substitution in config file 2026-07-05 14:03:49 +00:00
DEV-349-progress.md Consolidate Forgejo backup into forgejo namespace 2026-07-26 09:35:31 +00:00
README.md Add comprehensive Argo CD documentation 2026-07-12 10:04:51 +00:00

README.md

stack.basicstack.de

CD/CI deployment manifests and configurations for the basicstack.de Kubernetes cluster.

Repository Structure

stack.basicstack.de/
├── apps/                    # Application deployments
│   ├── stalwart/           # Stalwart mail server (example)
│   └── forgejo/            # Forgejo Git service (placeholder)
├── infrastructure/          # Infrastructure-level configurations
│   ├── networking/         # Network policies, ingress, DNS
│   └── monitoring/         # Monitoring, logging, observability
└── docs/                   # Documentation and guides

Purpose

This repository serves as the central source of truth for all deployment configurations targeting the basicstack.de Kubernetes cluster. It follows GitOps principles where infrastructure and application state is declaratively defined and version-controlled.

Directory Details

apps/

Contains deployment configurations for individual applications and services running on the cluster. Each application should have its own subdirectory with:

  • Kubernetes manifests (Deployments, StatefulSets, Services, etc.)
  • Helm values files
  • Configuration files
  • Application-specific documentation

Example: The stalwart/ directory contains the complete deployment configuration for the Stalwart mail server, including multiple deployment variants, monitoring setup, and operational guides.

infrastructure/

Contains cluster-wide infrastructure configurations:

  • networking/: Ingress controllers, network policies, DNS configurations, load balancers
  • monitoring/: Prometheus, Grafana, logging infrastructure, observability tools

docs/

General documentation including:

  • Deployment procedures
  • Cluster architecture
  • Troubleshooting guides
  • Best practices

GitOps with Argo CD

This repository is managed via Argo CD, the GitOps deployment platform for the cluster.

All changes pushed to the main branch are automatically synchronized to the cluster. Applications are defined in apps/app-*.yaml files and reference subdirectories for their manifests.

For details on managing applications, repository credentials, troubleshooting, and emergency procedures, see the Argo CD documentation.

Getting Started

  1. Clone this repository
  2. Review the example Stalwart deployment in apps/stalwart/
  3. Follow the pattern for new application deployments
  4. Ensure all manifests are tested before committing
  5. Argo CD will automatically sync changes to the cluster (or use manual sync for critical changes)

Contributing

All changes should be:

  1. Committed with clear, descriptive messages
  2. Tested in a development environment when possible
  3. Documented appropriately
  4. Reviewed before deployment to production

Cluster Information

  • Cluster: basicstack.de
  • Platform: K3s on Hetzner Cloud
  • Namespace Strategy: One namespace per application (recommended)
  • Ingress: Traefik (default K3s ingress controller)