3b4146de65
- infrastructure/OS_UPDATE_PROCEDURE.md: agent-facing rolling update procedure (drain -> apt -> reboot -> verify -> uncordon -> health -> next). Explicit MUST NOT list around k3s config, PVs, and manifests. - infrastructure/OS_UPDATE_ROUTINE.md: describes the weekly Paperclip routine (Sun 03:00 Europe/Berlin) that fires this procedure. - infrastructure/scripts/os-update/: cluster-health.sh, update-node.sh, os-update.sh, README. Enforces the same guardrails in code: workers-first-then-CP, one node at a time, no --force drains, halts on reboot/kubelet/health failure, never touches k3s config. Co-Authored-By: Paperclip <noreply@paperclip.ing>
50 lines
2 KiB
Markdown
50 lines
2 KiB
Markdown
# Infrastructure
|
|
|
|
This directory contains cluster-wide infrastructure configurations that support all applications.
|
|
|
|
## Operational procedures (agent-facing)
|
|
|
|
- **[OS_UPDATE_PROCEDURE.md](OS_UPDATE_PROCEDURE.md)** — weekly rolling Ubuntu OS update for all cluster nodes (drain → apt → reboot → verify → uncordon → cluster health → next). Never touches k3s config.
|
|
- **[OS_UPDATE_ROUTINE.md](OS_UPDATE_ROUTINE.md)** — the Paperclip routine that fires the above weekly.
|
|
- **[K3S_OPERATIONS.md](K3S_OPERATIONS.md)** — k3s version upgrades (separate concern from OS updates).
|
|
- **[ADD_WORKER_NODE.md](ADD_WORKER_NODE.md)** — adding a worker node.
|
|
- **[CLUSTER_ACCESS.md](CLUSTER_ACCESS.md)** — SSH / kubectl access.
|
|
|
|
## Structure
|
|
|
|
### `networking/`
|
|
Network-level configurations including:
|
|
- **[Network Architecture](networking/NETWORK_ARCHITECTURE.md)** - Comprehensive network architecture documentation
|
|
- **[DNS Requirements](networking/DNS_REQUIREMENTS.md)** - DNS records and configuration guide
|
|
- Ingress controller configurations (Traefik)
|
|
- Network policies
|
|
- Load balancer configurations (k3s ServiceLB)
|
|
- Certificate management (cert-manager, TLS)
|
|
- Certificate reloading (Stakater Reloader)
|
|
|
|
### `monitoring/`
|
|
Observability infrastructure:
|
|
- Prometheus operator and configurations
|
|
- Grafana dashboards and datasources
|
|
- Logging stack (Loki, Promtail, etc.)
|
|
- Alert rules and notification channels
|
|
- Service monitors and pod monitors
|
|
|
|
## Purpose
|
|
|
|
Infrastructure configurations in this directory are shared across all applications. Changes here can affect the entire cluster, so:
|
|
|
|
1. Test thoroughly before applying
|
|
2. Document all changes
|
|
3. Consider the impact on existing deployments
|
|
4. Coordinate with other team members
|
|
|
|
## Adding Infrastructure Components
|
|
|
|
When adding new infrastructure components:
|
|
|
|
1. Create appropriate subdirectories if needed
|
|
2. Include clear documentation
|
|
3. Define dependencies and prerequisites
|
|
4. Provide rollback procedures
|
|
5. Update this README with the new component
|