basicstack/stack.basicstack.de
2
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions
stack.basicstack.de/infrastructure/CP1_UPDATE_PROCEDURE.md
CTO Agent e848d225cd os-update: generalize CP procedure to HA cp-1/cp-2/cp-3 (DEV-515) 
Replaces single-CP CP1_UPDATE_PROCEDURE.md + update-cp-1.sh with an
HA-aware procedure that covers all three CPs. Rationale: after DEV-510
the cluster runs embedded etcd 3.6.12 across cp-1/cp-2/cp-3 and CPs
are NoSchedule-tainted, so:

- Kine cascade is no longer the driving risk; embedded 3-node etcd
  tolerates one member down. The kubectl-latency guardrail stays as a
  soft signal.
- CPs host no StatefulSets or single-replica Deployments. The old
  Phase C batched stateful eviction dance is obsolete.
- CP reboots no longer lose the api-server; two peers keep serving.
  /livez is polled from the operator machine and cross-checked with
  etcdctl endpoint status from a peer CP.

New CP ordering rule: one CP at a time, leader last, health gate
between CPs. Refuses to start if another CP is already cordoned.

Automation:
- New scripts/os-update/update-cp-node.sh <node> is the entry point.
  Supports --add-swap / --preflight / --drain / --apt / --reboot /
  --finalize / --run / --dry-run. Phase A also installs etcd-client
  on cp-2/cp-3 so the etcd-quorum probes work when cp-1 is later the
  target. Peer picker prefers a CP that has etcdctl; refuses cp-1
  target with a targeted error if no other CP has etcdctl yet.
- scripts/os-update/update-cp-1.sh is now a thin wrapper that forwards
  to update-cp-node.sh k3s-cp-1 "$@" for backwards compat.
- CP1_UPDATE_PROCEDURE.md is a redirect stub pointing at the new doc.
- OS_UPDATE_PROCEDURE.md §8 topology block, kine cascade section, and
  automation entry point list all point at the new HA-aware artifacts.

Verified: bash -n clean on both scripts; --dry-run walks all six
phases for each of cp-1/cp-2/cp-3; --preflight against live cluster
correctly aborts cp-1 with "install etcdctl on another CP first" and
cp-2/cp-3 with "run --add-swap first" (their current state).

Execution is board-approval-gated; DEV-497 (cp-1-only executor) is
superseded by a successor ticket.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-08-23 01:53:14 +00:00

1.3 KiB
Raw Permalink Blame History

k3s-cp-1 OS Update Procedure — SUPERSEDED

Superseded by: CP_UPDATE_PROCEDURE.md (DEV-515, 2026-08-23).

Why: this document was written when cp-1 was the sole control-plane node and the datastore was embedded SQLite via kine. After DEV-510 the cluster runs HA embedded etcd across k3s-cp-1, k3s-cp-2, k3s-cp-3, and all three CPs are NoSchedule-tainted (they host no StatefulSets or single-replica Deployments anymore). The single-CP kine cascade risk, the batched stateful eviction dance, and the external-/livez polling pattern no longer describe the current cluster.

The generalized procedure covers cp-1 and cp-2 and cp-3. Use it.

Scripts:

  • infrastructure/scripts/os-update/update-cp-node.sh <node> — the HA-aware entry point.
  • infrastructure/scripts/os-update/update-cp-1.sh — thin wrapper (calls update-cp-node.sh k3s-cp-1 "$@"). Kept for backwards compatibility with any existing runbook that names it explicitly.

History note: the original phase design (add swap → preflight → batched stateful eviction → drain → apt → reboot → finalize) is preserved in git history at commit 19af691 and 989033d for future reference. The new procedure retains Phases A/B/D/E/F essentially unchanged and drops Phase C (batched stateful eviction) as obsolete.