stack.basicstack.de/apps/observability/patches/node-exporter-dns-config.yaml
CTO Agent 1c3cd31381 fix(dns): cap hostNetwork/coredns pod DNS at 3 upstreams (DEV-527)
Hetzner OS publishes 4 systemd-resolved upstreams and Kubernetes limits
pod resolv.conf to 3 nameservers, so kubelet drops the 4th and fires a
DNSConfigForming Warning event on every hostNetwork or dnsPolicy=Default
pod restart. Silence the noise by pinning the pods to 3 explicit servers
(same 3 kubelet was already picking).

- apps/observability/patches/node-exporter-dns-config.yaml — strategic-
  merge patch adding dnsPolicy=None + dnsConfig to the
  kube-prometheus-stack node-exporter DaemonSet (Helm-managed, applied
  by hand)
- apps/observability/patches/coredns-dns-config.yaml — companion patch
  for the k3s built-in CoreDNS Deployment. kubectl patch alone is not
  durable because the k3s addon controller reverts dnsPolicy; kept as a
  quick manual re-apply hook
- infrastructure/k3s-manifests/coredns.yaml — the authoritative modified
  k3s addon manifest that must live at
  /var/lib/rancher/k3s/server/manifests/coredns.yaml on all 3 CP nodes
- infrastructure/k3s-manifests/README-DEV-527.md — apply procedure,
  verification steps, and upgrade caveat

Applied and verified on the live cluster:
- node-exporter DaemonSet rolled with dnsPolicy=None; no
  DNSConfigForming events on current pods
- coredns Deployment reconciled after pushing the modified manifest to
  all 3 CPs; new pod runs with dnsPolicy=None and 3-nameserver dnsConfig
- internal + external DNS resolution still works

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-08-23 12:05:12 +00:00

48 lines
1.9 KiB
YAML

---
# Strategic-merge patch capping the node-exporter pod's DNS at three
# upstream servers to silence Kubernetes' DNSConfigForming warning
# (DEV-527).
#
# The Hetzner OS publishes four systemd-resolved upstreams
# (2a01:4ff:ff00::add:2, 2a01:4ff:ff00::add:1, 185.12.64.1, 185.12.64.2),
# and kubelet drops the fourth because Kubernetes limits pod resolv.conf
# to three nameservers. On hostNetwork pods that inherit the node's
# resolv.conf, this fires a per-pod `DNSConfigForming` Warning event on
# every kubelet DNS refresh.
#
# We keep the same three servers kubelet would have picked (the two
# Hetzner IPv6 anycast entries plus the first IPv4 entry) so runtime
# behaviour is unchanged; only the noisy warning goes away. `dnsPolicy:
# None` is required for `dnsConfig` to be authoritative — otherwise
# kubelet still merges the node's resolv.conf on top and we would still
# exceed the three-nameserver limit.
#
# node-exporter is `hostNetwork: true` and does not talk to cluster DNS,
# so upstream-only resolution is correct.
#
# The kube-prometheus-stack chart is Helm-managed (release
# `kube-prometheus-stack` in `observability`) and is NOT currently
# tracked in ArgoCD, so a direct DaemonSet patch is the pragmatic
# wiring path. If the chart moves under GitOps, fold these values
# into the chart values as `prometheus-node-exporter.dnsPolicy` +
# `.dnsConfig` instead of maintaining this patch.
#
# Apply / re-apply with:
# kubectl -n observability patch daemonset \
# kube-prometheus-stack-prometheus-node-exporter \
# --type=strategic \
# --patch-file=apps/observability/patches/node-exporter-dns-config.yaml
spec:
template:
spec:
dnsPolicy: None
dnsConfig:
nameservers:
- 2a01:4ff:ff00::add:2
- 2a01:4ff:ff00::add:1
- 185.12.64.1
searches:
- .
options:
- name: edns0
- name: trust-ad