basicstack/stack.basicstack.de
2
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

fix(dns): cap hostNetwork/coredns pod DNS at 3 upstreams (DEV-527) #9

Open
forgejoadmin wants to merge 1 commit from fix/DEV-527-dns-config-hostnetwork into main
pull from: fix/DEV-527-dns-config-hostnetwork
merge into: basicstack:main
Conversation 0 Commits 1 Files changed 5 +493
forgejoadmin commented 2026-08-23 12:05:30 +00:00
Owner
Copy link

Summary

Hetzner OS publishes 4 systemd-resolved upstreams and Kubernetes limits pod resolv.conf to 3 nameservers, so kubelet drops the 4th and fires a DNSConfigForming Warning event on every hostNetwork or dnsPolicy=Default pod restart. This PR pins the affected pods to 3 explicit upstreams (the same 3 kubelet was already picking), silencing the warning without changing DNS behaviour.

Changes

  • apps/observability/patches/node-exporter-dns-config.yaml — strategic-merge patch adding dnsPolicy: None + dnsConfig to the kube-prometheus-stack node-exporter DaemonSet (Helm-managed, applied by hand)
  • apps/observability/patches/coredns-dns-config.yaml — companion patch for the k3s built-in CoreDNS Deployment. kubectl patch alone is not durable because the k3s addon controller reverts dnsPolicy; kept as a quick manual re-apply hook
  • infrastructure/k3s-manifests/coredns.yaml — the authoritative modified k3s addon manifest that must live at /var/lib/rancher/k3s/server/manifests/coredns.yaml on all 3 CP nodes
  • infrastructure/k3s-manifests/README-DEV-527.md — apply procedure, verification steps, and upgrade caveat

Applied + verified on the live cluster

  • node-exporter DaemonSet rolled with dnsPolicy=None; no DNSConfigForming events on the 9 current pods
  • coredns Deployment reconciled after pushing the modified manifest to all 3 CPs; new pod runs with dnsPolicy=None and 3-nameserver dnsConfig
  • internal + external DNS resolution still works (nslookup kubernetes.default + nslookup forgejo.basicstack.de from a busybox pod)

Test plan

  • kubectl -n observability rollout status daemonset kube-prometheus-stack-prometheus-node-exporter
  • kubectl -n kube-system rollout status deployment coredns
  • No DNSConfigForming events on current node-exporter or coredns pods
  • nslookup kubernetes.default.svc.cluster.local inside the cluster
  • nslookup forgejo.basicstack.de (external)
## Summary Hetzner OS publishes 4 systemd-resolved upstreams and Kubernetes limits pod resolv.conf to 3 nameservers, so kubelet drops the 4th and fires a `DNSConfigForming` Warning event on every hostNetwork or `dnsPolicy=Default` pod restart. This PR pins the affected pods to 3 explicit upstreams (the same 3 kubelet was already picking), silencing the warning without changing DNS behaviour. ### Changes - `apps/observability/patches/node-exporter-dns-config.yaml` — strategic-merge patch adding `dnsPolicy: None` + `dnsConfig` to the kube-prometheus-stack `node-exporter` DaemonSet (Helm-managed, applied by hand) - `apps/observability/patches/coredns-dns-config.yaml` — companion patch for the k3s built-in CoreDNS Deployment. `kubectl patch` alone is not durable because the k3s addon controller reverts `dnsPolicy`; kept as a quick manual re-apply hook - `infrastructure/k3s-manifests/coredns.yaml` — the authoritative modified k3s addon manifest that must live at `/var/lib/rancher/k3s/server/manifests/coredns.yaml` on all 3 CP nodes - `infrastructure/k3s-manifests/README-DEV-527.md` — apply procedure, verification steps, and upgrade caveat ### Applied + verified on the live cluster - node-exporter DaemonSet rolled with `dnsPolicy=None`; no `DNSConfigForming` events on the 9 current pods - coredns Deployment reconciled after pushing the modified manifest to all 3 CPs; new pod runs with `dnsPolicy=None` and 3-nameserver `dnsConfig` - internal + external DNS resolution still works (`nslookup kubernetes.default` + `nslookup forgejo.basicstack.de` from a busybox pod) ## Test plan - [x] `kubectl -n observability rollout status daemonset kube-prometheus-stack-prometheus-node-exporter` - [x] `kubectl -n kube-system rollout status deployment coredns` - [x] No `DNSConfigForming` events on current node-exporter or coredns pods - [x] `nslookup kubernetes.default.svc.cluster.local` inside the cluster - [x] `nslookup forgejo.basicstack.de` (external)
forgejoadmin added 1 commit 2026-08-23 12:05:30 +00:00 
Hetzner OS publishes 4 systemd-resolved upstreams and Kubernetes limits
pod resolv.conf to 3 nameservers, so kubelet drops the 4th and fires a
DNSConfigForming Warning event on every hostNetwork or dnsPolicy=Default
pod restart. Silence the noise by pinning the pods to 3 explicit servers
(same 3 kubelet was already picking).

- apps/observability/patches/node-exporter-dns-config.yaml — strategic-
  merge patch adding dnsPolicy=None + dnsConfig to the
  kube-prometheus-stack node-exporter DaemonSet (Helm-managed, applied
  by hand)
- apps/observability/patches/coredns-dns-config.yaml — companion patch
  for the k3s built-in CoreDNS Deployment. kubectl patch alone is not
  durable because the k3s addon controller reverts dnsPolicy; kept as a
  quick manual re-apply hook
- infrastructure/k3s-manifests/coredns.yaml — the authoritative modified
  k3s addon manifest that must live at
  /var/lib/rancher/k3s/server/manifests/coredns.yaml on all 3 CP nodes
- infrastructure/k3s-manifests/README-DEV-527.md — apply procedure,
  verification steps, and upgrade caveat

Applied and verified on the live cluster:
- node-exporter DaemonSet rolled with dnsPolicy=None; no
  DNSConfigForming events on current pods
- coredns Deployment reconciled after pushing the modified manifest to
  all 3 CPs; new pod runs with dnsPolicy=None and 3-nameserver dnsConfig
- internal + external DNS resolution still works

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This pull request can be merged automatically.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin fix/DEV-527-dns-config-hostnetwork:fix/DEV-527-dns-config-hostnetwork
git checkout fix/DEV-527-dns-config-hostnetwork

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git checkout main
git merge --no-ff fix/DEV-527-dns-config-hostnetwork
git checkout fix/DEV-527-dns-config-hostnetwork
git rebase main
git checkout main
git merge --ff-only fix/DEV-527-dns-config-hostnetwork
git checkout fix/DEV-527-dns-config-hostnetwork
git rebase main
git checkout main
git merge --no-ff fix/DEV-527-dns-config-hostnetwork
git checkout main
git merge --squash fix/DEV-527-dns-config-hostnetwork
git checkout main
git merge --ff-only fix/DEV-527-dns-config-hostnetwork
git checkout main
git merge fix/DEV-527-dns-config-hostnetwork
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: basicstack/stack.basicstack.de#9
No description provided.
Delete branch "fix/DEV-527-dns-config-hostnetwork"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?