The websocket connection drops were caused by oauth2-proxy's upstream-timeout
being set to only 30s. Websocket connections are long-lived and need much
longer timeouts. Increased to 3600s (1 hour) per Traefik websocket best practices.
Fixes: DEV-371
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Root cause: Dozzle was being rate-limited by the Kubernetes API when
fetching logs from many containers. The aggressive 1-second health check
timeouts caused the container to fail probes and restart repeatedly.
Changes:
- Increased all probe timeouts from 1s to 5s to handle API rate limiting
- Increased liveness probe period from 10s to 30s to reduce check frequency
- Increased readiness probe period from 5s to 10s
- Increased startup probe failure threshold from 15 to 20 (60s total startup time)
- Increased CPU limit from 500m to 1000m and request from 100m to 250m
- Increased memory limit from 512Mi to 1Gi and request from 128Mi to 256Mi
- Pinned image version to v10.6.10 instead of :latest
This prevents unnecessary restarts when Dozzle is waiting on rate-limited
Kubernetes API calls, fixing both the initial 503 errors and the connection
drops after prolonged use.
Resolves: DEV-351
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Fixes DEV-350. During pod startup, Dozzle takes ~11 seconds to start
accepting connections, but oauth2-proxy can receive and proxy requests
immediately. This causes "connection refused" errors when users access
the UI right after a pod restart.
Solution:
- Add startupProbe with 30s timeout (15 failures × 2s) to give Dozzle
time to start without failing readiness
- Add readinessProbe to prevent traffic routing until Dozzle is ready
- Add livenessProbe to restart container if Dozzle becomes unhealthy
All probes use the /healthcheck endpoint on port 8080.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
The oauth2-proxy container was failing with "unknown flag: --proxy-buffer-size".
This flag is not valid for oauth2-proxy and has been removed.
Fixes DEV-349
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Added oauth2-proxy flags to properly handle Server-Sent Events (SSE):
- --flush-interval=1s: Flush buffered data regularly for SSE
- --upstream-timeout=30s: Allow longer streaming connections
- --proxy-buffer-size=8k: Reduce buffering latency
Simplified ingress configuration to standard Traefik setup.
OAuth2-proxy was buffering responses by default, which broke the
WebSocket/SSE endpoints used for real-time log streaming. The
flush-interval flag ensures data is streamed properly.
Related: DEV-349
Co-Authored-By: Paperclip <noreply@paperclip.ing>
oauth2-proxy was resolving localhost to IPv6 [::1]:8080 instead of
IPv4 127.0.0.1:8080, causing connection refused errors on the streaming
endpoints (/api/events/stream and /api/hosts/.../logs/stream).
Changed upstream URL from http://localhost:8080 to http://127.0.0.1:8080
to force IPv4 resolution.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Added WebSocket support configuration to both oauth2-proxy and Traefik ingress:
- Enable explicit WebSocket proxying in oauth2-proxy (--proxy-websockets=true)
- Pass Host header through oauth2-proxy (--pass-host-header=true)
- Add WebSocket upgrade headers to Traefik ingress annotations
This fixes the "UI cannot connect to API" error by ensuring real-time
log streaming WebSocket connections work through the authentication proxy.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Trying to resolve "Could not connect to any Docker Engine" error by letting Dozzle auto-detect its environment without explicit flags.
Current blocker: Dozzle v10.6.10 is not auto-detecting Kubernetes mode despite having proper RBAC and service account configured.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Revert to :latest image (v8.9.4 doesn't exist) and add --agent flag to help Dozzle detect and work in Kubernetes mode.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Switch from :latest to v8.9.4 for stability and add --no-analytics flag to improve startup behavior in Kubernetes.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Switch from direct OIDC (not supported by Dozzle) to forward-proxy authentication using oauth2-proxy as a sidecar container.
Changes:
- Add oauth2-proxy sidecar container for OIDC authentication
- Configure Dozzle to use forward-proxy auth with user headers
- Update service and ingress to route to oauth2-proxy (port 4180)
- Add cookie-secret to sealed secret for oauth2-proxy session management
- Update documentation to reflect oauth2-proxy architecture
The oauth2-proxy authenticates users via Pocket ID and forwards requests to Dozzle with X-Forwarded-User, X-Forwarded-Email, and X-Forwarded-Preferred-Username headers.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Deploy Dozzle (https://dozzle.dev/) for real-time container log viewing in the k8s cluster.
Configuration:
- Namespace: dozzle
- Domain: dozzle.basicstack.de
- Storage: 1Gi PVC with hcloud-volumes-encrypted storage class
- Authentication: Pocket ID OIDC integration
- RBAC: Cluster-wide pod log access via service account
- Strategy: Recreate with ReadWriteOnce PVC for persistent settings
Created Pocket ID OIDC client:
- Client ID: 179c13f2-d251-4e1e-b1a0-c070df350c4e
- Callback URL: https://dozzle.basicstack.de/oauth/callback
ArgoCD application configured with automated sync and self-heal.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
