This reverts commit c060c83. The TOML configuration approach caused
Stalwart to fail to start due to configuration parsing errors.
Will implement a different approach that doesn't require a full
configuration file rewrite.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Stalwart was blocking the HTTP port (8080) from Traefik's internal IP
(10.244.2.227), causing 502 errors when accessing mail.basicstack.de.
Changes:
- Added complete Stalwart TOML configuration (stalwart-config.yaml)
- Configured HTTP listener security to allow internal pod network (10.244.0.0/16)
- Updated StatefulSet to use the new configuration file
- This allows Traefik ingress to reach the Stalwart web UI backend
The fix is non-destructive:
- PVC data is preserved
- Rolling update will restart the pod with new config
- Only security setting is changed (adding allowed IPs)
Fixes: DEV-422
Co-Authored-By: Paperclip <noreply@paperclip.ing>
OpenCloud has been migrated to use the built-in IDM service which
provides an internal LDAP server (localhost:9236). The external
OpenLDAP deployment is no longer needed.
Changes:
- Remove openldap-deployment.yaml and related configuration files
- Remove openldap-admin-sealed.yaml (admin credentials)
- Remove ldap-init-structure.ldif (initialization script)
- Remove opencloud-ldap-schema files (schema configuration)
- Remove OPENLDAP_DEPLOYMENT.md documentation
- Remove OPENLDAP_ADMIN_PASSWORD env var from OpenCloud deployment
The OpenCloud deployment now relies solely on the built-in IDM
service for user and group storage via its internal LDAP interface.
Resolves: DEV-421
Co-Authored-By: Paperclip <noreply@paperclip.ing>
The pod was crash-looping because Stalwart's security configuration
blocks the kubelet's IP (10.244.4.1) from accessing the HTTP health
endpoints. The kubelet's health checks were failing, causing the
startup probe to fail after 6 attempts, leading to pod restarts.
Changed all three health probes (startup, liveness, readiness) from
httpGet to exec with curl localhost. This bypasses the IP blocking
since the health check runs from inside the container using localhost,
which is not subject to Stalwart's external IP blocking rules.
This fix is non-destructive to Stalwart's configuration and state.
The pod will restart once with the new probe configuration, but no
data or configuration will be lost.
Root cause: Stalwart logs showed "Blocked IP address (security.ip-blocked)
listenerId=http, remoteIp=10.244.4.1" followed by "Shutting down Stalwart
Server (server.shutdown) causedBy=SIGTERM" in a repeating pattern.
Fixes: DEV-420
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Pangolin runs three servers:
- Dashboard API on port 3000
- Internal API on port 3001
- Web UI on port 3002
The current configuration was routing traffic to port 3001 (Internal API), which
returns 404 for the root path, causing health probe failures and restart loops.
Changes:
- Update containerPort from 3001 to 3002
- Update livenessProbe to check port 3002
- Update readinessProbe to check port 3002
- Update Service to expose port 3002
- Update Ingress to route to port 3002
This allows health probes to pass (port 3002 returns 200 for /) and routes
web traffic to the correct Web UI server.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
CONTEXT: DEV-403 - Stalwart pod not starting after ConfigMap revert
ROOT CAUSE:
- The stalwart-bootstrap-config ConfigMap was manually created on July 5
- It was added to GitOps at commit 191614f (12:27 today)
- It was reverted at commit fcc5042 (12:56 today)
- ArgoCD pruned the ConfigMap from the cluster after the revert
- Stalwart deployment requires this ConfigMap to start
RECOVERY:
This commit restores the ConfigMap to Git so ArgoCD can recreate it.
The ConfigMap content is identical to the working version from before.
This is a non-destructive recovery - no data or accounts are affected.
The actual Stalwart data on PVC stalwart-data remains intact.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
The stalwart-bootstrap-config ConfigMap was previously created manually
via kubectl and not tracked in Git. This adds it to the repository for
proper GitOps management via Argo CD.
Related to DEV-403 - Stalwart sync hanging issue.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Mount config file to /app/config/config.toml instead of /app/config.toml as the application expects it in the config directory.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Pangolin requires a config.toml file to start. Added ConfigMap with basic configuration including URL, database path, and OIDC settings. Updated deployment to mount the config file.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Cluster nodes are at 93-99% memory capacity. Reduced requests from 512Mi to 256Mi and CPU from 200m to 100m to allow pod scheduling.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Created apps/pangolin/ directory with namespace.yaml defining the pangolin namespace.
Configured DNS A record for pangolin.basicstack.de → 178.105.17.239 (cluster ingress IP).
Updated DNS_REQUIREMENTS.md to document the new Pangolin service.
This completes Phase 1 of the Pangolin deployment (DEV-390):
- Repository structure created with namespace definition
- DNS record configured and verified in Hetzner zone
- Documentation updated
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- Change PVC storage class from local-path to hcloud-volumes-encrypted
- Extend initContainer to create full directory structure including instances/default/data/run-logs
- This fixes permission errors when agents try to create run logs
Resolves: DEV-388
Co-Authored-By: Paperclip <noreply@paperclip.ing>
The /api/health endpoint returns 403 when accessed externally but works from localhost. Changed readiness and liveness probes to use exec with wget from inside the container to work around this restriction.
Resolves: DEV-387
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Change health check from /health to /api/health to match the actual API endpoint. The server is healthy but the readiness/liveness probes were failing because they were checking the wrong path.
Resolves: DEV-387
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- Change paperclip deployment to use paperclip-data-encrypted PVC (preserves existing data)
- Remove pvc.yaml (use existing PVC instead of creating new one)
- Remove postgres StatefulSet and service (use existing postgres deployment)
This preserves the existing 43-day-old data in the encrypted volumes instead of creating a fresh deployment.
Resolves: DEV-387
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- Reduce resource requests from 500m/1Gi to 200m/512Mi to fit available cluster capacity
- Reduce resource limits from 2000m/4Gi to 1000m/2Gi for better resource sharing
- Add IgnoreExtraneous annotation to paperclip-secrets SealedSecret to fix ArgoCD degraded status
Resolves: DEV-387
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Add all necessary Kubernetes manifests for Paperclip deployment:
- deployment.yaml: Main Paperclip application deployment
- service.yaml: ClusterIP service for Paperclip (port 3100)
- ingress.yaml: Traefik ingress for paperclip.home.imicros.de
- pvc.yaml: 50Gi persistent volume for Paperclip data
- postgres-statefulset.yaml: PostgreSQL 16 StatefulSet
- postgres-service.yaml: PostgreSQL service
This completes the GitOps configuration for Paperclip in the
apps/paperclip directory. The ArgoCD application at apps/app-paperclip.yaml
is already configured to deploy from this path.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Move backup resources from separate 'backup' namespace to 'forgejo':
- Change CronJob namespace from 'backup' to 'forgejo'
- Reuse existing forgejo-postgres-secret instead of duplicate backup secret
- Remove backup-namespace.yaml (no longer needed)
- Remove forgejo-backup-secret-sealed.yaml (using existing secret)
This simplifies the architecture by keeping all Forgejo-related
resources in a single namespace and eliminating credential duplication.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Move backup configuration from separate app to forgejo folder:
- Move forgejo-backup-cronjob.yaml to apps/forgejo/
- Move forgejo-backup-secret-sealed.yaml to apps/forgejo/
- Add backup-namespace.yaml to ensure backup namespace is created
- Remove apps/backup/ folder
- Remove apps/app-backup.yaml Argo CD application
This consolidates the backup configuration into the main Forgejo
application, eliminating the need for a separate Argo CD app.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Add IgnoreExtraneous annotation to MariaDB sealed secret template metadata.
This prevents ArgoCD from seeing the unsealed secret (created by sealed-secrets controller) as an extraneous resource.
Same fix pattern as DEV-377, DEV-378, and DEV-379.
Resolves DEV-385
Co-Authored-By: Paperclip <noreply@paperclip.ing>
This file was replaced by passbolt-ingress-fixed.yaml in the Passbolt
certificate fix (DEV-382). The new ingress uses cert-manager annotations
instead of Traefik-specific TLS configuration.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
The ingress was configured with ingressClassName: nginx, but the cluster
only has Traefik ingress controller installed. This caused the browser to
show Traefik's default certificate instead of the Let's Encrypt certificate.
Changed to ingressClassName: traefik to match all other ingresses in the
cluster (argocd, forgejo, directus, etc.)
Fixes: DEV-382
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- Add all Kubernetes manifest files for Passbolt deployment
- Add MariaDB deployment, service, PVC manifests
- Add Passbolt deployment, service, PVC, ingress manifests
- Add namespace manifest
- Update passbolt-secret to include SMTP authentication
- Add README with initial admin user setup instructions
The configuration was extracted from the running cluster and organized
into separate manifest files for better maintainability.
SMTP is configured to use Stalwart mail server with:
- Username: passbolt
- Email: passbolt@basicstack.de
- Host: stalwart-mail.stalwart.svc.cluster.local
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Change deployment strategy from RollingUpdate (default) to Recreate
to resolve volume attachment conflict. The forgejo-data PVC uses
ReadWriteOnce access mode, which only allows attachment to one pod
at a time. RollingUpdate tries to start new pod before terminating
old pod, causing multi-attach error.
With Recreate strategy, old pod terminates first, volume detaches,
then new pod starts and successfully attaches the volume.
Resolves: DEV-380
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Add argocd.argoproj.io/compare-options: IgnoreExtraneous annotation to
both Stalwart sealed secret templates (stalwart-admin-credentials and
stalwart-s3-backup) to prevent ArgoCD from seeing the unsealed secrets
(created by sealed-secrets controller) as extraneous.
This is the same fix applied in DEV-377 (Directus) and DEV-378 (Harbor).
Resolves: DEV-379
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Add argocd.argoproj.io/compare-options: IgnoreExtraneous annotation to
harbor-secrets sealed secret template to prevent ArgoCD from seeing the
unsealed secret (created by sealed-secrets controller) as extraneous.
This is the same fix applied in DEV-377 for Directus and previously in
DEV-289, DEV-290 for other services.
Resolves: DEV-378
Co-Authored-By: Paperclip <noreply@paperclip.ing>
This tells Argo CD to ignore the unsealed secrets created by the sealed-secrets controller, preventing the health status from being degraded.
This is the same fix applied previously in DEV-289, DEV-290, and to Pocket ID for the same issue.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Changed liveness probe from HTTP /server/health to TCP socket check
and readiness probe to use root path instead. The /server/health
endpoint in Directus v12+ requires authentication by default,
causing 403 responses that triggered pod restarts.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Adds the Directus license key (DP3WZ-8DT8F-79FME-5HJ1A-E4EMJ) as a sealed secret and configures the deployment to use it via the LICENSE_KEY environment variable.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Moved Directus deployment configuration from k8s cluster to Git repository:
- Main Directus deployment updated to v12.1.1
- PostgreSQL deployment (postgres:16-alpine)
- Services for both Directus and PostgreSQL
- Ingress with TLS/cert-manager
- Backup CronJob (daily at 2 AM)
- PVCs for database and uploads
This enables GitOps management via Argo CD.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Dozzle pod was crashing with:
pods.metrics.k8s.io is forbidden: User "system:serviceaccount:dozzle:dozzle"
cannot list resource "pods" in API group "metrics.k8s.io" at the cluster scope
Added permission for the metrics.k8s.io API group to allow Dozzle to collect
pod metrics for its monitoring dashboard.
Fixes: DEV-372
Co-Authored-By: Paperclip <noreply@paperclip.ing>
The websocket connection drops were caused by oauth2-proxy's upstream-timeout
being set to only 30s. Websocket connections are long-lived and need much
longer timeouts. Increased to 3600s (1 hour) per Traefik websocket best practices.
Fixes: DEV-371
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Add Traefik annotations for websocket support:
- Enable sticky sessions to ensure websocket connections go to same pod
- Add custom sticky cookie name for better tracking
- Pass host header for proper routing
Resolves DEV-371
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Changed both stalwart-smtp and stalwart-imap services from
externalTrafficPolicy: Local to externalTrafficPolicy: Cluster.
This enables all 4 k3s nodes to appear healthy in Hetzner Load
Balancer health checks, instead of only k3s-worker-1 (where the
single Stalwart pod runs).
With Cluster policy, traffic arriving at any node will be
forwarded via kube-proxy to k3s-worker-1 where the pod is running.
Verification:
- All 4 nodes now show in service EXTERNAL-IP field
- Mail ports (25, 587, 993) accessible from all nodes
- Connectivity tested from k3s-worker-1 and k3s-worker-2
Fixes: DEV-368
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Add securityContext configuration to the database StatefulSet to ensure
PostgreSQL can access its data directory with correct permissions.
The issue was that the PVC mount permissions did not match PostgreSQL's
requirements (u=rwx/0700 or u=rwx,g=rx/0750). This fix sets:
- runAsUser: 999 (postgres user)
- fsGroup: 999 (postgres group)
- fsGroupChangePolicy: OnRootMismatch (only change ownership when needed)
This resolves the crash loop where harbor-database-0 failed with:
'data directory has invalid permissions'
Fixes: DEV-364
Co-Authored-By: Paperclip <noreply@paperclip.ing>
