Add test annotation to argocd-cm ConfigMap to verify that Argo CD
automatically syncs changes to its own configuration.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
This allows the stack Application to manage Argo CD's own configuration
via GitOps, implementing the app-of-apps pattern.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Creates the Argo CD Application that manages all cluster infrastructure
applications from the stack.basicstack.de repository. Configuration:
- Points to cluster-internal Forgejo (forgejo.forgejo.svc.cluster.local)
- Excludes argocd directory to prevent self-management
- Enables selfHeal for automatic sync of changes
- Disables prune for manual deletion approval
- Deployed to the cluster and verified working
All existing services (Stalwart, Directus, Forgejo, BookStack, Pocket ID,
OpenCloud, Passbolt) remain operational. Some pods restarted due to applying
pending git changes (expected GitOps behavior).
Note: Warning about duplicate opencloud namespace definition in
openldap-deployment.yaml and opencloud-deployment.yaml - does not affect
functionality, can be addressed in follow-up cleanup.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- Downloaded upstream Argo CD install.yaml from stable branch
- Customized for basicstack.de cluster:
- Disabled dex deployment (using Pocket ID OIDC instead)
- Added --insecure flag to argocd-server (TLS at Traefik ingress)
- Configured OIDC integration with auth.basicstack.de
- Added RBAC policy mapping argo_admins group to admin role
- Mounted OIDC client credentials from argocd-oidc-secret
- Created Traefik ingress for argo.basicstack.de with cert-manager TLS
Sealed secrets for OIDC and repository credentials already in place.
Ready for deployment per DEV-249 plan.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Created three SealedSecrets for Argo CD integration:
- argocd-oidc-secret: Pocket ID OIDC client credentials
- repo-stack-basicstack-de: SSH deploy key for stack repository
- repo-basicstack-org: SSH deploy key for basicstack.org repository
Repository secrets include argocd.argoproj.io/secret-type label
for automatic Argo CD discovery.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- Use existing basicstack-backup bucket with opencloud/ prefix
- Configure endpoint: https://hel1.your-objectstorage.com
- Add pod affinity to run on same node as OpenCloud (RWO volume requirement)
- Schedule: daily at 2:00 AM UTC
- Retention: last 7 days
- First backup verified successful: 110.611 MiB transferred
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Document all four OIDC clients for OpenCloud:
- Web application (UUID-based client ID)
- Desktop client (OpenCloudDesktop)
- Android mobile (OpenCloudAndroid)
- iOS mobile (OpenCloudIOS)
Includes configuration details, security notes, and troubleshooting.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Implement scale-down/backup/scale-up pattern to work around PVC access mode limitation.
Changes:
- Add RBAC (ServiceAccount, Role, RoleBinding) with statefulsets/scale and pods permissions
- Switch to alpine:3.19 base image with kubectl and restic
- Scale down StatefulSet to 0 replicas before backup
- Run restic backup while pod is stopped
- Scale back up to 1 replica with error handling
- Add cleanup trap to ensure scale-up even on failure
- Set 10-minute timeout and backoff limit
Tested successfully: backup completes in ~32 seconds with minimal downtime.
Resolves DEV-236.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Root cause: k3s service ClusterIP routing instability causing intermittent
failures despite healthy pods. This is the 5th incident - prior fixes treated
symptoms, not the systemic networking fragility.
Changes:
- Add startup probe (60s delay, prevents premature service registration)
- Fix backup job env var substitution (use shell ${VAR}, not K8s $(VAR))
- Add comprehensive monitoring (ServiceMonitor, PrometheusRule, blackbox probes)
- Add alerting for service failures, high latency, pod restarts, backup failures
Evidence:
- Pod healthy (4d15h uptime, 0 restarts) but service ClusterIP routing broken
- Direct pod IP worked, service ClusterIP failed with "Connection reset by peer"
- Iptables rules correct, endpoints correct, but packets not flowing
- Required pod restart + Traefik restart to restore service
Monitoring now tests full service path from outside cluster, not just pod health.
Will alert immediately on failures instead of relying on reactive discovery.
Related: DEV-213, DEV-221, DEV-223, DEV-224, DEV-230, DEV-231
Co-Authored-By: Paperclip <noreply@paperclip.ing>
OpenCloud's IDM service provides an internal LDAP server on localhost:9125.
Updated configuration to connect to IDM's LDAP instead of external OpenLDAP:
- Base DN: o=libregraph-idm (IDM's base DN)
- Graph binds as uid=libregraph,ou=sysusers,o=libregraph-idm
- Users/Groups bind as uid=reva,ou=sysusers,o=libregraph-idm
- Uses IDM service user passwords from secrets
Co-Authored-By: Paperclip <noreply@paperclip.ing>
When using OpenCloud's built-in IDM service, the graph, users, and groups
services should not use external LDAP. Commented out all LDAP-specific
environment variables that were causing startup failures.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- Enable IDM service by removing it from OC_EXCLUDE_RUN_SERVICES
- Remove external LDAP configuration from configmap (graph, users, groups)
- Built-in IDM will handle user/group storage internally
- OIDC auto-provisioning via Pocket ID remains unchanged
This allows OpenCloud to save new users using its internal IDM service.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
The postStart lifecycle hook caused osixia/openldap to crash on startup:
its init script does chown -R on /container/service/slapd/assets/, and
the ConfigMap subPath mount there is read-only, killing the container.
Remove the postStart hook and the schema volume mount from the OpenLDAP
deployment. Add a standalone Kubernetes Job (opencloud-ldap-schema-job.yaml)
that connects via network LDAP as cn=admin,cn=config and loads the schema
after OpenLDAP is confirmed ready. The Job is idempotent (skips if the
schema already exists) and retries up to 10 times on failure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
User creation failed with "openCloudUUID: attribute type undefined" because
OpenLDAP was missing the OpenCloud schema (OIDs under 1.3.6.1.4.1.63016).
Changes:
- Add opencloud-ldap-schema.yaml ConfigMap with the official OpenCloud LDAP
schema defining openCloudUUID, openCloudUser, openCloudExternalIdentity,
openCloudUserEnabled, openCloudUserType, openCloudLastSignInTimestamp
- Mount the ConfigMap into the OpenLDAP pod
- Add lifecycle postStart hook to load schema via ldapadd -Y EXTERNAL -H ldapi:///
(idempotent: skips if already loaded)
- Re-exclude IDM in OpenCloud deployment (external LDAP handles user storage)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Graph service was routing auto-provisioning to external OpenLDAP which lacks
the OpenCloud schema (openCloudUUID: attribute type undefined). The explicit
graph.identity.ldap, users.drivers.ldap, and groups.drivers.ldap sections in
opencloud.yaml overrode the default IDM LDAP, causing user creation to fail.
Remove all external LDAP sections so Graph/Users/Groups default to IDM's
internal LDAP, which has the full OpenCloud schema. IDM is already running.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
The /etc/opencloud mount is read-only (ConfigMap), so opencloud init
cannot write the generated IDM password config. IDM fails to start with
"password of service user IDM has not been set properly".
Add the idm.service_user_passwords section to opencloud.yaml, referencing
the OC_IDM_* env vars that are already populated from the sealed secret.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
After Pocket ID OIDC flow completed, users got "Nicht angemeldet" because:
1. PROXY_ROLE_ASSIGNMENT_DRIVER=oidc required an 'opencloud_role' OIDC claim
that Pocket ID wasn't sending → users got no role → login rejected
2. IDM was excluded, removing the internal user store that auto-provisioning
needs to create user accounts when they first log in
Fixes:
- Switch to PROXY_ROLE_ASSIGNMENT_DRIVER=default so all OIDC-authenticated
users automatically receive the standard user role
- Re-enable IDM service so auto-provisioned accounts have a working user store
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Browser was blocking fetch of https://auth.basicstack.de/.well-known/openid-configuration
due to missing connect-src directive in Content-Security-Policy.
Adds csp.yaml to the ConfigMap (mounted at /etc/opencloud/csp.yaml) with
extended connect-src that includes auth.basicstack.de and WebSocket origins.
Sets PROXY_CSP_CONFIG_FILE_LOCATION env var so the proxy service picks it up.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Removed auth-basic configuration and environment variables:
- Removed auth-basic section from ConfigMap
- Removed AUTH_BASIC_LOG_LEVEL, OC_AUTH_BASIC_LDAP_BIND_PASSWORD,
and AUTH_BASIC_AUTH_PROVIDERS_LDAP_BIND_PASSWORD env vars
- Added auth-basic to OC_EXCLUDE_RUN_SERVICES
Result: OpenCloud pod running healthy (1/1 Ready)
- All services listening and operational
- https://opencloud.basicstack.de/ responding with HTTP 200
- No auth-basic bind_password errors
Ready for OIDC authentication testing with Pocket ID.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Changed from alpine+envsubst to busybox with simple shell-based
variable expansion using sed. This avoids permission issues with
apk and works with non-root security context.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Mounting opencloud-config ConfigMap to /etc/opencloud caused pod crashes
because the YAML contains placeholders like ${ENV_VAR} that aren't expanded.
Issue: OpenCloud expects actual values in config files, but ConfigMaps
don't perform environment variable substitution.
Solution: Use environment variables only (highest precedence per docs).
The current deployment with ENV vars works correctly - no file config needed.
Alternative: If file config is required, use an init container to:
1. Read the ConfigMap template
2. Substitute environment variables
3. Write the expanded config to /etc/opencloud
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Added volume mount for opencloud-config ConfigMap at /etc/opencloud
so OpenCloud services can read the opencloud.yaml configuration file.
This follows OpenCloud's standard configuration pattern:
- opencloud.yaml provides global defaults
- Environment variables override file configuration
- ConfigMap mounted at /etc/opencloud (container default)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
OIDC redirect still not working after multiple configuration attempts.
Enabled basic auth so user can test LDAP authentication.
Added user: andreas.leinen@basicstack.de
Password: OpenCloud2024!
TODO: Investigate why OIDC redirect isn't happening despite correct
configuration in deployment and configmap.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Problem: Frontend still not redirecting to OIDC after all config changes.
Created detailed troubleshooting doc covering:
- All 6 commits of configuration changes applied
- What's verified working (OIDC config, IDM, no local users)
- Current behavior vs expected behavior
- Possible root causes
- Next investigation steps
Suggests further investigation needed:
- Check OpenCloud web frontend source code
- Test OIDC flow with curl/API
- Consult OpenCloud community/support
- Browser dev tools debugging
All backend config appears correct. Issue likely in frontend SPA logic
or undocumented configuration requirement for OIDC-only mode.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Problem: Auth-service requires OC_JWT_SECRET to mint/validate JWT tokens.
Root cause: Only OC_TOKEN_MANAGER_JWT_SECRET was set, not OC_JWT_SECRET.
According to OpenCloud docs, auth-service needs:
- OC_JWT_SECRET or AUTH_SERVICE_JWT_SECRET
Changes:
- Added OC_JWT_SECRET environment variable
- Points to same jwt-secret as OC_TOKEN_MANAGER_JWT_SECRET
Why: Auth-service uses OC_JWT_SECRET to validate tokens from OIDC flow.
Without it, authentication fails silently.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Problem: Removing init entirely broke OpenCloud - needs init for config setup.
Solution: Keep 'opencloud init' but remove '--admin-password' flag.
Changes:
- Restored 'opencloud init' command (needed for jwt_secret and config)
- Removed '--admin-password' flag (prevents local admin user creation)
- IDM_CREATE_DEMO_USERS=false still set (prevents demo users)
Result: Init runs to setup config, but no local users are created.
Only OIDC auto-provisioned users will exist.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Problem: Frontend shows login page because local admin user exists.
Root cause: 'opencloud init --admin-password' creates local admin on every start.
Changes:
- Removed 'opencloud init' command from container startup
- Changed to direct 'opencloud server' execution
- Added IDM_CREATE_DEMO_USERS=false to prevent demo user creation
Why: With external OIDC, we don't need local users. The frontend detects
local users and shows a password login page. By removing local user creation,
the frontend will only offer OIDC authentication.
Auto-provisioning will create users in IDM on first OIDC login.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Problem: Login still showing local page instead of OIDC redirect.
Root cause: Excluded both IDP and IDM, but IDM is needed for user storage.
Architecture Fix:
- IDP (Identity Provider) = EXCLUDED (auth handled by Pocket ID OIDC)
- IDM (Identity Management) = KEPT (provides LDAP storage for users/groups)
- Proxy auto-provisioning = creates users in IDM LDAP on first OIDC login
Changes:
- deployment: OC_EXCLUDE_RUN_SERVICES changed from "search,idp,idm" to "search,idp"
- configmap: Re-enabled IDM service configuration
- configmap: Restored graph.identity.ldap (points to internal IDM)
- configmap: Restored users/groups LDAP drivers (connect to internal IDM)
Flow: User → OIDC (Pocket ID) → Proxy auto-provision → IDM LDAP → User created
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Problem: Users still routed to /login page instead of OIDC redirect.
Root cause: auto_provision_accounts was disabled in proxy OIDC config.
Changes:
- proxy.oidc.auto_provision_accounts: true (enable auto-provisioning)
- proxy.oidc.user_oidc_claim: email (user identification claim)
- proxy.oidc.role_assignment: Added OIDC role mapper for groups
- proxy.enable_basic_auth: false (explicitly disable basic auth)
Result: Users will be auto-created on first OIDC login with group-based roles.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Problem: OpenCloud was configured for internal authentication but login wasn't working.
Changes:
- Exclude internal IDP/IDM services (OC_EXCLUDE_RUN_SERVICES=search,idp,idm)
- Configure external OIDC with Pocket ID (auth.basicstack.de)
- Disable basic authentication (PROXY_ENABLE_BASIC_AUTH=false)
- Configure web service OIDC client settings
- Remove internal demo user creation (IDM_CREATE_DEMO_USERS)
Result: Users will authenticate via Pocket ID only, no internal auth methods.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Changed from random password to fixed password 'OpenCloud2024!'
to provide consistent login credentials.
Admin credentials:
- Username: admin
- Password: OpenCloud2024!
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Removed read-only ConfigMap mount that prevented initialization
- Added 'opencloud init' to startup command to auto-generate config
- Config is now generated in /etc/opencloud at container startup
- Admin user is automatically created with random password
- Service users are created by init process
- Fixes HTTP 500 error on login
This allows OpenCloud to properly initialize its internal IDM/IDP
services with the necessary service users for inter-service auth.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Removed all Pocket ID OIDC environment variables. OpenCloud now uses
its internal IDP for authentication instead of external OIDC provider.
The frontend config.json now correctly points to the internal IDP:
- Authority: https://opencloud.basicstack.de (was: auth.basicstack.de)
- Client ID: web (internal IDP client)
This fixes the login redirect issue where users were being sent to
the login page but couldn't see the login form.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
