stack.basicstack.de/apps/pocket-id/MIGRATION.md
CTO Agent 2cdbd04848 Add Pocket ID migration verification and rollout guide
Documents the complete migration process, safety measures, and verification steps
for the Pocket ID Argo CD migration.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-07-12 14:36:50 +00:00

5.7 KiB

Pocket ID Migration to Argo CD - Verification & Rollout

Status: ✅ Manifests committed to Git (commit 794cf51)
Date: 2026-07-12
Task: DEV-291

Pre-Migration State (Backed Up)

All existing manifests have been exported and saved to:

  • Location: /tmp/paperclip-run-dev-291-b33dbd3d-76c-1s9MK2/pocket-id-backup/
  • Includes: deployment, service, PVC, ingress, secrets (with TLS certificate)

Current Running State

  • Pod: pocket-id-644b9969b8-4b5fq - Running (2 restarts, 16h age)
  • Service: ClusterIP on 10.97.209.99:1411
  • PVC: pocket-id-data - Bound to pvc-d784e4fc-48e8-464a-8807-cee01ee70e66 (5Gi)
  • Ingress: https://auth.basicstack.de (TLS via cert-manager)
  • Secrets: pocket-id-secrets (managed by SealedSecret), pocket-id-tls (managed by cert-manager)

Migration Changes

Files Added to Git (apps/pocket-id/)

  1. deployment.yaml - Cleaned manifest for pocket-id deployment
  2. service.yaml - ClusterIP service definition
  3. pvc.yaml - PVC with Delete=false annotation for safety
  4. ingress.yaml - Traefik ingress with cert-manager TLS
  5. README.md - Documentation and safety notes
  6. MIGRATION.md - This document

Files Already in Git

  • pocket-id-secrets-sealed.yaml - SealedSecret (existed before migration)
  • ../app-pocket-id.yaml - Argo CD Application manifest (existed before)

Safety Measures Applied

  1. ✅ PVC Protection: Added argocd.argoproj.io/sync-options: Delete=false annotation

    • Prevents Argo CD from ever deleting the PVC
    • Existing volume pvc-d784e4fc-48e8-464a-8807-cee01ee70e66 will be preserved
  2. ✅ Secret Management:

    • Existing SealedSecret will continue to manage pocket-id-secrets
    • TLS secret excluded from Git (managed by cert-manager)
  3. ✅ Non-Destructive Sync:

    • All manifests match current cluster state
    • Argo CD will adopt existing resources without recreation
    • No service interruption expected
  4. ✅ Backup Created:

    • Full export of all resources saved
    • Can be restored if needed

Rollout Steps

Step 1: Refresh Argo CD Application

# Option A: Wait for automatic refresh (every 3 minutes by default)
# Option B: Manual refresh via UI or CLI
kubectl patch application pocket-id -n argocd -p '{"metadata":{"annotations":{"argocd.argoproj.io/refresh":"hard"}}}' --type merge

Step 2: Review Diff in Argo CD UI

  1. Open Argo CD UI: https://argocd.basicstack.de (or your Argo CD URL)
  2. Navigate to pocket-id application
  3. Click "APP DIFF" to review changes
  4. Expected diff:
    • New resources: deployment, service, ingress (should show as "in sync" if they match existing)
    • PVC: should show as "synced" with existing resource
    • No deletions should be shown

Step 3: Sync Application

# Via kubectl (if you want to trigger sync programmatically)
kubectl patch application pocket-id -n argocd -p '{"operation":{"sync":{"syncStrategy":{"hook":{}}}}}' --type merge

# OR via Argo CD UI:
# Click "SYNC" button, review the resources to be synced, then "SYNCHRONIZE"

Step 4: Verification Checklist

After sync completes, verify:

  • Application status is "Healthy" and "Synced" in Argo CD UI
  • Pod is running: kubectl get pods -n pocket-id (should still be the same pod)
  • Service endpoints are correct: kubectl get endpoints -n pocket-id
  • Ingress is working: curl -I https://auth.basicstack.de
  • Web UI is accessible: Open https://auth.basicstack.de in browser
  • Login flow works: Test OIDC login with existing client
  • Admin access works: Verify you can access admin UI with existing credentials
  • OIDC clients preserved: Check that all existing clients are still present
  • No data loss: Verify user accounts and configurations are intact

Step 5: Verify GitOps Workflow

Test that Argo CD is properly managing the deployment:

# Make a test change in Git (e.g., add a label to deployment)
# Wait for Argo CD to detect the change (or refresh manually)
# Verify change is applied automatically or shown as "OutOfSync"

Rollback Plan

If issues occur during or after sync:

Option 1: Revert Git Commit

cd stack.basicstack.de
git revert 794cf51
git push
# Wait for Argo CD to sync the revert

Option 2: Manual Restore from Backup

# Restore from exported manifests in backup directory
kubectl apply -f /tmp/paperclip-run-dev-291-b33dbd3d-76c-1s9MK2/pocket-id-backup/

Option 3: Disable Argo CD Management Temporarily

# Remove Application from Argo CD (does not delete resources)
kubectl delete application pocket-id -n argocd
# Manually manage resources with kubectl until issue is resolved

Success Criteria

Migration is considered successful when:

  1. ✅ All resources are "Healthy" and "Synced" in Argo CD
  2. ✅ https://auth.basicstack.de is accessible and functioning
  3. ✅ All existing OIDC clients work without reconfiguration
  4. ✅ User login and admin access work as before
  5. ✅ No data loss in PVC
  6. ✅ Subsequent Git commits to apps/pocket-id/ are automatically detected by Argo CD
  7. ✅ No service interruptions occurred during migration

Important Notes

  • DO NOT manually modify resources in the cluster after Argo CD takes over
  • All changes should go through Git commits to apps/pocket-id/
  • The PVC is protected by Delete=false but still be careful with any PVC-related changes
  • TLS certificate is managed by cert-manager - do not add it to Git
  • The ENCRYPTION_KEY must never change - it's needed to decrypt existing data

Contact

If issues arise during rollout:

  • Check Argo CD UI for detailed sync status and error messages
  • Review pod logs: kubectl logs -n pocket-id -l app=pocket-id
  • Escalate to CTO agent if automated rollback is needed