fd0b589c10
Added comprehensive documentation of the two-tier load balancing setup: - Hetzner Cloud Load Balancer (external layer, managed by Hetzner CCM) - Kubernetes LoadBalancer services (internal layer, k3s ServiceLB) Key points documented: - Traffic flow from external client through both LB layers to pod - Why LoadBalancer service type is required (CCM integration) - Historical context of the migration from hostPort to Hetzner LB - Service definitions and port configurations Updated: - apps/stalwart/README.md: Added Network Architecture section - infrastructure/networking/NETWORK_ARCHITECTURE.md: Enhanced Stalwart section with two-tier architecture details and updated traffic flows Resolves documentation gap identified in DEV-439. Co-Authored-By: Paperclip <noreply@paperclip.ing>
137 lines
4.5 KiB
Markdown
137 lines
4.5 KiB
Markdown
# Stalwart Mail Server v0.16.11
|
|
|
|
Clean deployment of Stalwart mail server with username/password authentication only.
|
|
|
|
## Architecture
|
|
|
|
- **Version**: v0.16.11
|
|
- **Authentication**: Username/password only (NO OAuth/OIDC)
|
|
- **Configuration**: API-based (stored in RocksDB)
|
|
- **Storage**: Encrypted hcloud-volumes (20Gi)
|
|
- **Backup**: Daily restic backup to S3 at 3 AM
|
|
- **Web UI**: https://mail.basicstack.de
|
|
|
|
## Network Architecture
|
|
|
|
Stalwart uses a two-tier load balancing setup:
|
|
|
|
1. **Kubernetes LoadBalancer Services**: Four separate LoadBalancer services expose SMTP (25, 587, 465) and IMAP (143, 993) ports. The service type `LoadBalancer` is required because the Hetzner Cloud Controller Manager (CCM) automatically provisions and manages the Hetzner Load Balancer when it detects this service type.
|
|
|
|
2. **Hetzner Load Balancer**: Automatically managed by the Hetzner CCM. The load balancer distributes traffic across:
|
|
- k3s-cp-1 (control plane node)
|
|
- k3s-worker-1 through k3s-worker-5 (worker nodes)
|
|
|
|
### Traffic Flow
|
|
|
|
```
|
|
External Client → Hetzner Load Balancer → NodePort → K8s Service → Stalwart Pod
|
|
```
|
|
|
|
The Hetzner CCM watches for LoadBalancer-type services and automatically:
|
|
- Creates/updates the Hetzner LB configuration
|
|
- Configures health checks
|
|
- Maps service ports to node ports
|
|
- Manages target nodes
|
|
|
|
**Important**: Changing the service type from `LoadBalancer` to `NodePort` would break the automatic Hetzner LB management. The current setup is the correct configuration for our infrastructure.
|
|
|
|
## Files
|
|
|
|
- `stalwart-fresh-deployment.yaml` - Main deployment manifest
|
|
- `stalwart-admin-credentials-sealed.yaml` - Sealed secret for admin password
|
|
- `stalwart-s3-backup-sealed.yaml` - Sealed secret for S3 backup credentials
|
|
|
|
## Deployment
|
|
|
|
```bash
|
|
# Apply sealed secrets first
|
|
kubectl apply -f stalwart-admin-credentials-sealed.yaml
|
|
kubectl apply -f stalwart-s3-backup-sealed.yaml
|
|
|
|
# Create bootstrap config
|
|
kubectl create configmap stalwart-bootstrap-config \
|
|
--from-literal=config.json='{"@type":"RocksDb","path":"/var/lib/stalwart"}' \
|
|
-n stalwart
|
|
|
|
# Deploy Stalwart
|
|
kubectl apply -f stalwart-fresh-deployment.yaml
|
|
```
|
|
|
|
## Initial Admin Login
|
|
|
|
After deployment, log in at https://mail.basicstack.de with:
|
|
- Username: `admin`
|
|
- Password: (from stalwart-admin-credentials secret)
|
|
|
|
## Configuration
|
|
|
|
> **IMPORTANT:** The configuration is stored in the RocksDB and cannot be overwritten by a configuration file!
|
|
|
|
All configuration is done via the web UI or API. The bootstrap config only points to the RocksDB database location. NO config.toml files are used.
|
|
|
|
Check https://stalw.art/docs/ref/ for configuration possibilities.
|
|
The API access via /jmap seems to be too complex for the agent and the configruation has been done manually.
|
|
|
|
Refer to [manual configuration steps](manual_config_steps.md)
|
|
|
|
## Network Architecture
|
|
|
|
### Load Balancing Setup
|
|
|
|
Stalwart uses a two-tier load balancing architecture:
|
|
|
|
1. **Hetzner Cloud Load Balancer** (External Layer)
|
|
- Provides the public-facing IP for mail services
|
|
- Configured with k3s-cp-1 and all worker nodes as targets
|
|
- Forwards traffic to Kubernetes NodePorts on the cluster nodes
|
|
|
|
2. **Kubernetes LoadBalancer Services** (Internal Layer)
|
|
- Service type: LoadBalancer (managed by k3s ServiceLB)
|
|
- Automatically configured by Hetzner CCM (Cloud Controller Manager)
|
|
- Creates NodePorts that the Hetzner LB targets
|
|
|
|
### Traffic Flow
|
|
|
|
```
|
|
External Mail Client
|
|
↓
|
|
Hetzner Load Balancer (public IP)
|
|
↓
|
|
k3s Node NodePort (automatically assigned)
|
|
↓
|
|
Kubernetes LoadBalancer Service (stalwart-smtp / stalwart-imap)
|
|
↓
|
|
Stalwart Pod
|
|
```
|
|
|
|
### Why LoadBalancer Service Type is Required
|
|
|
|
The Kubernetes services MUST remain type `LoadBalancer` because:
|
|
- The Hetzner CCM automatically manages the Hetzner Load Balancer configuration
|
|
- When a service is type LoadBalancer, the CCM creates/updates the Hetzner LB targets
|
|
- Changing to NodePort would break this automatic management
|
|
- Manual Hetzner LB configuration would be required and error-prone
|
|
|
|
### Services
|
|
|
|
- `stalwart-smtp` (LoadBalancer): Ports 25, 587
|
|
- `stalwart-imap` (LoadBalancer): Port 993
|
|
- `stalwart-http` (ClusterIP): Port 8080 (web UI via Traefik Ingress)
|
|
|
|
## Ports
|
|
|
|
- SMTP: 25, 587
|
|
- IMAPS: 993 (secure only, port 143 disabled per DEV-359)
|
|
- HTTP: 8080 (web UI)
|
|
|
|
## Storage
|
|
|
|
Data is stored in `/var/lib/stalwart` using the RocksDB database format. This includes:
|
|
- Email messages
|
|
- User accounts
|
|
- Server configuration
|
|
- TLS certificates configuration
|
|
|
|
## Certificate Renewal
|
|
|
|
Refer to [Automatic Renewal TLS Certificate](CERTIFICATE-RENEWAL.md)
|