Infrastructure¶

Overview¶

Three distinct infrastructure environments exist across the two product lines:

KRO — DigitalOcean Droplets¶

Servers¶

Container Architecture¶

All services run as Docker containers managed via docker-compose in the kro-devops repository.

Production stack (kro-devops/production/docker-compose.yml):

nginx (stable-alpine)          — reverse proxy, port 80/443
  └── backend (kro-backend)    — NestJS API, port 3000
  └── frontend (kro-frontend)  — React SPA
  └── admin-frontend (kro-admin) — Admin React SPA

Local dev stack (kro-devops/local/docker-compose.yml): - frontend on port 8088 - backend-db (Postgres 11) on port 5432 - backend on port 3000 - admin-frontend on port 8089

Nginx¶

Domain routing via Nginx with Let's Encrypt TLS (/etc/letsencrypt/live/krotrust.com/):

All HTTP traffic on port 80 is redirected to HTTPS (301).

WebSocket upgrades are enabled for api.krotrust.com (real-time notifications).

TLS Renewal¶

Certificates are managed by Certbot / Let's Encrypt. Stored at /etc/letsencrypt/ on the host and mounted into the Nginx container.

Databases (KRO)¶

The CA certificate file is mounted at /usr/src/app/kro-prod-db-cluster-ca-certificate.crt in the backend container.

CI/CD (KRO)¶

Repository: kro-devops/.github/workflows/main.yml

Trigger: push to main branch

Steps: 1. SSH into production droplet (188.166.145.68) using NEW_PREPROD_DROPLET_SSH_PRIVATE_KEY secret 2. cd /root/Kro/kro-devops && git pull origin main 3. docker system prune -f (clear disk space)

Note: The stage deployment (to 64.226.94.111) is commented out in the workflow. Currently only production is deployed automatically.

Application rebuild: After pulling updated devops config, you must manually rebuild and restart containers:

cd /root/Kro/kro-devops/production
docker-compose up --build -d

GIV (KROGiving) — DigitalOcean App Platform¶

Overview¶

KROGiving runs fully managed on DigitalOcean App Platform. There are no servers to SSH into; scaling, TLS, and routing are managed by DigitalOcean.

Components deployed to App Platform: - krogiving-backend — NestJS backend service - krogiving-frontend — React frontend static site - giv-admin-new — Admin dashboard static site

CI/CD (GIV)¶

Repository: krogiving-backend/.github/workflows/deploy.yml

Trigger: push any git tag (*)

Steps: 1. Calls DigitalOcean App Platform API via curl 2. Triggers re-deployment of the krogiving-backend component 3. Uses secrets: STAGING_APP_PLATFORM_ID, DIGITALOCEAN_ACCESS_TOKEN

Note: Only the backend is deployed via CI. The frontend apps may be deployed separately or via DO App Platform's built-in GitHub integration.

Databases (GIV)¶

File Storage (GIV)¶

DigitalOcean Spaces (S3-compatible) for campaign media: - SPACES_ACCESS_KEY_ID / SPACES_SECRET_ACCESS_KEY - SPACES_REGION, SPACES_BUCKET_NAME, SPACES_ENDPOINT

Cloudinary is used for video uploads (via useCloudinaryVideoUpload hook in frontend).

Pencom — On-Premises¶

Overview¶

The Pencom/eHub system runs entirely on-premises across four dedicated VPS nodes. All Docker containers are deployed via GitHub Actions to self-hosted runners on each VPS. There is no cloud provider dependency for eHub compute or databases.

VPS Topology¶

┌─────────────────────────────────────────────────────┐
│  VPS 1 — Internet-Facing (Ingress)                  │
│  Exposed to the public internet                     │
│                                                     │
│  ├── Nginx (TLS termination, reverse proxy)         │
│  ├── api-gateway          :3000                     │
│  └── external-gateway     :3010                     │
└────────────────────┬────────────────────────────────┘
                     │ Internal network only
┌────────────────────▼────────────────────────────────┐
│  VPS 2 — Core Services                              │
│  Not internet-accessible; internal network only     │
│                                                     │
│  ├── core                 :4000                     │
│  ├── payments             :5000                     │
│  ├── compliance           :6000                     │
│  ├── notifications        :7000                     │
│  ├── external-integrations :8000                   │
│  └── audit                :9000                     │
└────────────────────┬────────────────────────────────┘
                     │ Internal network only
┌────────────────────▼────────────────────────────────┐
│  VPS 3 — Databases                                  │
│  Not internet-accessible; internal network only     │
│                                                     │
│  ├── PostgreSQL                                     │
│  │   ├── pencom_core_db2          (core)            │
│  │   ├── pencom_payments          (payments)        │
│  │   ├── pencom_compliance        (compliance +     │
│  │   │                             external-gateway)│
│  │   ├── pencom_external_integrations               │
│  │   └── pencom_audit             (audit)           │
│  ├── MongoDB                                        │
│  │   ├── pencom_notifications     (notifications)   │
│  │   └── pencom_core_otp          (core OTPs)       │
│  └── Redis                                          │
│      └── Bull queues + magic-link cache             │
└─────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────┐
│  VPS 4 — Object Storage                             │
│                                                     │
│  └── MinIO (S3-compatible object store)             │
│      ├── PCC certificates (PDFs)                    │
│      ├── Employer registration documents            │
│      ├── PFC contribution CSVs                      │
│      └── Tutorial videos + thumbnails               │
└─────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────┐
│  Existing PenCom On-Prem (managed by PenCom)        │
│                                                     │
│  └── Oracle ECRS DB                                 │
│      Read-only access by core + compliance          │
└─────────────────────────────────────────────────────┘

Service Ports¶

Inter-service calls are HTTP with x-internal-api-key header. TCP microservice transport is declared on some services but only used internally within notifications for fire-and-forget email dispatch.

CI/CD (Pencom)¶

Deployment via GitHub Actions with self-hosted runners installed on the VPS nodes.

Each app has its own Dockerfile at apps/<app>/Dockerfile.

Databases (Pencom)¶

File Storage (Pencom)¶

MinIO on VPS 4, accessed via the @aws-sdk/client-s3 SDK (MinIO is S3-compatible). Configure with SPACES_* env vars pointing at the MinIO endpoint.

External Integrations (Pencom)¶

Monitoring¶

Prometheus + Grafana stack provisioned via Ansible (monitoring/ansible/), deployed via deploy-monitoring.yaml.

• Node-exporter collects host-level metrics from the VPS nodes.
• ⚠ No application-level Grafana dashboards exist — only host metrics via node-exporter.

Network Security Notes¶

• VPS 1 is the only internet-facing node. VPS 2, 3, and 4 communicate only over the internal network.
• Nginx on VPS 1 handles TLS termination. All external traffic is HTTPS.
• All inter-service HTTP calls carry an INTERNAL_API_KEY header. Backend services on VPS 2 (core, notifications, audit) verify it via ApiKeyAuthGuard. Other services rely on network isolation.
• JWT is used for end-user authentication (employer/admin). Separate JWT secret for external PFC partner tokens.
• Database connections do not use SSL (internal network only). Oracle connections are read-only.
• KRO database connections use SSL with CA certificate verification (DigitalOcean managed database).