CI/CD and the cloud
A commit on main should produce a tested, scanned, tagged image and put it in
production without anyone opening a terminal — and let you undo it just as easily.
The pipeline
name: build-and-deploy
on:
push:
branches: [main]
pull_request:
env:
REGISTRY: ghcr.io
IMAGE_API: ghcr.io/${{ github.repository }}/api
IMAGE_WEB: ghcr.io/${{ github.repository }}/web
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with: { java-version: '21', distribution: temurin, cache: maven }
- run: mvn -B verify # Testcontainers spins up PostgreSQL here
working-directory: api
- uses: actions/setup-node@v4
with: { node-version: '20', cache: npm, cache-dependency-path: web/package-lock.json }
- run: npm ci && npm run lint && npm run test -- --watch=false --browsers=ChromeHeadless
working-directory: web
images:
needs: test
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
id-token: write # for keyless signing
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.IMAGE_API }}
tags: |
type=sha,format=long
type=raw,value=latest
- uses: docker/build-push-action@v6
with:
context: ./api
push: true
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Scan for vulnerabilities
uses: aquasecurity/trivy-action@0.24.0
with:
image-ref: ${{ env.IMAGE_API }}:sha-${{ github.sha }}
severity: CRITICAL,HIGH
exit-code: '1' # fail the build, do not just warn
deploy:
needs: images
runs-on: ubuntu-latest
environment: production # requires an approval, records the deploy
steps:
- name: Roll out
run: |
kubectl set image deployment/oms-api \
api=${{ env.IMAGE_API }}:sha-${{ github.sha }}
kubectl rollout status deployment/oms-api --timeout=180s
Tests run on pull requests too, so main is protected rather
than repaired. Images are tagged by commit SHA, so any past state is one
command away. The scan uses exit-code: 1 — a warning nobody
reads is not a control.
Secrets
| Where | Mechanism |
|---|---|
| Local | .env, gitignored |
| CI | GitHub encrypted secrets / OIDC federation to the cloud |
| Runtime | Cloud secret manager, injected as env vars or mounted files |
Storing a permanent access key in CI means a compromised workflow is a compromised cloud account, forever. GitHub's OIDC federation exchanges a short-lived token for a scoped role per run — nothing durable to steal. Every major cloud supports it.
Choosing a target
| Single VM | Managed containers | Kubernetes | |
|---|---|---|---|
| Examples | EC2, Droplet + Compose | Cloud Run, App Runner, Container Apps | EKS, GKE, AKS |
| Setup | An afternoon | An hour | Days, then ongoing |
| Scaling | Manual | Automatic | Automatic, configurable |
| Zero-downtime deploy | DIY | Built in | Built in |
| Ops burden | You patch the OS | Near zero | High |
| Good for | Side projects | Most teams | Many services, a platform team |
Running the database yourself means owning backups, point-in-time recovery, failover, minor-version upgrades and disk growth. RDS, Cloud SQL or Azure Database do all of it for a predictable monthly cost — reliably cheaper than the incident you avoid. See PostgreSQL chapter 6.
Kubernetes, if you need it
apiVersion: apps/v1
kind: Deployment
metadata:
name: oms-api
spec:
replicas: 3
strategy:
type: RollingUpdate
rollingUpdate: { maxSurge: 1, maxUnavailable: 0 } # never dip below capacity
selector:
matchLabels: { app: oms-api }
template:
metadata:
labels: { app: oms-api }
spec:
securityContext:
runAsNonRoot: true
seccompProfile: { type: RuntimeDefault }
containers:
- name: api
image: ghcr.io/acme/oms/api:sha-9f2c1ab
ports: [{ containerPort: 8080 }]
env:
- name: SPRING_PROFILES_ACTIVE
value: prod
- name: SPRING_DATASOURCE_PASSWORD
valueFrom:
secretKeyRef: { name: oms-db, key: password }
resources:
requests: { cpu: 250m, memory: 512Mi }
limits: { memory: 1Gi } # memory limit yes, CPU limit no
startupProbe: # JVMs are slow to boot
httpGet: { path: /actuator/health/liveness, port: 8080 }
failureThreshold: 30
periodSeconds: 5
readinessProbe: # gates traffic
httpGet: { path: /actuator/health/readiness, port: 8080 }
periodSeconds: 10
livenessProbe: # restarts a wedged pod
httpGet: { path: /actuator/health/liveness, port: 8080 }
periodSeconds: 20
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities: { drop: ["ALL"] }
volumeMounts:
- { name: tmp, mountPath: /tmp }
volumes:
- name: tmp
emptyDir: {}
Readiness answers "should I get traffic?" — a pod that fails it is removed from the load balancer but left alone. Liveness answers "am I broken?" — failing it gets the pod killed. Pointing liveness at an endpoint that checks the database means a brief database blip restarts your entire fleet and turns a small outage into a large one.
Migrations during a rolling deploy
For a few seconds, version N and N+1 run simultaneously against one database. Every migration must therefore be compatible with both.
| Change | Safe? | How to do it |
|---|---|---|
| Add a nullable column | Yes | Deploy directly |
Add a NOT NULL column | No | Add nullable → backfill → add constraint later |
| Rename a column | No | Add new → write both → migrate reads → drop old |
| Drop a column | No | Stop using it, deploy, drop in the next release |
| Add an index | Careful | CREATE INDEX CONCURRENTLY, outside a transaction |
Every breaking schema change becomes three safe releases. It feels slow until the first
time a rename would have taken production down mid-deploy. And always set
lock_timeout in migrations — a blocked ALTER TABLE queues every
other query behind it.
Observability
management:
endpoints:
web:
exposure:
include: health,info,metrics,prometheus
endpoint:
health:
probes:
enabled: true # gives /health/liveness and /health/readiness
show-details: when-authorized
metrics:
tags:
application: oms-api
version: ${APP_VERSION:unknown}
server:
shutdown: graceful # finish in-flight requests on SIGTERM
spring:
lifecycle:
timeout-per-shutdown-phase: 25s
logging:
structured:
format:
console: ecs # JSON logs the platform can index
Error rate, p99 latency, saturation (CPU, memory, connection pool) and database replication lag. Almost every real incident shows up in one of them, and a short alert list is one people still read after six months.
Rollback
kubectl rollout undo deployment/oms-api # previous revision kubectl rollout history deployment/oms-api kubectl set image deployment/oms-api api=ghcr.io/acme/oms/api:sha-7a1b3cd # managed platforms: shift traffic back to the previous revision gcloud run services update-traffic oms-api --to-revisions=oms-api-00042=100
Redeploying the old image is seconds. Undoing a migration that dropped a column is a restore. This is the real reason for expand-migrate-contract: it keeps rollback a one-line operation.
Production checklist
- ☐ Images tagged by commit SHA;
latestnever deployed. - ☐ Tests and an image scan block the pipeline on failure.
- ☐ Secrets from a secret manager; CI authenticates via OIDC.
- ☐ Containers non-root, read-only root filesystem, all capabilities dropped.
- ☐ Startup, readiness and liveness probes configured separately.
- ☐ Graceful shutdown enabled and the termination grace period longer than it.
- ☐ Managed PostgreSQL with PITR; a restore tested.
- ☐ Migrations backwards compatible with a
lock_timeout. - ☐ TLS terminated at the ingress; HSTS on.
- ☐ Structured logs, metrics and four alerts that page a human.
- ☐ A rollback performed at least once, on purpose.
Why should a liveness probe avoid checking the database?
Liveness means "this process is unrecoverable, kill it". A dependency being briefly unavailable is not that. Put dependency checks in readiness, where the pod is simply taken out of rotation until it recovers.
You have shipped it
Four courses, a full application and a deployment pipeline. If you want the argument for why these four technologies belong together, read The Stack; if you want to go deeper on any one of them, every course is on the tutorials page.