Zero-Downtime Deployments: Blue-Green and Canary

The Cost of Downtime

Five minutes of downtime costs money. Not just lost transactions—lost trust. Users remember outages. For SaaS businesses, availability is a competitive advantage.

Traditional deployments require stopping services, running migrations, and restarting. If anything fails, you're debugging with customers watching. Zero-downtime deployments flip this: users don't notice deploys. Code ships, services stay live, traffic keeps flowing.

Blue-Green Deployments

Simplest zero-downtime strategy: run two identical production environments (blue and green). At any moment, one serves traffic; the other is idle.

Workflow:

1. Blue is live, serving all traffic.
2. Deploy new code to green.
3. Run smoke tests against green. If they pass, flip the load balancer.
4. Green is now live. Blue is standby for rollback.
5. If green fails, switch back to blue instantly.

Strengths:

• Instant rollback if issues appear.
• Zero downtime—traffic never stops.
• Identical environments eliminate "works in staging" bugs.

Weaknesses:

• 2x infrastructure cost (two full stacks running).
• Database migrations require special handling (see below).
• Not suitable for microservices with many services.

Database migrations in blue-green:

• Never break backward compatibility. New code must handle old schema.
• Deploy schema changes first, run migrations, then deploy code that uses new schema.
• Use expand-contract pattern: add new column, migrate data, deploy code using new column, then drop old column in later deploy.

Canary Deployments

A more nuanced approach: route a small percentage of traffic to the new version. If it's healthy, gradually increase traffic until 100% are on the new version.

Workflow:

1. Deploy new code to canary pods (e.g., 5% of traffic).
2. Monitor error rates, latency, and business metrics (conversion, revenue).
3. If metrics look good, increase canary traffic to 25%, then 50%, then 100%.
4. If metrics degrade, roll back canary to previous version instantly.

Strengths:

• Catch bugs in production with minimal blast radius.
• Gradual rollout reduces risk.
• Works well with microservices and Kubernetes.
• Infrastructure cost-efficient (no 2x environment).

Weaknesses:

• Requires sophisticated monitoring and alerting.
• Debugging issues affecting 10% of users is harder than binary working/broken.
• Requires service mesh (Istio, Linkerd) or smart load balancers.

Feature Flags for Maximum Control

Combine deployments with feature flags: code is deployed but features are off. Gradually enable them for users.

if (featureFlags.newCheckout) {
  showNewCheckout();
} else {
  showOldCheckout();
}

Flip the flag server-side without deploying code. Enable for internal users first, then 10% of customers, then everyone. If issues appear, disable instantly—no rollback needed.

Advantages:

• Decouple code deploy from feature release.
• A/B test features with subsets of users.
• Kill features instantly if performance suffers.
• Works across any architecture.

Kubernetes and Progressive Delivery

If you're on Kubernetes, use Flagger or Argo Rollouts for automated canary deployments. They:

• Route traffic based on weighted rules.
• Monitor Prometheus metrics automatically.
• Rollback if metrics deviate from baseline.
• Support multiple strategies (blue-green, canary, traffic splitting).

Example: deploy a new version, route 10% of traffic, wait 5 minutes, check if error rate increased. If not, gradually shift 20%, then 50%, then 100%. If error rate ever exceeds threshold, auto-rollback.

Recommended Strategy for Startups

Stage 1: Blue-green deployments. Simple, safe, works without complex monitoring.

Stage 2: Add feature flags. Decouple deploys from releases. Ship code faster.

Stage 3: Canary deployments with Kubernetes. Full production observability.

Most startups should start with blue-green + feature flags. It's simple, safe, and lets you deploy multiple times daily. Canary is a refinement for when you have the monitoring infrastructure to back it up.