Deployment Strategies: Blue-Green, Canary, and Rolling

Production deployment strategies for balancing release velocity against blast radius. Covers the architectural trade-offs between blue-green, canary, and rolling deployments—with specific focus on traffic shifting mechanics, database migration coordination, automated rollback criteria, and operational failure modes that senior engineers encounter during incident response.

Strategy taxonomy

Before comparing trade-offs, fix the vocabulary. Practitioners conflate at least seven distinct patterns; the rest of the article uses these definitions.

Shadow and dark launch are often conflated; the practical difference is where the new code runs and what happens to its output. Shadow traffic is request mirroring at the proxy/mesh — Envoy’s request_mirror_policy is the canonical primitive, and Google’s CRE team named the response-discarded variant “dark launch” in their original write-up. Dark launch in the Honeycomb / Charity Majors sense is application-layer: the new code runs inline but is gated behind a flag. Both leave user experience unchanged; only one duplicates load on the candidate.

Mental model: blast radius, rollback, cost

Strategy selection reduces to three variables: blast radius (percentage of users exposed to failures), rollback speed (time to restore previous behavior), and infrastructure cost (resource overhead during deployment). A fourth variable, operational complexity, decides whether your team can actually run the strategy on a Tuesday at 4 p.m.

Key architectural insight: The choice isn’t which strategy is “best” — it’s which failure mode is acceptable. Blue-green trades cost for instant rollback. Canary trades complexity for controlled exposure. Rolling trades rollback speed for simplicity. Recreate trades availability for everything else.

Deployment vs release

The single most useful lens for this article is Charity Majors’ framing — first published as “Deploys Are The Wrong Way To Change User Experience” on the Honeycomb blog in 2023:

• A deploy is an engineering event: building, testing, and rolling out a new artifact to production. Deploys should be small, frequent, and ideally invisible to users.
• A release is a product event: the moment a user-visible behavior changes. Releases should be timed, targeted, and reversible without a redeploy.

Confusing the two is what makes “deploy on Friday” terrifying. Decoupling them — usually with a feature flag — turns rollback from a 3-hour rebuild into a millisecond toggle and lets a product manager run the launch instead of a release engineer. Every deployment strategy in this article should be evaluated against whether it preserves this decoupling: blue-green and canary control the deploy side, feature flags control the release side, and the strongest pipelines combine both.

Blue-Green Deployments

Blue-green deployment maintains two identical production environments. At any time, one (“blue”) serves traffic while the other (“green”) is idle or receiving the new release. Deployment completes with an atomic traffic switch.

Architecture and Traffic Switching

The core requirement: a routing layer that can instantly redirect 100% of traffic between environments. Implementation options:

Why atomic switching matters: Partial deployments create version skew—users might receive HTML from v2 but JavaScript from v1. Blue-green eliminates this by ensuring all requests hit one environment or the other, never both simultaneously.

AWS Implementation

AWS CodeDeploy with ECS supports blue-green natively. Traffic shifting options from the AWS ECS deployment documentation:

Requirements: Application Load Balancer (ALB) or Network Load Balancer (NLB) with two target groups. CodeDeploy manages the target group weights.

Show 3 hidden lines4version: 0.05Resources:6  - TargetService:7      Type: AWS::ECS::Service8      Properties:9        TaskDefinition: "arn:aws:ecs:us-east-1:111122223333:task-definition/my-task:2"10        LoadBalancerInfo:11          ContainerName: "my-container"12          ContainerPort: 8080

Kubernetes Implementation

Kubernetes doesn’t provide native blue-green. Implementation requires managing two Deployments and switching Service selectors:

Show 5 hidden lines6kind: Service7metadata:8  name: my-app9spec:10  selector:11    app: my-app12    version: blue # Change to 'green' for switch13  ports:14    - port: 8015      targetPort: 8080

Operational pattern:

1. Deploy green Deployment with version: green label
2. Verify green pods pass readiness probes
3. Update Service selector from version: blue to version: green
4. Keep blue Deployment for rollback window
5. Delete blue Deployment after confidence period

Trade-offs and Failure Modes

Cost: 2x infrastructure during deployment window. For stateless services, this is the primary overhead. For stateful services with dedicated databases, costs compound.

Session handling: In-flight requests during switch may fail. Mitigations:

• Connection draining: ALB target groups wait for existing connections to complete via deregistration_delay.timeout_seconds (default 300 s, configurable 0–3600 s). Tune it down for short-RTT services or up for long-running connections.
• Session affinity: Sticky sessions prevent mid-session switches but complicate rollback because a sticky cookie may keep users pinned to the old environment.
• Stateless design: Store session state externally (Redis, DynamoDB, JWT). Removes the constraint entirely and is the only option that scales cleanly to canary and rolling.

Database schema compatibility: The critical constraint. If green requires schema changes incompatible with blue, rollback becomes impossible without data loss. See Database Migration Coordination.

Failure detection gap: Blue-green exposes 100% of users immediately. If monitoring doesn’t detect issues within minutes, all users experience the failure. Canary addresses this by limiting initial exposure.

Canary Deployments

Canary deployment routes a small percentage of traffic to the new version while the majority continues hitting the stable version. Traffic shifts gradually based on metrics thresholds, with automatic rollback if metrics degrade.

The term originates from coal mining: canaries detected toxic gases before miners were affected. In deployment, the canary cohort detects issues before full rollout.

Progressive Traffic Shifting

Traffic progression follows predefined stages with metric evaluation at each gate:

11% → [evaluate 10min] → 5% → [evaluate 10min] → 25% → [evaluate 10min] → 100%

Stage duration balances detection speed against deployment velocity. Too short: insufficient data for statistical significance. Too long: slow releases.

From Google SRE Workbook Chapter 16: manual graph inspection is insufficient for detecting canary issues. Automated analysis comparing canary metrics against baseline is required for reliable detection.

Metrics-Driven Release Gates

Effective canary analysis requires comparing canary cohort metrics against baseline (stable version) metrics, not absolute thresholds.

Core metrics (RED method):

• Rate: Request throughput—anomalies indicate routing issues or capacity problems
• Errors: Error rate comparison—canary error rate > baseline + threshold triggers rollback
• Duration: Latency percentiles (p50, p90, p99)—degradation indicates performance regression

Threshold configuration example:

Statistical significance: Google Cloud’s joint guidance with Waze — formalised in Spinnaker’s Kayenta best practices — recommends a minimum of 50 time-series data points per metric per canary run for the analysis to be statistically meaningful. The default marginal / pass thresholds are 75 and 95, with a starting canary lifetime of 3 hours split across three 1-hour runs. At 10 requests/second to the canary cohort, that’s 5 seconds for fast-moving signals (errors, latency) — but slow-burn issues like memory leaks or connection-pool exhaustion still need hours to manifest. Plan for a multi-stage canary that lasts hours, not minutes.

From the same guidance: defining acceptable thresholds is iterative. Too strict causes false positives (unnecessary rollbacks); too loose misses real issues. When uncertain, err on the conservative side.

Kayenta-style automated canary analysis

Netflix’s Kayenta — open-sourced jointly with Google in 2018 and now the canary judge inside Spinnaker — is the canonical implementation of automated canary analysis (ACA). Two design choices in Kayenta have become industry defaults:

1. Three clusters, not two. Production runs the stable version at full scale. A separate baseline cluster is spun up on the same code as production but at the canary’s size, and a canary cluster runs the candidate. Comparing canary-vs-baseline (instead of canary-vs-production) controls for cache warm-up, heap size, and the long-tail effects of long-running processes; this is also the first line of the Spinnaker best-practices doc.
2. Mann-Whitney U judgment. Each metric is classified Pass, High, or Low using the Mann-Whitney U test at 98% confidence; the canary’s score is the ratio of Pass metrics. Spinnaker promotes when the score is ≥ pass and aborts when it is ≤ marginal.

Kubernetes Implementation with Argo Rollouts

Argo Rollouts extends Kubernetes Deployments with canary and blue-green strategies, including automated analysis.

Show 5 hidden lines6kind: Rollout7metadata:8  name: my-app9spec:10  replicas: 1011  strategy:12    canary:13      steps:14        - setWeight: 515        - pause: { duration: 10m }16        - setWeight: 2017        - pause: { duration: 10m }18        - setWeight: 5019        - pause: { duration: 10m }20        - setWeight: 8021        - pause: { duration: 10m }22      analysis:23        templates:24          - templateName: success-rate25        startingStep: 226        args:27          - name: service-name28            value: my-app

AnalysisTemplate with Prometheus:

Show 4 hidden lines5apiVersion: argoproj.io/v1alpha16kind: AnalysisTemplate7metadata:8  name: success-rate9spec:10  args:11    - name: service-name12  metrics:13    - name: success-rate14      successCondition: result[0] >= 0.9515      failureLimit: 316      interval: 60s17      provider:18        prometheus:19          address: http://prometheus:909020          query: |21            sum(rate(http_requests_total{status!~"5.*",app="{{args.service-name}}"}[5m])) /22            sum(rate(http_requests_total{app="{{args.service-name}}"}[5m]))

Service Mesh Traffic Splitting

Istio provides fine-grained traffic control via VirtualService. The networking.istio.io/v1 API was promoted to stable in Istio 1.22 and is now preferred over the older v1beta1:

Show 4 hidden lines5apiVersion: networking.istio.io/v16kind: VirtualService7metadata:8  name: my-app9spec:10  hosts:11    - my-app12  http:13    - route:14        - destination:15            host: my-app16            subset: stable17          weight: 9018        - destination:19            host: my-app20            subset: canary21          weight: 10

Flagger automates canary progression with Istio, Linkerd, or NGINX:

• Creates canary Deployment from primary
• Manages VirtualService weights automatically
• Monitors Prometheus metrics for promotion/rollback decisions
• Supports webhooks for custom validation

Trade-offs and Failure Modes

Complexity: Canary requires traffic splitting infrastructure (service mesh, ingress controller with weighted routing), metrics collection, and analysis automation. Blue-green is simpler.

Consistent user experience: Without session affinity, a user might hit canary on one request and stable on the next. For stateless APIs, this is acceptable. For stateful interactions (multi-step forms, shopping carts), implement sticky routing based on user ID hash.

Metric lag: Some issues (memory leaks, connection pool exhaustion) take hours to manifest. Fast canary progression may promote before slow-burn issues appear. Mitigation: extend observation windows for critical releases, monitor resource metrics alongside request metrics.

Insufficient traffic: Low-traffic services may not generate enough requests during canary stages for statistical significance. Solutions:

• Extend stage durations
• Use synthetic traffic for baseline
• Accept higher uncertainty with manual review gates

Rolling Updates

Rolling updates replace pods incrementally: terminate old pods and start new pods in controlled batches. Kubernetes Deployments use rolling updates by default.

Kubernetes Rolling Update Mechanics

Two parameters control rollout pace:

Configuration examples:

1# Conservative: no capacity loss, requires surge resources2spec:3  strategy:4    rollingUpdate:5      maxSurge: 16      maxUnavailable: 078# Aggressive: faster rollout, temporary capacity reduction9spec:10  strategy:11    rollingUpdate:12      maxSurge: 50%13      maxUnavailable: 25%

Rollout sequence with maxSurge: 1, maxUnavailable: 0 (10 replicas):

1. Create 1 new pod (11 total)
2. Wait for new pod readiness
3. Terminate 1 old pod (10 total)
4. Repeat until all pods updated

Readiness Probes and Failure Detection

Kubernetes uses probes to determine pod health:

Critical for rolling updates: Readiness probe failures prevent traffic routing to unhealthy new pods, but the rollout continues. A pod that passes readiness but has application-level issues (returning errors, slow responses) will receive traffic.

Show 3 hidden lines4spec:5  containers:6    - name: app7      readinessProbe:8        httpGet:9          path: /health10          port: 808011        initialDelaySeconds: 512        periodSeconds: 513        successThreshold: 114        failureThreshold: 315      livenessProbe:16        httpGet:17          path: /health18          port: 808019        initialDelaySeconds: 1520        periodSeconds: 1021        failureThreshold: 3

Manual Rollback

Kubernetes maintains revision history for Deployments:

1# Check rollout status2kubectl rollout status deployment/my-app34# View revision history5kubectl rollout history deployment/my-app67# Rollback to previous revision8kubectl rollout undo deployment/my-app910# Rollback to specific revision11kubectl rollout undo deployment/my-app --to-revision=21213# Pause rollout mid-progress14kubectl rollout pause deployment/my-app

Limitation: Native Kubernetes does not support automated metric-based rollback. If pods pass probes, rollout continues regardless of application metrics. Use Argo Rollouts or Flagger for automated rollback.

Trade-offs and Failure Modes

Version coexistence: During rollout, both old and new versions serve traffic simultaneously. This requires:

• Backward-compatible APIs (new version must handle old client requests)
• Forward-compatible APIs (old version must handle new client requests during rollback)
• Schema compatibility for shared databases

Rollback speed: Reversing a rolling update requires another rolling update. For a 100-pod Deployment with conservative settings, full rollback may take 10+ minutes. Compare to blue-green’s instant switch.

Blast radius growth: Unlike canary’s controlled percentage, rolling update exposure grows linearly. With 10 replicas, updating 1 pod exposes 10% of traffic immediately. No pause for metric evaluation unless manually configured with pause in Argo Rollouts.

No traffic control: Rolling updates don’t support routing specific users to new version. All traffic is distributed across available pods. Use canary or feature flags for targeted exposure.

Feature Flags and Deployment Decoupling

Feature flags separate deployment (code reaching production) from release (users experiencing features). This enables deploying code with features disabled, then enabling gradually—independent of infrastructure changes.

Architectural Pattern

Key insight: Rollback becomes a flag toggle (milliseconds) rather than a deployment (minutes to hours). Even if the underlying deployment strategy is a rolling update, feature flags still provide instant rollback for the specific feature.

Implementation Considerations

Flag evaluation location:

• Server-side: Flag service called per-request. Higher latency, always current.
• Client-side SDK: Flags cached locally, synced periodically. Lower latency, eventual consistency.
• Edge: Flags evaluated at CDN/load balancer. Zero application latency, limited context.

Consistent assignment: Users must receive the same flag value across requests for coherent experience. Implementation: hash user ID to bucket (0-100,000), compare against percentage threshold.

Show 4 hidden lines5function shouldEnableFeature(userId, percentage) {6  const hash = hashUserId(userId) // Returns 0-999997  const bucket = hash % 1000008  return bucket < percentage * 1000 // percentage as 0-1009}

Flag lifecycle management: Technical debt accumulates if flags aren’t cleaned up. Establish policy:

1. Feature fully rolled out → remove flag within 2 weeks
2. Feature abandoned → remove flag and code immediately
3. Long-lived flags (A/B tests, entitlements) → document and review quarterly

Integration with Deployment Strategies

LaunchDarkly progressive rollouts from their documentation: automatic percentage increase over time with metric monitoring. Similar to canary, but at application layer rather than infrastructure.

Database Migration Coordination

Database schema changes constrain all deployment strategies. If the new code version requires schema changes incompatible with the old version, rollback becomes impossible without data loss. Martin Fowler calls this out explicitly in the original blue-green write-up: “first apply a database refactoring to change the schema to support both the new and old version of the application, deploy that, check everything is working fine so you have a rollback point, then deploy the new version of the application.”

The compatibility problem

Consider adding a required column:

1ALTER TABLE users ADD COLUMN email_verified BOOLEAN NOT NULL DEFAULT false;

Timeline during blue-green deployment:

1. Green code expects email_verified column
2. Switch traffic to green
3. Issue detected, switch back to blue
4. Problem: Blue code doesn’t know about email_verified, may fail or ignore it

Worse: If green code wrote rows with email_verified = true, blue code can’t interpret them correctly.

Expand-Contract Pattern

The expand-contract pattern (also called parallel change) from Martin Fowler solves this by splitting migrations into backward-compatible phases:

Phase 1 - Expand:

1. Add new column as nullable (no constraint)
2. Deploy code that writes to both old and new columns (dual-write)
3. Backfill existing rows

1-- Expand: Add nullable column2ALTER TABLE users ADD COLUMN email_verified BOOLEAN;34-- Application dual-write pseudocode5INSERT INTO users (name, is_verified, email_verified)6VALUES ('Alice', true, true);  -- Write both columns

Phase 2 - Migrate:

1. Verify all rows have new column populated
2. Monitor for any rows missing new column data

Phase 3 - Contract:

1. Deploy code that reads only from new column
2. Stop writing to old column
3. Add NOT NULL constraint
4. Drop old column

1-- Contract: Add constraint after all rows populated2ALTER TABLE users ALTER COLUMN email_verified SET NOT NULL;34-- Later: Drop old column5ALTER TABLE users DROP COLUMN is_verified;

Rollback safety: At any phase, the previous code version works because old columns remain valid until the final contract phase.

Online Schema Migration Tools

Large tables can’t be altered with ALTER TABLE—the operation locks the table for the duration, causing downtime. Online schema migration tools solve this.

gh-ost (MySQL)

gh-ost from GitHub creates a “ghost” table, applies changes, then performs atomic swap:

1. Create ghost table with new schema
2. Stream binary log changes to ghost table
3. Copy existing rows in batches
4. Atomic table rename when caught up

Key advantages over trigger-based tools:

• No triggers on original table (triggers add write latency)
• Pausable and resumable
• Controllable cut-over timing

1gh-ost \2  --host=db.example.com \3  --database=mydb \4  --table=users \5  --alter="ADD COLUMN email_verified BOOLEAN" \6  --execute

Requirement: Row-Based Replication (RBR) format in MySQL.

pt-online-schema-change (MySQL)

Percona Toolkit uses triggers to capture changes:

1. Create new table with desired schema
2. Create triggers on original to capture INSERT/UPDATE/DELETE
3. Copy rows in chunks
4. Swap tables atomically

1pt-online-schema-change \2  --alter "ADD COLUMN email_verified BOOLEAN" \3  D=mydb,t=users \4  --execute

Limitation: Foreign keys require special handling (--alter-foreign-keys-method).

PostgreSQL: pgroll

pgroll provides zero-downtime PostgreSQL migrations following expand-contract natively:

• Serves multiple schema versions simultaneously
• Automatic dual-write via triggers
• Versioned schema views for each application version

Migration Timing and Deployment Coordination

Rule: Schema migration must complete before deploying code that depends on it.

Deployment sequence:

1. Run expand migration (add column, create table)
2. Deploy new code (dual-write enabled)
3. Run backfill (populate new column for existing rows)
4. Verify data consistency
5. Deploy contract code (read from new column only)
6. Run contract migration (add constraints, drop old columns)

Rollback windows: Each phase should allow rollback for at least 24 hours before proceeding to the next phase. This catches issues that appear under production load patterns.

Automated Rollback Systems

Automated rollback reduces Mean Time to Recovery (MTTR) by eliminating human decision latency. The system monitors metrics and triggers rollback when thresholds are breached.

Metric Selection and Thresholds

Effective automated rollback requires metrics that:

1. Correlate strongly with user-visible issues
2. Change quickly when problems occur
3. Have low false-positive rates

Recommended metrics:

The standard Apdex bands (apdex.org technical specification) place 0.85–0.93 in “Good” and 0.94–1.00 in “Excellent”; rolling back the moment a service drops below 0.85 catches regressions before they leak into the “Fair” band.

Baseline comparison: Compare canary/new version metrics against the stable version, not absolute thresholds. A service with a 0.5% baseline error rate triggering on > 1% absolute error rate will fire immediately; triggering on > 0.5% increase relative to stable is appropriate.

Argo Rollouts Analysis

Argo Rollouts integrates with Prometheus, Datadog, New Relic, and Wavefront for automated analysis:

Show 5 hidden lines6kind: AnalysisTemplate7metadata:8  name: error-rate-comparison9spec:10  args:11    - name: canary-hash12    - name: stable-hash13  metrics:14    - name: error-rate-comparison15      successCondition: result[0] <= 0.01 # Canary error rate <= 1% higher16      failureLimit: 317      interval: 60s18      provider:19        prometheus:20          address: http://prometheus:909021          query: |22            (23              sum(rate(http_requests_total{status=~"5.*",rollout-pod-template-hash="{{args.canary-hash}}"}[5m]))24              /25              sum(rate(http_requests_total{rollout-pod-template-hash="{{args.canary-hash}}"}[5m]))26            )27            -28            (29              sum(rate(http_requests_total{status=~"5.*",rollout-pod-template-hash="{{args.stable-hash}}"}[5m]))30              /31              sum(rate(http_requests_total{rollout-pod-template-hash="{{args.stable-hash}}"}[5m]))32            )

Observability Integration

Datadog deployment tracking from their documentation:

• Correlates deployments with metric changes
• Watchdog AI detects faulty deployments within minutes
• Compares new version against historical baselines

Prometheus alerting for deployment health:

Show 3 hidden lines4groups:5  - name: deployment-health6    rules:7      - alert: DeploymentErrorRateSpike8        expr: |9          (10            sum(rate(http_requests_total{status=~"5.*"}[5m]))11            /12            sum(rate(http_requests_total[5m]))13          ) > 0.0514        for: 2m15        labels:16          severity: critical17        annotations:18          summary: "Error rate exceeded 5% for 2 minutes"

Rollback Trigger Design

Avoid single-metric triggers: Combine multiple signals to reduce false positives.

1# Rollback when ANY of these conditions is true2conditions:3  - error_rate_increase > 1%4  - p99_latency_increase > 100ms5  - apdex < 0.8567# But suppress rollback if any of these gating exceptions hold8exceptions:9  - traffic_volume < 100_requests_per_minute # Insufficient data10  - baseline_error_rate > 5% # Already unhealthy

Cooldown periods: After rollback, wait before allowing re-promotion. Prevents flip-flopping between versions during transient issues.

Notification and audit: Log all automated rollback decisions with:

• Timestamp
• Triggering metric values
• Baseline comparison values
• Affected deployment/revision

Operational Playbooks

Pre-Deployment Checklist

Deployment Monitoring

First 5 minutes (critical window):

• Error rate by endpoint
• Latency percentiles (p50, p90, p99)
• Throughput anomalies
• Pod restart count

First hour:

• Memory usage trends (leak detection)
• Connection pool utilization
• Downstream service error rates
• Business metrics (conversion, orders)

First 24 hours:

• Full traffic pattern coverage (peak hours)
• Long-running job completion
• Background worker health
• Cache hit rates

Incident Response: Deployment-Related

Symptoms suggesting bad deployment:

• Metrics degraded immediately after deployment
• Only new pods showing errors (pod-specific issues)
• Errors correlate with traffic to new version (canary analysis)

Immediate actions:

1. Pause rollout (if in progress): kubectl rollout pause deployment/my-app
2. Check recent deployments: kubectl rollout history deployment/my-app
3. Compare error rates between old and new pods
4. Rollback if confident: kubectl rollout undo deployment/my-app

Rollback decision tree:

1Is the issue definitely caused by the deployment?2├── Yes → Rollback immediately3├── Unsure, but deployment was recent → Rollback (safe default)4└── No (unrelated issue) → Continue troubleshooting

Post-incident:

• Document what metrics detected the issue
• Determine if automated rollback should have triggered
• Adjust thresholds if false negative occurred
• Add regression test for the specific failure mode

Conclusion

Deployment strategy selection isn’t about finding the “best” approach—it’s about choosing which failure mode your organization can tolerate.

Blue-green provides instant rollback at 2x infrastructure cost. Choose when rollback speed is paramount and budget allows parallel environments.

Canary provides controlled blast radius at operational complexity cost. Choose when you need gradual exposure with automated metric gates, and have service mesh or traffic splitting infrastructure.

Rolling provides simplicity at rollback speed cost. Choose for straightforward deployments where version coexistence is acceptable and instant rollback isn’t critical.

Feature flags complement all strategies by moving release control to the application layer. Even with rolling deployments, flags provide instant feature-level rollback.

Database migrations constrain all strategies. The expand-contract pattern enables backward-compatible changes that preserve rollback capability throughout the deployment lifecycle.

The mature approach: combine strategies based on change risk. Routine dependency updates use rolling. New features with uncertain impact use canary. Critical infrastructure changes use blue-green with extended observation.

Appendix

Prerequisites

• Kubernetes Deployments, Services, and Ingress concepts
• Load balancer target group configuration (AWS ALB/NLB)
• Prometheus metric queries and alerting
• Database schema migration fundamentals
• Service mesh concepts (Istio VirtualService, DestinationRule)

Terminology

• Blast radius: Percentage of users affected by a deployment issue
• MTTR (Mean Time to Recovery): Average time from incident detection to resolution
• Expand-contract: Migration pattern that adds new structures before removing old ones
• Dual-write: Writing to both old and new data structures during migration
• Apdex: Application Performance Index—ratio of satisfactory response times
• Readiness probe: Kubernetes health check determining if a pod can receive traffic
• Rollout: Kubernetes term for a deployment’s update progression

Summary

• Blue-green offers instant rollback at 2x infrastructure cost—atomic traffic switching between parallel environments
• Canary enables metric-driven progressive exposure (1% → 100%)—requires service mesh or weighted routing
• Rolling updates are Kubernetes-native with maxSurge/maxUnavailable control—no automated metric-based rollback without Argo Rollouts
• Feature flags decouple deployment from release—enable instant feature rollback without infrastructure changes
• Expand-contract pattern enables backward-compatible schema migrations—preserves rollback capability
• Automated rollback requires baseline comparison, not absolute thresholds—minimum 50 data points for statistical significance

References

Foundational Articles

• Martin Fowler - Blue Green Deployment - Original definition and architecture, plus the database refactoring guidance
• Martin Fowler - Canary Release - Progressive traffic shifting concept; also defines IMVU’s “cluster immune system”
• Martin Fowler - Parallel Change - Expand–migrate–contract pattern (Joshua Kerievsky, 2006)
• Google SRE Workbook - Canarying Releases - Manual graph-staring is insufficient; automated analysis is required
• Charity Majors - Deploys Are The Wrong Way to Change User Experience - The deploy-vs-release distinction and the “feature flags as scalpel” framing
• Google CRE - What is a dark launch and what does it do for me? - The original definition of dark launch as response-discarded mirroring

Kubernetes Documentation

• Kubernetes Deployments - Rolling update parameters
• Kubernetes Probes - Liveness, readiness, startup configuration
• kubectl rollout - Rollout management commands

Progressive Delivery Tools

• Argo Rollouts Canary - Kubernetes canary implementation
• Argo Rollouts Analysis - Prometheus integration
• Flagger - Automated canary with service mesh integration
• Istio Traffic Management - VirtualService and DestinationRule
• Istio v1 APIs (1.22) - networking.istio.io/v1 promotion rationale
• Spinnaker Canary Best Practices - 50 data points, marginal/pass thresholds, baseline-vs-canary
• Netflix - Automated Canary Analysis with Kayenta - Three-cluster model, Mann-Whitney judgment

AWS Documentation

• AWS ECS Blue/Green Deployments - CodeDeploy integration
• AWS Blue/Green Whitepaper - Architecture patterns

Database Migration Tools

• gh-ost - GitHub’s online schema migration for MySQL
• pt-online-schema-change - Percona Toolkit
• pgroll - Zero-downtime PostgreSQL migrations

Feature Flags

• LaunchDarkly Progressive Rollouts - Automated percentage increase
• LaunchDarkly Deployment Strategies - Integration patterns

Observability

• Datadog Deployment Tracking - Correlation and analysis
• Google Cloud Canary Analysis - Threshold tuning guidance