Retries, Circuit Breakers, Bulkheads, and Fallbacks

Retries recover from transient failures, but must be bounded and safe. Exponential backoff increases the delay after each failed attempt; jitter spreads retries so clients do not synchronize into a retry storm.

A circuit breaker temporarily stops calls to a dependency after a failure threshold, allowing recovery and preventing repeated work against an unhealthy service. It typically moves among closed, open, and half-open states. Thresholds and probe behavior should be tuned from observed traffic and failure patterns.

Bulkheads isolate resources—for example, separate connection pools or concurrency limits for different dependencies—so one slow service does not consume every worker. Fallbacks should be semantically safe: serving a stale product description may be acceptable, while inventing a successful payment result is not.

Circuit breaker states
Key Takeaways
  • Retries, breakers, and bulkheads solve different failure problems.
  • Fallbacks must preserve business correctness.
  • Test recovery and half-open behavior, not only failure detection.