Durability, Retries, and Human-in-the-Loop

Long-running workflows can outlive a single HTTP request or process. Use a durable job/workflow system when tasks may suspend, retry, wait for approval, or resume after deployment. Persist step status and idempotency keys; make each step safe to retry or compensate where possible.

Retries must be selective. Retry transient network errors, rate limits, and temporary service failures with exponential backoff and jitter, respecting provider retry guidance. Do not blindly retry invalid input, policy denials, or actions that may have already succeeded. Escalate ambiguous outcomes for side-effecting operations.

Retry with bounded exponential backoff

async function retry(operation, attempts = 4) {
  let lastError;
  for (let i = 0; i < attempts; i++) {
    try { return await operation(); }
    catch (error) {
      lastError = error;
      if (!isRetryable(error) || i === attempts - 1) throw error;
      const delay = Math.min(500 * 2 ** i, 8000) + Math.random() * 250;
      await new Promise(resolve => setTimeout(resolve, delay));
    }
  }
  throw lastError;
}

💡 isRetryable must classify errors carefully. A retry policy should also honor Retry-After headers where applicable, cap total elapsed time, and avoid repeating non-idempotent side effects without deduplication.

Key Takeaways

Persist long-running state durably.
Retry only errors classified as transient.
Use human approval for sensitive or ambiguous actions.

Knowledge Check

Which failure should generally not be retried automatically without changing the request?

Durability, Retries, and Human-in-the-Loop

STAY UPDATED WITH BYTES!