Retry, Reset & Recovery
Recovery behavior should match the failure. A transient network failure may be retryable; a permission error requires authentication or authorization; invalid user input requires correction.
Resetting a failed subtree with a new key can be useful when local state has become inconsistent. Retrying should avoid creating infinite request loops and should provide feedback about the attempt.
Logging should capture enough context to diagnose the failure without exposing secrets or sensitive user data.
Key Takeaways
- Different failures need different recovery actions.
- Retries need limits and backoff where appropriate.
- Observability should be useful without leaking sensitive information.