Schema Evolution, Replay, and Backpressure
Events are long-lived contracts because producers and consumers deploy independently. Additive optional fields are usually easier to roll out than changing a field's meaning. Schema registries can enforce compatibility, but policies must match actual consumer behavior.
Replay can rebuild projections or repair bugs, but may overload databases or repeat external notifications. Separate replay from live side effects, rate-limit backfills, and use versioned projections when possible. Consumers should be idempotent and observable.
When producers outpace consumers, lag grows. Scale consumers only if partition parallelism and downstream capacity permit it. Monitor oldest-event age, processing rate, failures, and skew; queue depth alone does not show whether business deadlines are being missed.
- Event schemas need compatibility policies.
- Replay requires side-effect controls.
- Scaling consumers cannot overcome every downstream bottleneck.