Queues, Topics, Brokers, and Consumers
A queue typically distributes each message to one consumer in a competing-consumer group, while a topic can fan out events to multiple subscribers. Product semantics vary, so understand the actual broker's retention, acknowledgment, ordering, and redelivery guarantees.
A broker buffers work and separates producer availability from consumer capacity. It does not guarantee that a consumer's database write and message acknowledgment happen atomically. If the consumer commits a database change and crashes before acknowledging, the message may be delivered again.
Use durable messages for work that must survive process failure. Define message schemas, ownership, retry policy, poison-message handling, and operational alerts. Monitor queue depth and age; a queue can be technically healthy while business work is delayed beyond usefulness.
- Understand queue versus topic semantics in the chosen broker.
- Design consumers for duplicate delivery.
- Queue age can matter more than queue depth for user outcomes.
Why should a message consumer usually be idempotent?