Q5: What is the difference between latency and throughput?
Latency is the time taken for one operation, often reported as p50, p95, and p99 percentiles. Throughput is the amount of work completed per unit time, such as requests per second. A system can have high throughput but poor tail latency if queues build up. Measure both under realistic load and consider end-to-end latency, including network and downstream calls.
Key Takeaways
- Latency measures time per operation; throughput measures work per unit time.
- Tail latency often reveals problems hidden by averages.