Vertical scaling: increase resources on one machine. It is operationally simple but has hardware limits and may preserve a large failure domain.

Horizontal scaling: add instances. It requires traffic distribution, health checks, capacity planning, and usually stateless or externally coordinated application state.

Stateless application servers make horizontal scaling and replacement easier. Store durable shared state in appropriate external systems rather than process memory when requests may land on different instances.

Common bottlenecks

• CPU: optimize algorithms, profile hot paths, parallelize carefully, or add compute.
• Memory: bound caches and queues, inspect leaks, reduce object retention.
• Database: tune queries and indexes, reduce round trips, batch writes, partition when justified.
• Network: reduce payload sizes, reuse connections, move data closer to users.
• Lock contention: shorten critical sections, reduce shared mutable state, or partition contention domains.
• External dependencies: apply timeouts, concurrency limits, caching, and graceful degradation.

Latency percentiles

• p50 describes typical median latency.
• p95 and p99 expose slower experiences and tail latency.
• Averages can hide severe latency outliers.

Little's Law: average work in progress = throughput × average time in the system.

Capacity planning checklist

• Measure per-instance sustainable throughput under realistic load.
• Include peak traffic, dependency limits, and failure headroom.
• Load test normal, peak, and degraded conditions.
• Scale on meaningful signals such as saturation and queue age, not CPU alone.
• Verify that downstream databases and APIs can support added application instances.

Adding more application servers does not solve a bottleneck in a single database, a hot partition, or a rate-limited third-party service.