Vertical scaling gives a machine more CPU, RAM, or I/O capacity. It is operationally simple until hardware limits, cost, or maintenance constraints become problematic. Horizontal scaling adds instances and can improve throughput and fault tolerance, but requires load distribution, shared or partitioned state, and coordination.

Stateless application servers are comparatively easy to scale horizontally. Local in-memory sessions, uploaded files, and scheduled jobs tied to one process make scaling harder unless state is externalized or ownership is coordinated.

Scaling out does not fix every bottleneck. A shared database, hot key, global lock, or external API quota can remain the limiting component. Measure the end-to-end critical path and scale the constrained resource rather than adding application instances blindly.

Average latency can hide severe user experiences. p95 latency is the value below which 95% of measured requests complete; p99 highlights the slowest tail. Tail latency can be driven by garbage collection, noisy neighbors, queueing, slow dependencies, or a small number of overloaded partitions.

As utilization approaches capacity, queueing delay can rise sharply. A service that accepts work faster than it can process it accumulates a queue, increases memory usage, and eventually times out or crashes. Backpressure limits admission, slows producers, or rejects excess work before the system becomes unstable.

Use bounded queues, concurrency limits, load shedding, and explicit overload responses. A queue is not infinite storage. Track queue depth, oldest-message age, processing rate, and rejection rate to understand whether the system is recovering or falling further behind.

A load test should represent realistic request mixes, payload sizes, think time, authentication, cache hit rates, and data distributions. Define whether the goal is baseline capacity, stress behavior, spike response, endurance, or recovery testing.

Increase load gradually and record throughput, latency percentiles, errors, CPU, memory, garbage collection, database saturation, connection pools, queue depth, and downstream latency. A system may show rising latency before throughput collapses; that is often a sign of saturation.

Avoid unrealistic tests that warm every cache, use tiny datasets, or run from a single client that becomes the bottleneck. Compare runs with a known baseline and change one major variable at a time. Production canaries and gradual rollout complement, but do not replace, controlled load testing.