2 Topics
Chapter•100 Minutes

Operating Systems, Runtime Behavior, and Resource Limits

Understand processes, threads, event loops, memory, file descriptors, connection pools, and graceful shutdown as production design constraints.

Processes, Threads, and Event Loops

A process has an isolated virtual address space and operating-system resources. Threads within a process share memory and can execute concurrently, but shared state requires synchronization. Concurrency means tasks make progress during overlapping periods; parallelism means tasks execute at the same time.

An asynchronous Node.js server can serve many requests while network I/O is pending, but CPU-heavy JavaScript can still block the event loop and delay unrelated requests. Move CPU-heavy work to worker threads or separate worker processes when profiling demonstrates a bottleneck. A queue does not create capacity by itself; it moves work across time and needs a bounded size and a defined overload policy.

When designing a service, identify which tasks are I/O-bound, CPU-bound, memory-bound, or constrained by an external quota. Select process count, worker count, concurrency, and shutdown behavior based on those constraints rather than using a fixed rule of thumb.

Key Takeaways
  • Async I/O does not make CPU-heavy work non-blocking.
  • Concurrency and parallelism are distinct.
  • Bound queued work and isolate CPU-intensive tasks.

Memory, File Descriptors, and Connection Pools

A memory leak is often an object that remains reachable, not merely a slow garbage collector. Unbounded caches, retained closures, event listeners, buffers, and queues can retain memory. Monitor heap, resident memory, allocation rate, garbage-collection pauses, and container limits because native memory may not appear in a language heap snapshot.

Sockets, files, and pipes consume file descriptors or similar handles. Leaks can prevent a service from accepting connections even when CPU is low. Connection pools reduce setup cost but multiply across instances: 20 application replicas with 30 database connections each can request 600 connections before administrative or background clients are counted.

Graceful shutdown should stop accepting new traffic, mark the instance unready, drain active requests, finish or release jobs safely, close pools, and exit within the orchestrator's deadline. Test shutdown while requests and jobs are in flight.

Key Takeaways
  • Garbage collection cannot reclaim reachable objects.
  • Pool sizing must account for all service replicas.
  • Graceful shutdown needs a bounded drain and recovery plan.