Q32: How would you design a large-scale logging platform?

Difficulty:AdvancedTopic:Design Problems

Applications emit structured logs to agents or collectors, which batch and buffer events before sending them to a durable stream. Consumers enrich, redact, and route events to indexed search storage or cheaper object storage. Use partitioning by time and service, retention tiers, sampling for high-volume debug logs, and access controls for sensitive fields. Protect application performance with bounded buffers and a defined policy for backpressure or dropped logs.

Key Takeaways
  • Separate ingestion, processing, indexing, and long-term storage.
  • Bound logging overhead and redact secrets before broad distribution.