System design · Day 7
Queues and asynchronous work: decouple the slow path
Learn when to move work out of a request, how workers consume messages, and why delivery guarantees demand idempotency.
A user should not wait for work that does not need to finish before the response. A queue separates accepting a task from completing it.
Separate the fast path from the slow path
Imagine image upload: the request can validate and store the original, then publish a message for thumbnail generation. Workers process the message later. The API responds quickly, and a temporary processor slowdown does not make uploads unavailable.
Queues work well for email, analytics, media processing, search indexing, and fan-out notifications. Keep the synchronous path for results the user needs now; move independent side effects behind a durable message.
Design for at-least-once delivery
Many practical queues deliver a message at least once. A worker may finish but crash before acknowledging, so the queue sends the message again. Consumers must be idempotent: processing the same message twice should not create two charges or two welcome emails.
Give each operation a stable ID and record completion, or make the write naturally upsert the same result. Failed messages should retry with backoff and eventually enter a dead-letter queue for inspection rather than looping forever.
Backpressure is information
Queue depth and message age reveal whether producers are outrunning consumers. Add workers when the job is parallelizable, limit producers when downstream capacity is fixed, and alert before the oldest message violates the product's promise.
Queues decouple time, but data still needs to scale. Day 8 returns to the database and explains how replication and partitioning distribute reads, storage, and failure risk.
“Asynchronous does not mean eventually forgotten; it means explicitly owned by another path.”