System Design & Architecture Interview Question 8: Retry Backoff Jitter
System Design & Architecture Interview Questions · Test 1 · question 8 of 10
A storage service had a 20-second outage. About 5,000 client processes call it, and each retries failed requests with exponential backoff of exactly 1 s, 2 s, 4 s and 8 s. After the outage ends, the service's request graph shows sharp spikes at the same offsets on every client. Each spike pushes the service back into overload, so recovery takes minutes instead of seconds.
What change to the clients would most directly fix this?
Show answer and explanation
Answer:
Every client failed at almost the same moment and uses the same deterministic schedule, so they all retry at the same moments. Exponential backoff alone spaces the retries out in time, but it does not stop the clients from staying synchronized. The waves line up and reinforce each other. Randomizing the delay (jitter, for example a random value between 0 and the backoff cap) spreads the same number of retries evenly across each interval. The service then sees a smooth, lower rate it can absorb. Limiting how many retries a client makes in total (a retry budget) complements this.
Option A keeps the retries synchronized and fires them sooner and more often, which makes the spikes taller.
Option B adds more retry waves to an overloaded service, so it multiplies the extra load instead of reducing it.
Option D is still deterministic, so the clients stay in lockstep. It also removes the growing gap that gives a struggling service time to recover.