# Exponential Backoff Exponential backoff is a retry strategy in which the wait between attempts grows exponentially after each failure. A common form is `wait = min(cap, base × 2^attempt)`: with a 100 ms base, the waits are 100 ms, 200 ms, 400 ms, 800 ms and so on, up to a maximum. The same idea schedules retransmissions after collisions in Ethernet. In the vault, [[Cloudflare Workflows]] and [[Cloudflare Queues]] retry with exponential backoff, for example. ## Why it matters [[Retry pattern|Retries]] help when failures are rare and short. When failures come from overload, they make things worse: every retry is extra traffic sent to a service that is already struggling. Marc Brooker calls retries "selfish": a client that retries spends more of the server's time to raise its own chance of success. Backing off spreads that extra load over time and gives the service room to recover, instead of feeding the retry storms behind many [[Cascading Failures]]. ## Add jitter Backoff alone isn't enough. Clients that failed at the same moment back off by the same amounts, so they come back at the same moments and collide again. Jitter adds randomness to the wait. In "Exponential Backoff And Jitter" (AWS Architecture Blog, 2015), Brooker compared variants with a simulation of 100 contending clients: - **Full Jitter**: `wait = random(0, min(cap, base × 2^attempt))`. It cut the number of calls by more than half compared with plain exponential backoff, and finished sooner - **Equal Jitter**: keep half of the backoff and randomize the other half, which avoids very short waits - **Decorrelated Jitter**: grow the maximum wait based on the previous random value Plain exponential backoff without jitter was the clear loser on both work and completion time. Most AWS SDKs now apply exponential backoff with jitter in their standard and adaptive retry modes. ## Limit the retries - **Retry at one layer.** In a stack five services deep with three attempts at each layer, a failing database receives up to 243 times the original load (AWS Builders' Library) - **Use a retry budget.** AWS SDKs have used a token bucket since 2016: calls retry while tokens remain, then at a fixed rate. Google's SRE book gives the example of a process-wide budget of 60 retries per minute - **Retry only what is safe to repeat.** A timeout doesn't mean the operation failed; it may have succeeded without the client hearing back. APIs with side effects need [[Idempotency]] (e.g., idempotency keys) before retries are safe - **Stop retrying when the dependency is down.** That's the job of a [[Circuit Breaker]] ## References - Marc Brooker, "Exponential Backoff And Jitter", AWS Architecture Blog (4 March 2015, updated May 2023): https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/ - Marc Brooker, "Timeouts, retries, and backoff with jitter", Amazon Builders' Library (2019): https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/ - Google SRE book, "Addressing Cascading Failures" (retry budgets): https://sre.google/sre-book/addressing-cascading-failures/ - Wikipedia, Exponential backoff: https://en.wikipedia.org/wiki/Exponential_backoff - https://concepts.dsebastien.net/concept/exponential-backoff/ ## Related - [[Cascading Failures]] - [[Idempotency]] - [[Circuit Breaker]] - [[Timeouts (software)]] - [[Load Shedding]] - [[Backpressure]] - [[Site Reliability Engineering (SRE)]] - [[Cloudflare Workflows]] - [[Cloudflare Queues]] - [[Fault Tolerance]] - [[Rate Limiting]] - [[Retry pattern]]