# Timeouts (software) A timeout is an upper bound on how long code waits for something outside its control: a network call, a database query, a lock, a connection from a pool. When the bound passes, the caller stops waiting, releases what it was holding, and handles the failure. Without one, a caller waits as long as the other side takes, which can mean forever. ## Why it comes first [[Michael Nygard]] lists Timeouts first among the stability patterns in [[Release It! (Book)|Release It!]], and Blocked Threads and Slow Responses among the antipatterns. The book grew out of production outages he debugged: application threads that took a connection from a pool with no timeout and waited indefinitely, while another piece of code held a connection, threw an exception and never released it. Every waiting thread is a thread that can't serve anything else, so one slow dependency can drain a whole server. Libraries don't always set one for you. gRPC sets no deadline unless you set one, so a client can end up waiting effectively forever. Amazon's practice is to set a timeout on every remote call, and generally on every call across processes, even on the same machine. ## The different timeouts - **Connect timeout**: the limit on establishing a connection - **Request (read) timeout**: how long to wait for the response once connected - **Pool acquisition timeout**: the wait for a free connection or thread from a pool, the one missing in Nygard's story - **Overall deadline**: how long the whole operation may take, across [[Retry pattern|retries]] and downstream calls Some client implementations don't cover every step (DNS lookups or TLS handshakes, for example), so check what a given timeout actually bounds. ## Deadline propagation When a service calls other services, each hop should know how much time is left. gRPC expresses this as a deadline, a point in time after which the client no longer wants the answer; some implementations pass it on to outgoing calls automatically (Java and Go by default, C++ when enabled). A server that is past the deadline cancels the call instead of finishing work nobody will use. Amazon propagates the remaining time between hops for the same reason, which also feeds [[Load Shedding]]. ## Choosing a value Too long, and resources stay tied up while the client waits. Too short, and requests that would have succeeded get retried, adding load; a small rise in backend latency can then turn into a full outage when every request starts retrying. Amazon's starting point for calls inside a region: pick an acceptable rate of false timeouts (e.g., 0.1%) and use the matching latency percentile of the downstream service (p99.9). That needs adjusting for clients far away on the internet, and for services where p99.9 sits very close to p50. ## How it fits with the other patterns - A [[Circuit Breaker]] counts timeouts as failures; without timeouts, it never trips - [[Bulkheads]] cap how many resources can be stuck; timeouts cap how long - A timed-out call may still have succeeded on the other side. Retrying it needs [[Idempotency]], and retries need [[Exponential Backoff]] with jitter so they don't pile onto a struggling service - Timeouts are what turn a slow dependency into a visible failure the rest of the system can handle, which is how they stop [[Cascading Failures]] ## References - Michael Nygard, Release It! Second Edition, Pragmatic Bookshelf, 2018 (Stability Antipatterns and Stability Patterns chapters): https://pragprog.com/titles/mnee2/release-it-second-edition/ - GOTO Unscripted, "Building Software That Survives" (2025), Nygard on the outages behind the book: https://gotopia.tech/articles/408 - Marc Brooker, "Timeouts, retries, and backoff with jitter", Amazon Builders' Library (2019): https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/ - gRPC documentation, Deadlines: https://grpc.io/docs/guides/deadlines/ - David Yanacek, "Using load shedding to avoid overload", Amazon Builders' Library (2019): https://aws.amazon.com/builders-library/using-load-shedding-to-avoid-overload/ - Google SRE book, "Addressing Cascading Failures": https://sre.google/sre-book/addressing-cascading-failures/ ## Related - [[Circuit Breaker]] - [[Bulkheads]] - [[Exponential Backoff]] - [[Idempotency]] - [[Cascading Failures]] - [[Load Shedding]] - [[Fail Fast]] - [[Fault Tolerance]] - [[Release It! (Book)]] - [[Michael Nygard]] - [[Retry pattern]]