Retries, Backoff, and Deadlines

A retry is another attempt after a failure believed to be transient. It cannot repair invalid input, an invariant violation, missing authorization, or a persistently overloaded dependency. Retrying consumes additional capacity, so it is an overload mechanism unless bounded carefully.

Classify before retrying

Retry only failures whose contract says another attempt may succeed: selected connection failures, temporary unavailability, or explicit rate-limit responses. The decision must also account for whether the previous attempt may already have committed an effect. Use idempotency when an ambiguous result can otherwise duplicate work.

Total deadline first

A per-attempt timeout bounds one call. A deadline bounds the complete user operation, including queueing, attempts, and delays. Each attempt should receive only the remaining budget. Stop when success would arrive too late to be useful, and propagate cancellation where the API supports it.

Capped exponential backoff with jitter

Without jitter, clients that fail together tend to retry together. One full jitter policy chooses a uniform random delay from zero up to, but excluding, the capped exponential value:

static Duration retryDelay(int retryIndex, Duration base, Duration cap,
                           RandomGenerator random) {
    if (retryIndex < 0 || base.isNegative() || base.isZero() || cap.isNegative()) {
        throw new IllegalArgumentException();
    }
    long baseMillis = base.toMillis();
    long capMillis = cap.toMillis();
    long shift = Math.min(retryIndex, 30);
    long exponential;
    try {
        exponential = Math.multiplyExact(baseMillis, 1L << shift);
    } catch (ArithmeticException overflow) {
        exponential = Long.MAX_VALUE;
    }
    long upperExclusive = Math.min(capMillis, exponential);
    return upperExclusive <= 0
            ? Duration.ZERO
            : Duration.ofMillis(random.nextLong(upperExclusive));
}

Avoid retry multiplication

If three layers each make three attempts, one request can cause up to 27 calls. Choose one layer with enough semantic context, set a maximum attempt count or retry budget, honor server retry hints when trustworthy, and cap concurrency. A circuit breaker or rate limit solves a different problem and should not be treated as another kind of retry.

Test retryable and permanent failures, ambiguous completion, deadline expiry, cancellation, jitter bounds, Retry-After, and recovery. Observe attempts per operation, final outcomes, delay, and budget exhaustion without logging secrets.