Imagine a small service that creates a downloadable export. A reader selects a date range and clicks Create. The server accepts the request, records the job, and queues the work. Before the response reaches the browser, the connection drops.
The browser shows an error and offers Try again. The reader clicks it. A second job appears, both workers run, and two files eventually arrive. Every component followed its local instructions. The missing piece was a shared answer to what the first attempt had done.
Follow the lost response
Consider this illustrative sequence. The timings are invented; the ordering is the detail that matters.
- 0 ms · Send
- The browser requests one export for a selected date range.
- 90 ms · Commit
- The server durably records job E-42 for that request.
- 120 ms · Reply
- The server sends the job identifier, but the connection fails.
- 5 s · Timeout
- The browser gives up waiting while E-42 continues to exist.
A longer timeout might help on a slow connection, but it cannot guarantee delivery of that reply. Cancellation has a similar limit: closing the browser request does not establish that the server canceled the work. A cancellation operation needs its own confirmed outcome.
The Amazon Builders’ Library discussion of timeouts makes this uncertainty explicit: a timeout can occur after side effects have happened. Before repeating an operation, establish what repetition would mean.
“No response received” is a useful observation. “Nothing happened” requires additional evidence.
Inspect before repeating
In the export example, a useful recovery screen can ask the service about the original operation. That requires an identifier the client knows before the reply is lost. If the only identifier is generated by the server and returned in the missing response, the client has no reliable handle for this lookup.
Define the lookup results carefully. Completed means open the existing export. Running means keep following that job. Rejected before admission means explain the rejection. Unknown means preserve uncertainty. A search returning no rows may be reading a delayed replica, or the original request may still be arriving.
For maintenance work, the equivalent handle may already exist: the expected release revision, a unique job name, or an operation record. After a deployment command loses its connection, inspect the active revision and the service behavior before sending the deployment again. The release record provides something concrete to compare.
Preserve the same intent across attempts
HTTP defines idempotence in terms of intended effect: repeating an identical request has the same intended server effect as sending it once. The response need not be identical. RFC 9110, section 9.2.2, limits automatic retries of non-idempotent requests to cases where their semantics or non-application are known.
For a creation API, a caller-supplied operation key can express one intent across several attempts. Keep the same key when recovering an uncertain request; use a new key for a deliberately new operation. The server must define the key’s caller scope, reject changed parameters under an existing key, and make recording the key and its protected effect atomic. Key retention also needs a documented limit. AWS covers these requirements in Making retries safe with idempotent APIs.
For our export service, that protection should create one durable job record for the reader’s key. A retry returns E-42 or its current state. The worker still has a separate obligation: if it retries delivery to an external destination, that destination needs appropriate duplicate protection too. Protecting job creation alone does not protect every later effect.
An operation key is not an access credential. Looking up E-42 must still check that the signed-in reader owns it. Likewise, disabling the button improves the interface but cannot protect against a second tab, a restarted client, or two requests already in flight.
Give retries a budget
Once repetition is safe, decide how much of it is useful. Put the retry policy at a deliberate layer, cap total attempts, and apply an overall deadline that includes waits. Backoff spaces attempts apart; jitter spreads clients across that wait so they do not all return together. These are the load-control reasons behind the Builders’ Library retry guidance.
Write “three attempts total” if that is the limit. “Three retries” often means four attempts, and nested policies multiply: a wrapper with three attempts around a client with three attempts can issue nine downstream calls for one action.
For the export screen, budget admission attempts separately from status polling. A slow export should not cause repeated creation requests. A poll that cannot reach the server can wait within its own budget while retaining the same job identity. When that budget expires, the screen should preserve a path back to the job.
Show an honest status
A useful message for the lost reply is: “We could not confirm whether your export started. Checking this request…” It tells the reader what is uncertain and what the application is doing next. If checking remains unavailable, retain the request reference and let the reader check again later.
For operators, keep the operation key, attempt number, timestamps, observed response, reconciliation result, and final disposition together. Avoid logging the whole request body when a reference will do. This small record can explain whether a retry recovered an answer, repeated work, or stopped with an unresolved outcome.
Stopping retries is a local decision. Marking the job failed is a claim about the service. Give those events different names in the interface and in logs.
Test success without a reply
The most revealing test for the export service drops the response after the job record commits. The client then follows its normal recovery path. Assert that it reaches the original job and that only one job exists for the operation key. A test that fails before the server receives anything cannot establish this behavior.
- Lost reply: commit the job, lose the response, and recover E-42.
- Concurrent attempts: submit the same key together and admit one job.
- Changed intent: reuse the key with another date range and reject the mismatch.
- Client restart: recover the pending reference instead of silently creating a fresh request.
- Expired protection: exercise a late retry after key retention ends and verify the documented recovery path.
- Budget exhausted: stop sending attempts while keeping the uncertain outcome visible.
A timeout becomes manageable when the next action follows evidence. Preserve the original intent, locate its state, and repeat only under a contract that makes repetition safe. Then a dropped reply costs time without quietly turning one request into two.