Idempotency Rules – HTTP, REST API, Idempotency Key & ETag

This post is a concept guide to idempotency, complete with exam-style questions and key terms.

In a Nutshell

Idempotency means executing an identical operation any number of times and getting the same result. Formally: f(f(x)) = f(x).

Core Definition

Idempotency is a property of an operation, not an endpoint. It makes the outcome stable under repetition. In HTTP, GET, HEAD, PUT, and DELETE are idempotent by specification; POST is not, but can be made idempotent using an Idempotency Key. In practice, idempotency ensures that retries produce no additional side effects (no duplicate orders, no double charges). Technically, you achieve this through deterministic state transitions, version checks with ETags, and deduplicating server logic. In distributed systems with at-least-once delivery, retries are normal—idempotency makes them safe.

Key Points for Study

• Definition: f(f(x)) = f(x); idempotency as an operation property, not a path property
• HTTP mapping: GET/HEAD are safe and idempotent; PUT/DELETE are idempotent; POST is not
• Making POST idempotent using Idempotency Key, request hashing, deduplicating store, deterministic responses
• Status and headers: 201 with Location on first attempt; retry returns 200/201 consistently; 412 on If-Match conflict
• Traceability of side effects; correlation ID logging; documented retry rules
• No sensitive data in errors; protection against replay via time-limited keys and signatures
• Fewer duplicate charges and support tickets; robust integrations; clearly defined retry policies
• OpenAPI describes idempotency, error catalog, key lifetime, example responses

Core Components

1. Formal definition and distinction from safe operations
2. Method semantics in HTTP: GET, HEAD, PUT, PATCH, DELETE, POST
3. Conditional requests with ETag, If-Match, If-None-Match
4. Deduplication: storage, key-to-response mapping, expiration
5. Deterministic state transitions and conflict detection (409, 412)
6. Idempotency Key generation: client creates a stable key per business event
7. Outbox/Inbox pattern for reliable side effects (events, payments)
8. Retry strategy: exponential backoff, respecting 429 and Retry-After
9. Observability: correlation IDs, tracing idempotent operations
10. Testing: retry tests, chaos tests, concurrency and race condition tests

Practical Example

// Idempotent POST for payment with Idempotency Key and ETag
POST payments
Headers: Content-Type: application/json, Idempotency-Key: pay-123-abc
Body: { orderId: 42, amount: 59.98, method: "card" }

Server logic:
if exists DedupeStore[key: pay-123-abc]:
    return storedResponse
else:
    if PaymentAlreadyConfirmed(orderId: 42):
        resp = { id: "P9001", status: "confirmed" }, code: 200
    else:
        create Payment(id: "P9001", status: "confirmed")
        resp = { id: "P9001", status: "confirmed" }, code: 201, headers: ETag: W/"r77"
    DedupeStore.save(key: pay-123-abc -> resp with ttl: 24h)
    return resp

Retry with the same key:
Client sends POST again with Idempotency Key; server returns the same response. First attempt yields 201, later retries yield 200, and the charge never happens twice.

Advantages and Disadvantages

Advantages

• Safe retries on network failures
• Protection against duplicate charges
• Clearer error modeling
• More robust integrations
• Better user experience

Disadvantages

• Extra storage and management overhead for deduplication
• More complex logic around side effects
• Requires precise definition of key lifetime and semantics
• Potential locks or conflicts under concurrency

Common Exam Questions (with Brief Answers)

1. Difference between safe and idempotent? Safe means no server-side state change (GET); idempotent means multiple executions produce no additional side effects (PUT/DELETE).

2. How do you make POST operations idempotent? Use an Idempotency Key per business event, maintain a deduplication store mapping key to response, apply deterministic logic, return consistent status codes, and define key expiration.

3. What role do ETag and If-Match play in idempotency? They prevent lost updates by allowing changes only when the version matches; return 412 Precondition Failed otherwise.

4. Is DELETE truly idempotent? Yes. The first DELETE returns 200/204; later DELETEs may return 204/404, but the system state (resource deleted) remains consistent.

5. Typical status codes for idempotent operations and retries? 200/201/204 on success; 409 for business conflicts; 412 for failed preconditions; 429 for rate limits (with Retry-After); 5xx for temporary server errors.

6. How do you define Idempotency Key lifetime? Semantically until the business event is final; technically as a time window (e.g., 24 hours), documented in your API.

7. What are the risks of skipping idempotency in distributed systems? Duplicate side effects (duplicate charges), inconsistent state, manual corrections, higher support costs.

8. How does idempotency work with Event Sourcing and Outbox? Write the event to an Outbox in the same transaction; publish it reliably; the consumer uses an Inbox to track processed event IDs and runs an idempotent handler.

Key Sources

1. https://www.rfc-editor.org/rfc/rfc9110
2. https://www.rfc-editor.org/rfc/rfc7807
3. https://docs.stripe.com/idempotency