--- name: queue-flow-control-decision description: Invoke when a queue grows at peak with no capacity plan, tail latency spikes despite healthy throughput, someone says "the queue will absorb spikes", message value decays over time, or rate-limiting a third-party API behind a queue. Chooses TTL, tail drop, or backpressure. --- # Queue Flow Control Decision ## Queue Flow Control Decision **Source:** Enterprise Integration Patterns, Gregor Hohpe & Bobby Woolf (2003) + "Queues Require Flow Control" (Gregor Hohpe, addendum 4, ~2022) — Message Expiration pattern + addendum §"Three main mechanisms provide flow control" ______________________________________________________________________ ### R — Reading (Original Source) > "Queues invert control flow but require flow control. Three main mechanisms provide flow control to avoid filling a queue beyond its manageable or useful size. Time-to-live (TTL) drops old messages when capacity is reached. A Tail drop does the opposite by dropping new messages that arrive when the queue is full. Backpressure informs upstream systems that the queue isn't able to handle incoming messages so that those systems can reduce the arrival rate. Although neither option appears particularly appealing, implementing explicit flow control is much better than letting excess traffic take its course." ______________________________________________________________________ ### I — Interpretation A queue smooths traffic spikes and decouples producers from consumers, but it is not a free buffer. When arrivals consistently exceed processing capacity, something must give. The worst outcome is to let the queue grow without bound: by Little's Result (W = L/λ — wait time equals queue length divided by arrival rate), an unbounded queue produces unbounded wait times. The system may look healthy on every dashboard — workers are processing, the messaging system reports no errors, queue depth is "just high" — but the messages it delivers are so stale they carry no business value. This is the "all lights are green, the system is down" failure mode. Three mechanisms exist for preventing this. Each one makes a different assumption about what is more valuable to preserve: **TTL (Time-to-Live):** Drop the oldest messages when the queue reaches capacity. Right choice when message value decays over time — a stock quote, a sensor reading, or a coffee order is worth acting on now but not in ten minutes. Newer messages are fresher and therefore more valuable. Downside: high-value in-flight messages can be silently dropped; senders may not know. **Tail Drop:** Reject new arrivals when the queue is full. Right choice when old messages are more valuable — an order that was placed first deserves to be processed first; the backlog represents committed work. New requests see an error immediately (HTTP 503 equivalent) and can retry or seek alternatives. Downside: looks like an outage to new users. **Backpressure:** Signal upstream systems to slow their arrival rate before the queue fills. Right choice when senders are capable of responding to a "slow down" signal — a UI can display "service busy, please wait"; a producer can pause. Produces the cleanest user experience: no messages are dropped, processing rate is preserved, and users get an honest status signal. Downside: requires sender-side support; not all senders can throttle. A proactive variant — **rate limiting** — sets a fixed maximum throughput upfront rather than reacting to queue depth. EventBridge Pipes' Invocation Rate and GCP Pub/Sub's Slow Start Algorithm are examples. All three reactive mechanisms plus rate limiting can coexist in a single system. ______________________________________________________________________ ### A1 — Past Application (Author's Cases) **Serverlesspresso — backpressure in a coffee shop demo.** The AWS Serverless DA team built a demo coffee-ordering app with two baristas as the backend. Free-coffee demand at events easily overwhelmed capacity. They initially used a queue, but discovered that customers abandon orders after about two minutes — so processing a queued order after five minutes wastes the physical coffee and produces no value. Instead of TTL (which would have silently prepared abandoned orders), they implemented backpressure: new orders are rejected once the queue reaches a threshold that corresponds to roughly a two-minute wait. Users see "queue full, try later" immediately rather than waiting and then abandoning. **AWS SQS message visibility timeout as TTL proxy.** SQS does not have a native TTL that drops messages when the queue is full, but placing an upper bound on how long a message can sit in the queue (via visibility timeout and message retention period) is functionally equivalent: if the consumer cannot process a message within that window, the message expires. This is appropriate for requests where the requester has already given up or retried. **AWS ALB 503 responses as tail drop.** An Application Load Balancer that rejects excess traffic with HTTP 503 is implementing tail drop: existing connections are served, new arrivals are turned away. This is appropriate when in-flight requests represent committed work (partial transactions, ongoing sessions) that must complete. **RabbitMQ connection-level flow control as backpressure.** RabbitMQ throttles publisher connections when internal queues exceed configured thresholds. Producers that respect flow control signals pause their sends, giving consumers time to drain the backlog. This is appropriate when producers are application services under the operator's control. **GCP Pub/Sub Slow Start Algorithm as rate-limiting backpressure.** GCP automatically backs off message delivery rates when acknowledgment rates drop below 99% or processing latency exceeds one second. This provides automatic, producer-transparent rate limiting without requiring explicit flow-control logic from the subscriber. **The coffee shop and the "stale orders" failure mode.** Hohpe describes a queue without TTL at a busy coffee shop: orders keep arriving, baristas keep making drinks for customers who have long since left. Every order gets made; the queue is drained; nothing is technically wrong. But no value is delivered. Little's Result makes this quantitative: a queue of 50 orders draining at 10 orders/minute produces an average wait of 5 minutes. A queue of 100 orders at the same rate produces 10 minutes. If customers abandon after 2 minutes, any queue deeper than 20 orders is producing wasted work. ______________________________________________________________________ ### A2 — Future Trigger ★ Invoke this skill when: - A queue is growing during peak load and there is no explicit plan for what happens when it reaches capacity - A service is showing high tail latency during traffic spikes despite healthy throughput — suspect a queue that is growing, increasing wait times for all messages - A system processes messages successfully during load tests but delivers stale or irrelevant results in production — suspect an unbounded queue absorbing spikes and delaying delivery past the useful window - You are asked to choose between SQS message retention settings, ALB connection limits, or RabbitMQ flow control thresholds — these are all flow control mechanism choices - A team says "the queue will handle traffic spikes" with no mention of what happens at the queue's capacity limit - Message value clearly decays over time (real-time pricing, sensor data, alerts, user session events) — TTL should be explicitly sized - Upstream senders are services or UIs you control — backpressure is worth designing in - You are implementing a rate limit on calls to a third-party API downstream of a queue — choose the mechanism that fits the message value and sender capabilities ______________________________________________________________________ ### E — Execution (Steps) 1. **Apply Little's Result to the current design.** Estimate: peak arrival rate (λ), target maximum acceptable wait time (W), and therefore the maximum queue depth that produces acceptable latency (L = W × λ). Any queue that can grow beyond L is delivering stale messages to at least some consumers. 2. **Determine whether message value decays over time.** Ask: if this message sits in the queue for 2× the normal processing time, is it still worth acting on? If no: TTL is the primary mechanism. If yes (in-flight orders, financial transactions, committed work): TTL is dangerous; proceed to step 3. 3. **Determine whether senders can respond to a slow-down signal.** Ask: can the sending application pause, queue locally, or display a "service busy" message? If yes: backpressure is the cleanest option. If no: the sender cannot throttle, so backpressure won't help; proceed to step 4. 4. **Select tail drop if neither TTL nor backpressure applies.** Configure explicit rejection of new arrivals when the queue exceeds a defined depth. Ensure the rejection is visible to callers (HTTP 503, error response, DLQ entry) rather than a silent discard. 5. **Define the flow control threshold.** For all three mechanisms, the threshold is the queue depth corresponding to the maximum acceptable wait time from step 1. Set the threshold explicitly; do not rely on the messaging system's default capacity limit as an implicit ceiling. 6. **Consider stacking multiple mechanisms.** TTL removes stale messages from the backlog (freeing capacity), tail drop prevents new arrivals from making the backlog worse, and backpressure reduces the arrival rate. All three can be active simultaneously and address different parts of the overload scenario. 7. **Test the flow control path explicitly.** Generate load that exceeds consumer capacity. Verify that the expected mechanism engages — not a different implicit one. Monitor the mechanism in production (TTL expirations, 503 rates, RabbitMQ flow control events) as a leading indicator of sustained overload. ______________________________________________________________________ ### B — Boundary (When Not to Apply) **Flow control is irrelevant for consistently under-loaded queues.** If the consumer reliably drains the queue faster than the producer fills it under all load scenarios (including peak), flow control mechanisms are dormant and their design cost is overhead. Size queues and consumers against actual load profiles before investing in flow control design. **Little's Result assumes steady-state.** W = L/λ holds for average behavior over time. During a sudden spike, the instantaneous queue depth and wait time can deviate significantly from the steady-state prediction. Don't use steady-state calculations to justify eliminating flow control for burst scenarios. **Backpressure requires end-to-end design.** Designing backpressure into a single queue segment does not protect the whole pipeline if upstream components buffer messages locally rather than pausing. True backpressure must propagate to the ultimate source of load (typically an external request or user action). In practice, this is architecturally complex and sometimes impossible for externally controlled senders. **TTL creates silent data loss.** In systems where every message represents a committed business transaction (payment instruction, irreversible order), TTL is the wrong mechanism regardless of message arrival patterns. Use tail drop or backpressure to preserve the semantics of "every accepted request will eventually be processed." **The "no limit" claim by cloud providers.** AWS SQS documentation says queues have "no limit" on message count. This means the messaging system will not reject messages at a software-imposed limit — it does not mean infinite storage or zero latency growth. Physical retention windows and cost per message still apply. "No limit" does not mean "no flow control needed." ______________________________________________________________________ ## Related Skills - depends-on: `queue-control-flow-model` — The control-flow model identifies who controls the cadence of a pipeline; flow control decision applies once you know whether the active component is a Driver (can slow fetch rate) or a Sender (requires a queue buffer before flow control is possible). - **Competing Consumers vs. Dispatcher** — *composes-with* → The number of competing consumers directly affects queue drain rate and therefore the correct flow control threshold; flow control sizing and consumer count are co-determined. - depends-on: `messaging-observability-design` — Detecting queue overflow and measuring wait times (the inputs to Little's Result) requires Message Store and Wire Tap instrumentation; flow control cannot be tuned without observability in place. - depends-on: `integration-style-selection` — Flow control is only a concern after Messaging is selected; if RPC or File Transfer is the integration style, queue overflow is not a relevant problem. ______________________________________________________________________ ## Provenance - **Source:** Enterprise Integration Patterns, Gregor Hohpe & Bobby Woolf (2003) + Addendum 4 (Gregor Hohpe, ~2022)