Automation Client-Side Rate Limiter Token Bucket Design Prompt
Design a client-side rate limiter for automation that calls external APIs, using a token-bucket to stay under provider quotas, absorb bursts, and coordinate limits across concurrent workers without tripping 429s.
- Target user
- Automation engineers integrating with rate-limited external APIs
- Difficulty
- Intermediate
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are an automation engineer whose fleet of workers hammered a third-party API past its quota, got rate-limited, and then made it worse by retrying immediately in lockstep. I will provide: - The target API, its documented rate limits, and the units they are enforced in (req/s, req/min, concurrent, cost-weighted) - How the limit is scoped (per key, per IP, per account) and any burst allowance - The number of concurrent workers/processes making calls and whether they share state - The response format for throttling (429, Retry-After header, custom error) Your job: 1. **Limit model** — translate [API]'s documented limits into a token-bucket configuration (rate, burst capacity, refill interval), accounting for cost-weighted or per-endpoint limits. 2. **Shared vs. local** — decide whether the bucket is per-worker or shared (Redis, a limiter service) given that [N] workers share one quota, and explain the coordination trade-off. 3. **Acquire semantics** — define how a caller acquires a token (block, timeout, or reject) and how that backpressure propagates to the automation instead of piling up in-flight requests. 4. **Respect server signals** — consume the API's Retry-After / rate-limit headers to adapt the bucket dynamically, so the client honors the server's actual state, not just its static config. 5. **Backoff on 429** — specify exponential backoff with full jitter for the throttled path, and why lockstep retries must be broken up. 6. **Headroom and fairness** — leave margin below the hard limit and ensure no single worker starves the others when the bucket is contended. Output as: a token-bucket configuration table, acquire/backoff pseudocode, the shared-state design if needed, and a load-test plan that verifies the client stays under quota at peak concurrency. Reconcile the limiter against the API's OFFICIAL documented limits and observed 429s under a staging load test; do not infer the limit from trial and error against production.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Why this prompt works
Automation that calls an external API is only as reliable as its worst burst. The failure this prompt addresses is self-inflicted: a fleet of workers collectively blows past a provider quota, earns 429s, and then makes it worse by retrying in unison. A token bucket is the right abstraction because it models exactly what quotas enforce — a sustained rate plus a bounded burst — so the prompt starts by translating the provider’s documented limits, including awkward cases like cost-weighted or per-endpoint limits, into concrete bucket parameters rather than a hand-picked sleep.
The decision that most implementations get wrong is per-worker versus shared state. A local token bucket in each of ten workers still allows ten times the intended rate, because the quota is enforced on the shared key, not per process. Forcing the model to reason about whether the bucket must be shared — and to pay the coordination cost when it must — is what makes the limiter actually bound aggregate traffic. The prompt then couples static configuration with dynamic server signals: honoring Retry-After and rate-limit headers means the client tracks the provider’s real remaining quota instead of a guess that drifts out of date.
Jitter is non-negotiable here, and the prompt says so. Uniform backoff after a 429 just reassembles the stampede a few seconds later, so full jitter on the retry path is what breaks the lockstep. The model can produce the configuration table and pseudocode quickly, but you reconcile the limits against official documentation and a staging load test rather than probing production, because discovering a quota by trial and error means discovering it by getting throttled — sometimes into a longer ban than a single 429.
Related prompts
-
Outbound Webhook Dispatcher with Backoff Design Prompt
Design the sender side of a webhook system — a dispatcher that delivers events to customer endpoints with bounded retries, exponential backoff with jitter, per-endpoint circuit breaking, and a dead-letter path — so a slow or flapping receiver cannot cause a retry storm or unbounded queue growth.
-
Automation Retry-Budget and Timeout Topology Design Prompt
Design end-to-end timeout and retry budgets across a multi-hop automation chain (trigger to queue to worker to downstream API) so retries do not stack into retry storms, exceed the caller's deadline, or hammer a degraded dependency.
-
Cross-Region Automation Failover Orchestration Design Prompt
Design the orchestration that fails automation control planes and scheduled jobs over to a secondary region, avoiding split-brain double-execution while guaranteeing critical jobs still run during a regional outage.
-
Automation Circuit Breaker Design Prompt
Design a circuit breaker around an ops automation so that when its actions start failing — or succeeding in ways that look like a runaway loop — it trips, stops acting, and escalates to a human instead of amplifying an outage.
More Automation prompts & error guides
Browse every Automation prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.