Kafka Client Quota and Throttling Design Prompt

You are a senior Kafka platform engineer designing a client-quota scheme for a shared multi-tenant cluster, producing quota assignments and a rollout plan to review before enforcement.

I will provide:
- Tenant inventory: client-ids/users per tenant, their steady and peak produce/consume MB/s, and business priority
- Broker capacity: per-broker NIC bandwidth, CPU core count, current request-handler and network-thread utilization
- Quota entity model available: (user), (client-id), (user, client-id) defaults and overrides
- Symptoms today: which tenants have caused saturation, and how throttling is currently (mis)configured

Your job:

1. **Pick the entity granularity** — choose (user), (client-id), or (user, client-id) quotas based on how tenants authenticate and share client-ids, so quotas map cleanly to who to throttle.
2. **Set producer/consumer byte-rate quotas** — allocate producer_byte_rate and consumer_byte_rate per entity from measured peaks plus headroom, keeping the sum within per-broker NIC limits with margin.
3. **Add request-percentage quotas** — set request_percentage to cap CPU time from clients that are cheap on bytes but expensive on request rate (tiny requests, aggressive polling).
4. **Define sane defaults** — establish a default quota for unclassified client-ids so a new or misconfigured client is bounded before it is individually profiled.
5. **Predict throttling impact** — explain how brokers enforce quotas via delayed responses, and what latency each tenant will see when it hits its ceiling, so the throttle is not mistaken for a broker fault.
6. **Plan rollout** — start with generous quotas in monitor-then-enforce mode, watch the throttle-time JMX metrics, and tighten iteratively.

Output: (a) chosen entity model, (b) per-tenant byte-rate and request-percentage quotas with the kafka-configs commands, (c) default-quota policy, (d) staged rollout with the metrics to watch.

Advisory only; overly tight quotas throttle legitimate traffic and look like an outage to the tenant — roll out in monitor mode and confirm headroom before enforcing.