RabbitMQ on Kubernetes Cluster Operator Deployment Design Prompt

You are a senior platform engineer who has run RabbitMQ on Kubernetes in production using the RabbitMQ Cluster Operator. Help me design a deployment that survives node drains, pod rescheduling, and rolling upgrades without losing messages or splitting the cluster.

I will provide:
- Target throughput and message profile: publish/consume rates, average and max message size, persistence requirement, and how many queues/connections [DESCRIBE]
- The Kubernetes environment: managed or self-hosted, node count and zones, available StorageClasses, and whether pod anti-affinity across zones is possible [DESCRIBE]
- Availability target: acceptable downtime, whether publishers can tolerate brief backpressure, and RPO for durable messages [DESCRIBE]
- Any existing manifests: the RabbitmqCluster CR, StorageClass, and resource requests/limits [PASTE IF ANY]

Your job:

1. **Size the cluster** — recommend an odd `replicas` count (3 or 5, never 2) so quorum queues and the underlying Raft/Mnesia layer keep a majority when one pod is down. Explain why an even count is dangerous under a single node drain, and set CPU/memory requests and limits that leave headroom below the `vm_memory_high_watermark`.

2. **Choose queue types** — default to quorum queues for durability and predictable failover on Kubernetes, and explain when streams or classic queues are appropriate. Call out that classic mirrored queues are deprecated and a poor fit for a rescheduling-heavy environment.

3. **Persistence and identity** — specify a StorageClass with `volumeBindingMode: WaitForFirstConsumer`, per-pod PVCs via the StatefulSet the Operator manages, and why stable network identity (headless service, `rabbitmq-peer-discovery-k8s`) matters for cluster formation and recovery.

4. **Scheduling and resilience** — pod anti-affinity (ideally topology-spread across zones/nodes), a PodDisruptionBudget with `maxUnavailable: 1`, readiness/liveness probes, and `terminationGracePeriodSeconds` long enough for a clean node shutdown so queues migrate leadership before the pod dies.

5. **Upgrades** — how the Operator performs a rolling StatefulSet update (one pod at a time, ordered), why you must wait for each pod to fully rejoin and rebalance quorum leaders before the next, and how feature-flag state and mixed-version windows are handled.

6. **Config and observability** — the `additionalConfig` / `rabbitmq.conf` settings that matter on Kubernetes (partition handling `pause_minority`, watermarks, disk_free_limit relative to the PVC size), plus the Prometheus plugin and the metrics to alarm on.

Output as: (a) a recommended RabbitmqCluster CR with the key fields and rationale inline, (b) the StorageClass and PDB, (c) the upgrade runbook step order, and (d) the alerts and watermarks to set. Flag every place my described constraints force a trade-off.

Prefer quorum queues and `pause_minority` partition handling. Do not recommend a 2-replica cluster or classic mirrored queues for new deployments, and never suggest clearing PVCs as a recovery step without an export-first caveat.