🗝️ Syncing Secrets Into Kubernetes With the External Secrets Operator

The fastest way to leak credentials is to put a Kubernetes Secret in Git. They’re base64, not encrypted, and base64 is not a security boundary — it’s an encoding. I’ve watched teams discover this the hard way during a routine audit. The External Secrets Operator (ESO) is the clean answer: keep the real secrets in a real secret store, and let the operator project them into the cluster as native Secrets.

Here’s how I set it up and the operational details that actually matter.

The model

ESO is a controller plus two main custom resources:

• A SecretStore (or ClusterSecretStore) describes how to reach a backend — AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, HashiCorp Vault, and many more.
• An ExternalSecret describes what to fetch and where to put it. The operator reads from the store on an interval and writes a normal Kubernetes Secret.

Your pods consume a perfectly ordinary Secret. They have no idea ESO exists. That decoupling is the whole point: your manifests in Git reference secret names, never secret values.

Installing the operator

helm repo add external-secrets https://charts.external-secrets.io
helm install external-secrets external-secrets/external-secrets \
  --namespace external-secrets --create-namespace

Confirm it’s up before defining stores:

kubectl get pods -n external-secrets

Defining a SecretStore

Backends differ, but the shape is consistent. For AWS Secrets Manager using IRSA (the cleanest auth path — no static keys anywhere):

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: aws-secrets
  namespace: payments
spec:
  provider:
    aws:
      service: SecretsManager
      region: us-east-1
      auth:
        jwt:
          serviceAccountRef:
            name: payments-eso

Pair that ServiceAccount with an IAM role that grants secretsmanager:GetSecretValue on a tightly scoped resource ARN. Scope it to a path prefix, not *. The most common mistake I see is a SecretStore whose IAM policy can read every secret in the account — that turns one compromised namespace into a full credential breach.

Fetching a secret

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
  namespace: payments
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets
    kind: SecretStore
  target:
    name: db-credentials
    creationPolicy: Owner
  data:
  - secretKey: password
    remoteRef:
      key: prod/payments/db
      property: password
  - secretKey: username
    remoteRef:
      key: prod/payments/db
      property: username

This produces a Secret named db-credentials with username and password keys, refreshed hourly. Verify:

kubectl get externalsecret db-credentials -n payments
kubectl describe externalsecret db-credentials -n payments

The STATUS column should read SecretSynced. If it doesn’t, the describe output names the exact failure — usually a permissions error or a key path typo.

Pulling everything from one store entry

If you keep a whole JSON blob in one secret, dataFrom extracts all keys at once instead of listing each:

  dataFrom:
  - extract:
      key: prod/payments/db

I lean on this for app configs that bundle a dozen values. One source of truth in the store, one Secret out, no per-key boilerplate.

The refresh-interval tradeoff

refreshInterval controls how often ESO polls the backend. Tighter intervals mean faster propagation when you rotate a credential, but more API calls (and AWS Secrets Manager charges per 10,000 calls). I use one hour for most things and tighten to a few minutes only for credentials that rotate frequently.

Important caveat: updating the Secret does not restart your pods. Most apps read env vars or mounted files once at startup. If you rotate a secret, you still need to roll the deployment — or pair ESO with a tool like Reloader that watches Secrets and triggers a rollout. Don’t assume rotation is end-to-end automatic; it gets the value into the cluster, not into the running process.

Generators for dynamic credentials

ESO can also generate values — for example, short-lived database credentials via Vault’s database engine, or an ECR auth token that’s only valid for 12 hours. This is where it goes from “secret syncer” to genuine secret-rotation infrastructure, because the credential a pod uses never lives long enough to be worth stealing.

What this buys you operationally

• Git stays clean. Your repo references db-credentials; the value lives in the store. A leaked repo leaks nothing.
• One source of truth. Rotate in the backend, every consuming cluster picks it up.
• Audit trail. Access logging lives in your secret store, where security can actually see it.

Failure modes to watch

• ClusterSecretStore sprawl. A cluster-wide store with broad read access is convenient and dangerous. Prefer namespaced SecretStores scoped to least privilege.
• Silent sync failures. Alert on ExternalSecrets where status isn’t SecretSynced. A stale secret fails quietly until something tries to use it.
• Deleting the ExternalSecret deletes the Secret when creationPolicy: Owner. Know that before you kubectl delete during a cleanup.

Before you ship the IAM policy and SecretStore, get them reviewed — an over-scoped policy is easy to write and hard to spot. Our AI code review flags wildcard resource ARNs and missing least-privilege scoping.

ESO is one of those tools that quietly removes an entire class of incident. For more on running clusters safely, browse the Kubernetes & Helm category.

Secret-management configuration is high-stakes. Verify IAM scoping and store access against your own security requirements before applying to production.