Securing Kafka with TLS, SASL, and ACLs

A default Apache Kafka cluster is wide open. Any client that can reach the broker port can produce, consume, create topics, and read every message on the wire in plaintext. That is fine for a laptop, and dangerous everywhere else. Securing Kafka is a three-layer job: encryption in transit with TLS, authentication with SASL or mutual TLS, and authorization with ACLs. Each layer is independent, each is configured separately, and skipping any one of them leaves a real hole. This guide walks through all three on Kafka 3.x, with the keystore commands, server.properties settings, JAAS files, and kafka-acls invocations you actually need.

The three layers and why order matters

It helps to be precise about what each layer does, because people routinely conflate them.

Encryption (TLS): Protects data in transit so an attacker on the network cannot read messages or steal credentials. This is purely confidentiality and integrity. It says nothing about who the client is.
Authentication (SASL or mTLS): Establishes a verified identity — a principal — for each connection. Without authentication, ACLs are meaningless because the broker has no idea who is connecting.
Authorization (ACLs): Decides what an authenticated principal is allowed to do: which topics it can read, write, or administer.

The dependency runs one direction. Authorization needs authentication to know who the principal is. Authentication over an unencrypted channel leaks credentials. So in practice you build from the bottom up: TLS first, then SASL, then ACLs. Turning on ACLs without authentication just locks everyone out or, worse, defaults to allowing the anonymous User:ANONYMOUS principal.

Setting up TLS encryption

Kafka uses standard Java keystores. Each broker needs a keystore holding its own certificate and private key, and a truststore holding the CA certificate it trusts. Start by creating a CA, then sign a certificate for each broker.

# Create a CA key and self-signed cert (use a real CA in production)
openssl req -new -x509 -keyout ca-key -out ca-cert -days 3650 \
  -subj "/CN=kafka-ca" -nodes

# Generate the broker keystore with a keypair
keytool -keystore kafka.broker0.keystore.jks -alias broker0 \
  -validity 3650 -genkey -keyalg RSA \
  -dname "CN=broker0.kafka.internal" \
  -storepass changeit -keypass changeit

# Create a CSR, sign it with the CA, then import the chain
keytool -keystore kafka.broker0.keystore.jks -alias broker0 \
  -certreq -file broker0.csr -storepass changeit
openssl x509 -req -CA ca-cert -CAkey ca-key -in broker0.csr \
  -out broker0-signed.crt -days 3650 -CAcreateserial
keytool -keystore kafka.broker0.keystore.jks -alias CARoot \
  -import -file ca-cert -storepass changeit -noprompt
keytool -keystore kafka.broker0.keystore.jks -alias broker0 \
  -import -file broker0-signed.crt -storepass changeit -noprompt

# Truststore just needs the CA cert
keytool -keystore kafka.broker0.truststore.jks -alias CARoot \
  -import -file ca-cert -storepass changeit -noprompt

Make sure the certificate CN or a SAN entry matches the hostname clients use, because Kafka 3.x enables hostname verification by default. A mismatch produces a confusing No subject alternative names handshake failure.

Now wire it into server.properties. The key concept is listeners: a broker can expose multiple named listeners with different security protocols.

listeners=INTERNAL://:9092,SSL://:9093
advertised.listeners=INTERNAL://broker0.kafka.internal:9092,SSL://broker0.kafka.internal:9093
listener.security.protocol.map=INTERNAL:PLAINTEXT,SSL:SSL
inter.broker.listener.name=INTERNAL

ssl.keystore.location=/etc/kafka/secrets/kafka.broker0.keystore.jks
ssl.keystore.password=changeit
ssl.key.password=changeit
ssl.truststore.location=/etc/kafka/secrets/kafka.broker0.truststore.jks
ssl.truststore.password=changeit

For a production cluster you would not leave an internal PLAINTEXT listener exposed; it is shown here only to illustrate that listeners are independent. Encrypt broker-to-broker traffic too by pointing inter.broker.listener.name at an SSL listener once you trust the setup.

Pro Tip: Verify the TLS handshake from a client host before touching authentication. Run openssl s_client -connect broker0.kafka.internal:9093 -CAfile ca-cert. If you see the broker certificate chain and Verify return code: 0 (ok), TLS is working and you have isolated it from later SASL problems.

Choosing and configuring a SASL mechanism

SASL gives connections an identity. Kafka supports several mechanisms, and the right choice depends on your environment.

Mechanism	Credentials	Best for	Caution
`SASL/PLAIN`	Username + password	Quick setups behind TLS	Passwords are plaintext in config; never use without TLS
`SASL/SCRAM-SHA-256/512`	Salted hashed creds in metadata	Most self-managed clusters	Manage users with `kafka-configs`
`SASL/GSSAPI`	Kerberos tickets	Enterprises with existing Kerberos	Operationally heavy
`SASL/OAUTHBEARER`	OAuth2 tokens	Cloud-native, federated identity	Needs a token provider

For most teams running their own cluster, SCRAM is the sweet spot. Credentials are stored salted and hashed in cluster metadata rather than sitting in a flat config file, and you can add or revoke users without restarting brokers.

Create SCRAM users with kafka-configs:

kafka-configs --bootstrap-server broker0.kafka.internal:9093 \
  --command-config admin.properties \
  --alter --add-config 'SCRAM-SHA-512=[password=appsecret]' \
  --entity-type users --entity-name app-producer

Add a SASL_SSL listener to the broker — SASL for authentication, SSL for encryption underneath:

listeners=SASL_SSL://:9094
advertised.listeners=SASL_SSL://broker0.kafka.internal:9094
listener.security.protocol.map=SASL_SSL:SASL_SSL
sasl.enabled.mechanisms=SCRAM-SHA-512
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-512

Brokers authenticate to each other too, so supply broker credentials via a JAAS config. With SCRAM you can embed it inline:

listener.name.sasl_ssl.scram-sha-512.sasl.jaas.config=\
  org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="kafka-broker" password="brokersecret";

A producer or consumer then connects with a client properties file:

bootstrap.servers=broker0.kafka.internal:9094
security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
ssl.truststore.location=/etc/kafka/secrets/client.truststore.jks
ssl.truststore.password=changeit
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule \
  required username="app-producer" password="appsecret";

Pro Tip: If you are running KRaft mode (no ZooKeeper, the default for new Kafka 3.x clusters), SCRAM credentials live in the metadata log. You must bootstrap the inter-broker SCRAM user at cluster format time with kafka-storage format --add-scram, otherwise brokers cannot authenticate to each other on first boot.

When to use mTLS instead of SASL

Mutual TLS is an authentication mechanism in its own right. If you set ssl.client.auth=required on an SSL listener, each client must present a certificate, and Kafka derives the principal from the certificate’s distinguished name (for example User:CN=app-producer,OU=apps). This avoids passwords entirely and is attractive when you already run a certificate authority and service mesh. The tradeoff is certificate lifecycle management — rotation, revocation, and short-lived certs — which is its own discipline.

Authorization with ACLs

With identities established, turn on the authorizer. In KRaft mode use the standard authorizer:

authorizer.class.name=org.apache.kafka.metadata.authorizer.StandardAuthorizer
allow.everyone.if.no.acl.found=false
super.users=User:kafka-broker

The critical line is allow.everyone.if.no.acl.found=false. With it set to false, Kafka denies any action that has no matching allow rule — a deny-by-default posture. Leaving it true (the default in some setups) means the moment you stop adding ACLs, everything is permitted again. Always set it false in production and declare super.users for your operational accounts so you do not lock yourself out.

ACLs are managed with kafka-acls. Grant the producer write access to a topic:

kafka-acls --bootstrap-server broker0.kafka.internal:9094 \
  --command-config admin.properties \
  --add --allow-principal User:app-producer \
  --operation Write --operation Describe \
  --topic orders

Grant a consumer read access plus the consumer group:

kafka-acls --bootstrap-server broker0.kafka.internal:9094 \
  --command-config admin.properties \
  --add --allow-principal User:app-consumer \
  --operation Read --operation Describe \
  --topic orders --group order-processors

A few rules that trip people up:

Consumers need a group ACL. Read on the topic is not enough; the consumer also needs Read on its consumer group resource. The example above grants both.
Producers using idempotence need extra rights. An idempotent producer (the default in modern clients) requires IdempotentWrite on the cluster, or Write on the topic if you are on Kafka 2.8+ where topic write implies idempotent write.
Use prefixed ACLs to scale. Instead of one ACL per topic, grant on a prefix: --resource-pattern-type prefixed --topic team-a. lets a team own everything under team-a. without per-topic churn.
List before you change. kafka-acls --list --topic orders shows exactly what is allowed, which is invaluable when debugging a TopicAuthorizationException.

Pro Tip: When a client suddenly gets authorization errors, set the authorizer logger to DEBUG temporarily. The broker logs every allow/deny decision with the principal, operation, and resource, which tells you immediately whether the problem is a missing ACL, a wrong principal name, or a typo in the topic.

Putting it together and verifying

Once all three layers are live, verify end to end rather than assuming. A clean validation sequence:

Confirm TLS with openssl s_client against the SASL_SSL port.
Produce a test message with a properly scoped principal and confirm it succeeds.
Produce with a principal that has no ACL and confirm it is denied — a passing negative test is the one people skip.
Consume with the consumer principal and confirm both topic and group access work.
Audit the full ACL set with kafka-acls --list and store it in version control so authorization is reviewable.

That last point matters more than it looks. ACLs drift. Treat them as code: keep the kafka-acls commands in a repository, review changes, and apply them through automation rather than ad hoc terminal sessions. The same discipline that keeps your Kafka monitoring honest keeps your access control honest.

Where teams actually go wrong

Most Kafka security incidents are not exotic. They are one of a handful of predictable mistakes. The internal PLAINTEXT listener that was “temporary” and got advertised to the whole VPC. The allow.everyone.if.no.acl.found=true that nobody flipped back. The SASL/PLAIN setup that someone deployed without TLS underneath, leaking every password to anyone running tcpdump. The consumer that worked in staging because staging had no authorizer, then failed in production with a cryptic group authorization error.

None of these need clever attackers. They need only a missing layer. The value of treating TLS, SASL, and ACLs as three distinct, independently verified steps is that each one has a clear pass/fail test, and you never end up shipping a cluster that is “mostly” secure. Build from encryption up, test the denials as carefully as the allows, and keep your ACLs in source control.

If you want to go further, securing the broker is only half the story — the clients, schemas, and the platform around them all deserve the same scrutiny. Pair this with a broader look at DevOps security automation and you have a defensible Kafka deployment rather than a hopeful one.

— James