Table of Contents
Table of Contents
A shared Amazon MSK cluster can serve several teams, products, or customers, but a shared broker is not a tenant boundary. A topic prefix can keep names tidy while one misconfigured role still reads another tenant’s data. A client quota can slow a producer while its consumer group, network path, and chargeback record remain unattributed.
Tenant isolation on MSK works when several controls agree on the same tenant identity. The practical sequence is identity → namespace → authorization → quotas → network → evidence. ACLs and IAM policies decide who may act. Topic and group naming makes that decision expressible. Apache Kafka® client quotas limit the damage from a noisy tenant. VPC controls reduce the reachable surface, and audit data proves that the model is still true after teams and workloads change.
1Start with the tenant model
Before writing an ACL, decide what “tenant” means in this cluster. It may be a customer, a business unit, an application team, or an environment such as payments-prod. Those choices lead to different failure modes. A customer-facing platform usually needs a customer boundary that survives application replacement. An internal platform may prefer an owning team and environment as the stable unit.
Write the model as a tuple rather than a label:
Tenant = owner + environment + data class + allowed workloads
For example, acme-prod may own topics beginning with acme.prod., consumer groups beginning with acme.prod., and a set of producer and consumer roles. The data class says whether the tenant can publish only operational events, or also read sensitive topics. The workload list identifies the actual clients that may use the namespace.
A good model answers five questions before deployment:
- Which identity represents the tenant when a client connects?
- Which topic and consumer-group names belong to it?
- Can one tenant create or delete resources, or is that platform-only?
- What happens when a tenant exceeds its throughput or partition budget?
- Which logs and usage records can be joined back to the tenant?
Keep tenant identity separate from deployment identity. A CI role that creates topics is not automatically a tenant producer. If one role is shared by all workloads, Kafka cannot enforce per-tenant authorization without another mediation layer. That is an identity design problem, not an ACL syntax problem.
2Map identities before permissions
Amazon MSK IAM access control combines authentication and authorization for IAM identities. AWS documents IAM access control for Java and non-Java Kafka clients, and states that IAM policies control IAM identities. Apache Kafka ACL APIs can still be invoked on an IAM-enabled cluster, but those ACLs have no effect on authorization for IAM identities. That boundary is easy to miss when a team copies a Kafka ACL recipe into an IAM deployment.
Choose one primary authorization path for each client class:
| Client class | Authentication | Authorization source | Stable tenant key |
|---|---|---|---|
| AWS workload using MSK IAM | IAM role or assumed role | IAM identity policy | Role ARN or mapped tenant role |
| Kafka client using SCRAM | SASL/SCRAM credential | Kafka ACL | SCRAM username |
| Kafka client using TLS client authentication | Client certificate | Kafka ACL principal mapping | Certificate subject or mapped principal |
| Platform automation | IAM role | IAM policy plus platform workflow | Automation role, never a tenant role |
Record this mapping in version control. A role such as tenant-acme-producer is useful only when its trust policy prevents another tenant from assuming it. For EKS, test the service-account trust relationship during onboarding. For SCRAM or certificate clients, rotate credentials under the same logical principal.
IAM policy resources are more specific than a broad kafka-cluster:* grant. A tenant producer normally needs to connect to the cluster, describe the cluster and its topics, and write only to its topic namespace. A consumer also needs read access to its topics and its consumer groups. Topic creation, deletion, partition changes, and transactional-id access should remain separate grants because they expand the blast radius of a compromised client.
Use the smallest policy that matches the workload. Then test it with a canary role before onboarding a tenant. A policy that is syntactically valid but cannot describe the cluster produces a connection failure that looks like a network problem; a policy that grants ReadData on a broad topic ARN can pass a happy-path test while leaking neighboring data.
3Make the namespace enforceable
A tenant namespace should cover every resource that carries data or state. That usually means topics, consumer groups, and, when transactions or idempotent producers are used, transactional IDs. Pick a delimiter and reserve it. For example:
tenant-a.prod.orders
tenant-a.prod.payments
tenant-a.prod.analytics
The separator matters less than centralized validation. Do not let a client choose an arbitrary topic name and expect an ACL to repair the mistake later. A topic provisioning service can check the requested tenant, environment, data class, retention policy, and owner before it calls Kafka.
Kafka ACLs support resource patterns, including literal and prefixed patterns. A prefixed rule such as tenant-a.prod. can cover a tenant’s topic namespace, while a separate prefixed rule covers its consumer groups. Keep the operations narrow:
kafka-acls.sh --bootstrap-server "$BOOTSTRAP" \
--command-config client.properties \
--add --allow-principal User:tenant-a \
--operation Read --operation Write \
--topic tenant-a.prod. --resource-pattern-type prefixed
kafka-acls.sh --bootstrap-server "$BOOTSTRAP" \
--command-config client.properties \
--add --allow-principal User:tenant-a \
--operation Read --operation Describe \
--group tenant-a.prod. --resource-pattern-type prefixedTreat those commands as a pattern, not a copy-and-paste production policy. Confirm the Kafka version, listener, principal format, and MSK authentication mode first. If tenants can create topics, give them a constrained creation path and verify the resulting name. If the platform owns topic creation, deny Create and Delete to tenant clients and expose a request workflow instead.
For IAM clients, express the same boundary in IAM policies using tenant-specific topic and group resources. Do not try to use both ACLs and IAM policies as if they were additive filters for the same principal. They are different authorization models, and the model selected for a client determines which policy actually controls access.
4Add quotas for noisy-neighbor control
Authorization answers “may this client access the topic?” It does not answer “how much of the cluster may this tenant consume?” Kafka client quotas fill that gap. Kafka supports quotas associated with users and client IDs, including producer and consumer byte-rate limits and request-rate limits. A client ID convention is therefore part of the isolation design, rather than a dashboard preference.
Set a tenant identity and client ID on every producer and consumer. Then choose limits from an observed workload envelope:
- Measure normal and peak throughput for the tenant.
- Reserve headroom for retries, rebalances, and recovery traffic.
- Set a producer and consumer ceiling separately when the workload is asymmetric.
- Alert before the limit becomes the tenant’s first explanation for elevated latency.
- Revisit the value when partitions, retention, or traffic contracts change.
Kafka quota configuration is typically applied through dynamic broker configuration, for example with producer_byte_rate, consumer_byte_rate, or request_percentage. The exact command and supported quota types depend on the Kafka version and MSK configuration surface; validate them in a staging cluster before changing production. A quota throttles requests. It does not stop a client from trying to access a topic that its authorization policy already permits.
MSK service quotas are a different layer. The AWS quota page lists account, cluster, broker, connection, and other service limits; it also explains how to request an increase. Those limits protect the service and account, but they are not a tenant budget. Track both layers so an account-level limit is not mistaken for a fair-share policy.
A fair-share policy also needs a response when a tenant is throttled. The platform can reject a deployment that asks for more than its contract, page the owner when sustained throttle time exceeds a threshold, or move a high-volume tenant to a dedicated cluster. A quota with no escalation path merely turns contention into a slower outage.
5Reduce the reachable surface with network controls
VPC routing and security groups are useful isolation layers, but they do not replace Kafka authorization. Keep MSK brokers private, restrict client security groups to the required ports and subnets, and separate administrative paths from application paths. If multiple VPCs or accounts connect to the cluster, record which attachment belongs to which tenant or platform zone.
Network separation is strongest when it reflects the tenant model. A regulated workload may need a dedicated VPC or cluster because its boundary is contractual. Several internal tenants may safely share a VPC when their IAM roles, topic namespaces, and quotas are separate. Do not call a tenant isolated merely because its packets arrive through a different subnet; a role with broad topic permissions can still read across that subnet boundary.
Use network logs to answer reachability questions and Kafka audit data to answer authorization questions. The two evidence streams complement each other. AutoMQ’s zero-trust Kafka framework uses the same boundary lens. Flow logs can show that a client reached a broker. They cannot prove that the broker allowed a read of tenant-b.prod.payments.
6Turn usage into audit and chargeback evidence
Isolation decays when ownership data lives in a spreadsheet that no longer matches the cluster. Keep a small registry for every tenant containing its IAM role or Kafka principal, topic and group prefixes, client ID prefix, quota values, owner, data class, and approved network attachment. Changes should go through code review or an equivalent approval path.
For chargeback, start with a model you can explain. Attribute directly measurable usage such as producer bytes, consumer bytes, partition count, retention footprint, connector tasks, and cross-AZ traffic where those records are available. Shared broker hours and support overhead may need an allocation rule; document whether they are split by reserved capacity, measured bytes, or an agreed fixed share. Do not present an allocation as a cloud-provider invoice line.
A monthly tenant report should contain:
- Identity and namespace changes during the period.
- Produce and consume throughput, throttle time, and rejected requests.
- Topic, partition, and retained-byte growth.
- Network path or cross-AZ evidence used in the allocation.
- Open exceptions, owner, expiry date, and next review.
This catches the failure that ACL reviews miss: a tenant whose policy is correct but whose usage, ownership, or network attachment has drifted.
7Decide when a shared cluster has reached its boundary
A shared MSK cluster remains a good fit when tenants have compatible retention and throughput patterns, the platform can issue distinct identities, and the team can test policy changes without slowing every workload. For consolidation trade-offs, see AutoMQ’s Kafka consolidation guide. A dedicated cluster is easier to justify when one tenant needs a different compliance boundary, dominates storage growth, requires independent upgrades, or repeatedly exhausts the shared quota envelope.
There is a second boundary: storage and broker elasticity. If tenant growth is forcing broker-local storage expansion, partition movement, or capacity bought for short-lived peaks, evaluate the Kafka platform as well as the policies. AutoMQ uses a Kafka-compatible interface with a shared-storage architecture, so broker compute and durable object storage can be evaluated as separate scaling concerns. That does not create tenant authorization automatically; the same identity, namespace, quota, and audit rules still need to be designed and tested.
The useful question is not whether a new platform has a “multi-tenant” label. Ask which boundary it changes: storage growth, compute allocation, network placement, or operator ownership. Keep the access-control contract explicit while testing any migration or BYOC deployment. AutoMQ’s architecture overview describes the storage model; a proof of concept should still verify your Kafka clients, ACL or IAM integration, quota behavior, recovery, and chargeback data.
8A review checklist for each tenant
Use this checklist before enabling production traffic:
- The tenant has a stable principal and an owner.
- Topic, consumer-group, and transactional-id prefixes are reserved and validated.
- IAM policy or Kafka ACLs allow required operations and deny cross-tenant access.
- Producer, consumer, and request quotas have a measured rationale.
- Client IDs support usage and throttle attribution.
- Network attachments, security groups, and bootstrap endpoints are recorded.
- A canary proves allowed writes, allowed reads, denied neighboring reads, and expected throttling.
- Audit and usage records can be joined back to the tenant.
- Exceptions have an owner and expiration date.
If the denied neighboring read has never been tested, the isolation boundary is still an assumption. Run that test whenever a role, prefix, listener, or provisioning workflow changes.
9FAQ
9.1Are Kafka ACLs enough to isolate Amazon MSK tenants?
No. ACLs can constrain Kafka principals to topics and consumer groups, but isolation also needs a stable identity, quotas, network controls, provisioning rules, and audit evidence. For IAM-authenticated MSK clients, AWS states that IAM policies control authorization; Kafka ACLs do not authorize IAM identities.
9.2Should each tenant have a separate MSK cluster?
Use a separate cluster when contractual, compliance, upgrade, or capacity boundaries require independent operation. A shared cluster can work when identities, namespaces, quotas, and ownership are distinct and continuously tested. The right boundary depends on the failure and cost you are trying to contain.
9.3What is the difference between an MSK service quota and a Kafka client quota?
An MSK service quota limits an AWS account, cluster, broker, connection, or service feature. A Kafka client quota limits traffic or request use by a user or client ID. Service quotas are platform capacity guardrails; client quotas are tenant fairness controls.
9.4Can IAM policies and Kafka ACLs be combined for one tenant?
They can coexist for different client classes, but do not assume that a Kafka ACL further restricts an IAM identity. Select the authorization model per client, map it to a stable tenant principal, and test the effective permissions with a canary.
9.5When should I evaluate AutoMQ for a multi-tenant MSK design?
Evaluate it when broker-local storage, partition movement, or peak capacity is becoming the limiting boundary, and when a Kafka-compatible platform with shared storage fits your deployment model. Keep tenant identity, namespace, quotas, network, and audit requirements in the migration acceptance test.
When you review the next tenant request, start with the same question as the first one: which identity is allowed to do what, on which namespace, at what rate, and with what evidence? If the answers are explicit, a shared cluster can be governed. If they are not, adding another ACL only hides the gap. For an architecture review of a Kafka-compatible shared-storage deployment, use the AutoMQ deployment assessment.
