Manage Kafka ACLs

This article introduces the basic concepts of Kafka ACL functionality, relevant constraints, and how to enable and use ACL functionality in AutoMQ instances.

Kafka ACL Concepts

Overview

Apache Kafka provides ACL (Access Control List) functionality, supporting authentication and authorization for Kafka clients. AutoMQ is fully compatible with Apache Kafka, so when using an AutoMQ instance, the Kafka ACL functionality can be enabled and utilized.

When using Kafka ACL, please be aware of the following constraints:

Version Constraint: Ensure that the AutoMQ instance version is >= 1.10.x. If not, please upgrade the version before enabling it.

Optional Enablement: By default, AutoMQ instances do not have ACL enabled. Users can manually enable ACL during instance creation or afterward.

Access Protocols: AutoMQ currently supports accessing the cluster through the VPC internal network using SASL_PLAINTEXT, SASL_SSL, and mTLS protocols.

Resource Types: AutoMQ currently supports ACL permission control for resource types such as Kafka Cluster, Topic, Consumer Group, and Transaction ID, but not for DelegationToken. If needed, please refer to Obtaining Services▸ to contact us.

ACL User

In the Kafka ACL feature, the user is the subject for ACL identity recognition. The Kafka client transmits its user identity through the access protocol, and the Kafka server confirms the identity of the user corresponding to the current client request before proceeding with subsequent permission verification and other operations.

Note the difference between a Kafka ACL user and an AutoMQ environment member:

Scenario Comparison	Kafka ACL User	AutoMQ Environment Member
Usage Scenario	Access via Kafka client API and CLI	Access via AutoMQ environment web
Scope of Coverage	Acts on a single Kafka instance	Acts on the entire environment, allowing cross-instance operations

Note:

When accessing AutoMQ instances via the mTLS protocol, the server will by default map the client's certificate identity to an ACL user. The default mapping rule is set as RULE:.CN=([^,]+)./$1/.

Example: For a client certificate identity of: CN=user01,O=xxxx,OU=xxx,L=xxx,S=xxx,C=xxx

AutoMQ extracts the CN field and constructs the Principal= User:user01.

ACL Authorization Policy

Apache Kafka specifies the range of resources that different users can access and the corresponding list of operations that can be executed through authorization policies. Refer to Apache Kafka's design documentation. The authorization policy for an AutoMQ instance includes the following information:

Parameter	Description
Resource Type	Description: Resource type refers to the categories of Kafka resources applicable to Kafka ACL policies, currently supporting the following types: Cluster: Defines operation permissions at the instance (cluster) level, such as viewing cluster configurations and cloud information. Topic: Defines operation permissions at the Topic level, such as deleting, subscribing, sending messages, etc. Consumer Group: Defines the permissions for Kafka Consumer clients to join Groups and participate in receiving messages. TransactionID: Defines the usage permissions for transactional messages.
Resource Name	Description: Resource Name defines the scope of resources covered by the ACL policy. It can be an exact name or a prefix defined through custom string conventions. Restrictions: Refer to Restrictions▸ .
Match Type	Description: Match Type describes whether the Resource Name field in the current ACL rule is an exact match or a prefix match. Types: literal: Exact match, meaning the resource name is the actual effective name. prefixed: Prefix match, meaning the resource name is the prefix of the actual effective resource.
Operation Group	Description: An Operation Group is a bundled combination of fine-grained atomic authorization operations (Operations) in Apache Kafka by AutoMQ. Each operation group contains multiple Kafka operations. Operation groups simplify the authorization process for a series of typical use cases. For the mapping relationship between operation groups and Kafka ACL operations, refer to the table below.
Authorization Types	Description: Apache Kafka ACL supports the following two authorization types: ALLOW: This ACL policy permits users to perform operations on the associated resources. DENY: This ACL policy prevents users from performing operations on the associated resources. Note: A DENY type authorization policy functions like a blacklist capability, is exclusive, and takes precedence over ALLOW.
Access Source	Description: Access Source defines when the current ACL policy is effective only for the current IP access source. It is not effective for an IP access source that is not matched. Limitation: Access Source supports only precise IP addresses and does not support CIDR networks or domain names.

AutoMQ defines operation groups to simplify authorization processes, and the mapping between operation groups and Kafka operations is displayed in the table below:

Resource Type	Operation Group	Included Operations	Recommended Scenarios
Cluster	All	Alter AlterConfigs ClusterAction Create Describe DescribeConfigs All	The All operation group grants full permissions for all actions on the Kafka cluster (instance). It is generally advised to assign this to user identities utilizing Kafka UI or CLI tools.
Topic	All	Alter AlterConfigs Create Delete Describe DescribeConfigs Read Write	The All operation group provides full permissions for all actions on a single Topic. It is typically suggested to assign this to user identities requiring both production and subscription permissions.
	Produce	Create Write Describe	The Produce operation group defines the minimum permissions required for sending messages to a single Topic. It is advised to grant this authority to the user identity responsible for message production.
	Consume	Read Describe	The Consume operation group defines the minimum permissions required for subscribing to messages from a single Topic. It is advised to grant this authority to the user identity responsible for message subscription. If you intend to use a Consumer Group to subscribe to messages, besides granting permissions for the Consume operation group on the Topic, you must also authorize the target Consumer Group.
Consumer Group	All	Delete Describe Read	The All operation group grants complete operational permissions for a specific Consumer Group. It is advised to grant this authority to the user identity responsible for message subscription. To utilize a Consumer Group for subscribing to messages, besides authorizing the Consumer Group, you must also grant permissions for the Consume operation group on the target Topic.
TransactionID	All	Describe Write	All operation groups agreed on granting permission to read state and commit transactions for a specific TransactionID. It is recommended to authorize this for user identities that need to send transactional messages.

Using Kafka ACL

To use the Kafka ACL feature in an AutoMQ instance, follow the process outlined below:

It is important to note that:

• When creating an instance, ACL is not enabled by default. At this time, clients can access the cluster anonymously via the PLAINTEXT protocol and have permissions for all operations.

• After enabling ACL, the PLAINTEXT protocol will continue to provide services. However, all client applications should promptly switch to SASL_PLAINTEXT endpoints that support authentication and authorization to prevent unauthorized access. Once the switch is complete, close the PLAINTEXT protocol endpoints.