Skip to main content
Refer to Overview▸, the AutoMQ data plane service is based on the Kafka API and offers mTLS authentication capabilities. This document outlines how to configure mTLS authentication for instances and provides examples of client access.

​
Enable MTLS:

The mTLS protocol verifies clients by issuing a unique TLS certificate to each one. During communication, it authenticates the legitimacy of the certificate and conducts ACL access control checks on verified identities.
In the AutoMQ BYOC environment, enabling TLS transport encryption requires users to provide a CA and server certificate, ensuring that the certificates align with the domain name. Users must also regularly update certificates to prevent service disruptions due to expired certificates.

​
Step 1: Create a CA and Certificate

If your organization cannot obtain certificates from a trusted CA, you can follow the document below to issue and maintain a private CA and certificates on your own. Additionally, it is recommended to use AWS Private CA for managing CA and certificates in a production environment.
  1. Configure the CA signing policy file ca.cnf.
[ ca ]
default_ca = CA_AutoMQ_Default

[ CA_AutoMQ_Default ]
default_days    = 3650
database        = index.txt
serial          = serial.txt
default_md      = sha256
copy_extensions = copy
unique_subject  = no
policy          = signing_policy

[ req ]
prompt             = no
distinguished_name = distinguished_name
x509_extensions    = extensions

[ distinguished_name ]
organizationName = <Replace with your Orgnaization Name>
commonName       = <Replace with your Common Name>

[ extensions ]
keyUsage         = critical,digitalSignature,nonRepudiation,keyEncipherment,keyCertSign
basicConstraints = critical,CA:true,pathlen:1

[ signing_policy ]
organizationName = supplied
commonName       = optional
  1. Generate the CA private key ca.key and set file access permissions.
openssl genrsa -out ca.key 2048
chmod 400 ca.key
  1. Generate the CA public certificate ca.crt.

openssl req -new -x509 -config ca.cnf -key ca.key -days 3650 -batch -out ca.crt
The commands above will generate the ca.crt file as the CA public certificate, which will need to be deployed to AutoMQ instances and Kafka clients subsequently.
  1. Create a Broker Certificate Configuration File broker.cnf .
In the AutoMQ BYOC environment, the Broker directly serves clients using IP addresses; therefore, domain name validation is currently not supported. It is advisable to keep the default value for the SAN information, and clients can subsequently disable domain name validation if needed.
[ req ]
prompt             = no
distinguished_name = distinguished_name
req_extensions     = extensions

[ distinguished_name ]
organizationName = <Replace with your Organization>
commonName       = <Replace with your Common Name>
[ extensions ]
subjectAltName = @alt_names

[ alt_names ]
DNS.1 = <Replace with your Broker Host Name>
  1. Generate the Broker Private Key broker.key .

openssl genrsa -out broker.key 2048
  1. Generate the Broker Certificate Signing Request CSR broker.csr .

openssl req -new -key broker.key -out broker.csr -nodes -config broker.cnf
  1. Sign the CSR using the previously generated CA private key to create the Broker Certificate broker.crt .

openssl x509 -req -CA ca.crt -CAkey ca.key -in broker.csr -out broker.crt -days 365 -CAcreateserial -extensions extensions  -extfile broker.cnf
The output file broker.crt is the signed Broker certificate. It’s recommended to set it with read-only permissions. You will need the broker.crt, broker.key, and ca.crt files when creating the AutoMQ instance later.

​
Step 2: Create a Client Certificate

  1. Refer to Manage Kafka ACLs▸ , navigate to the AutoMQ console to create ACL users for clients, grant read and write permissions for the necessary Topics and Groups, and take note of the ACL username.
  2. Create a Client certificate configuration file client.cnf, and set the CN information to the ACL username.
Note:When accessing an AutoMQ instance using the mTLS protocol, the server will, by default, map the client certificate’s identity to the ACL user. The default mapping rule is set as RULE:.CN=([^,]+)./$1/.For example, with client certificate identity information as: CN=user01,O=xxxx,OU=xxx,L=xxx,S=xxx,C=xxxAutoMQ extracts the CN field and constructs Principal=User:user01.
[ req ]
prompt             = no
distinguished_name = distinguished_name
req_extensions     = extensions

[ distinguished_name ]
organizationName = <Replace with your Organization>
CN               = <Replace with your ACL User Name>

[ extensions ]
subjectAltName = @alt_names

[ alt_names ]
DNS.1 = localhost
  1. Generate Client private key client.key.

openssl genrsa -out client.key 2048
  1. Generate Client Certificate Signing Request (CSR) client.csr.

openssl req -new -key client.key -out client.csr -nodes -config client.cnf
  1. Use the previous CA private key to sign the CSR and generate the client certificate client.crt.

openssl x509 -req -CA ca.crt -CAkey ca.key -in client.csr -out client.crt -days 365 -CAcreateserial -extensions extensions  -extfile client.cnf
The output file client.crt is the signed client certificate. The client application will need both client.crt and client.key files during connection.

​
Step 3: Server Configuration

  1. For mTLS, click on Advanced Options >> Enable the following parameters when creating the instance. Refer to the interface diagram below:
    • TransitEncryption: Enable TLS Encryption.
    • Upload CA: Upload the CA certificate file obtained in the previous step ca.crt.
    • Upload Server Cert: Upload the Broker certificate file obtained in the previous step broker.crt.
    • Upload Private Key: Upload the key file for the Broker certificate obtained in the previous step broker.key.
  1. After enabling mTLS, navigate to the instance details page to view the corresponding connection point information.
Note: The SASL_SSL and mTLS protocols are only supported at the time of instance creation and do not currently support enabling transport encryption for existing instances.

​
Step 4: Client Configuration

Kafka clients use the mTLS protocol to connect to the server. Refer to the Apache Kafka documentation for examples and configure the appropriate parameters.
  1. Convert the previously generated CA certificate into a JKS trustStore. Be sure to replace the parameters in the command below based on your actual situation.
    1. alias: Enter the alias of the CA certificate.
    2. file: Enter the CA certificate file obtained in the previous step.
    3. keystore: Enter the name of the JKS, which will need to be configured in the Kafka client later.
    4. storepass: Enter the access password for the JKS, which will need to be configured in the Kafka client later. 

keytool -importcert -alias automq-ca -file ca.crt -keystore truststore.jks -storepass changeit
  1. Convert the previously generated client certificate into a PKCS12 keystore (including the client key). Be sure to replace the parameters in the command below according to your actual situation.
    1. in: Enter the client certificate file, client.crt, obtained in the previous step.
    2. inkey: Enter the client private key, client.key, obtained in the previous step.
    3. CAfile: Enter the CA certificate, which is the ca.crt obtained from the previous step.
    4. out: The name of the output keystore file.
    5. name: The alias for the output keystore.
    6. password: Set the password. 
openssl pkcs12 -export -in client.crt -inkey client.key -chain -CAfile ca.crt -out client.p12 -name automq-client -password pass:changeit
  1. Add the following configuration information to the Kafka client configuration file.
# Configure SSL
security.protocol=SSL
# Set Trust Store Type to JKS
ssl.truststore.type=JKS
# Set the Location of Jks
ssl.truststore.location=/path/to/truststore.jks
# Set the Password of Jks
ssl.truststore.password=changeit
# Disable Ssl Hostname Validation, Set the Algorithm to Empty
ssl.endpoint.identification.algorithm=


# Set the Client Keystore Type to PKCS12
ssl.keystore.type=PKCS12
# Set the Client Keystore Jks
ssl.keystore.location=/path/to/client.p12
# Set the Client Keystore Password
ssl.keystore.password=changeit
# Set the Client Key Password
ssl.key.password=changeit
Note:When signing a private CA and certificates, it’s not feasible to ensure they always match the IP of the AutoMQ cluster. As a result, the client’s SSL configuration requires disabling hostname verification, which means the ssl.endpoint.identification.algorithm parameter should be set to an empty string.

​
Certificate Expiry Monitoring

In a BYOC environment, TLS certificates are supplied by the users, so it’s essential for users to monitor the certificates’ validity periods to ensure renewal before expiration. The AutoMQ server provides the following metrics to monitor the server certificates’ expiration time:
  • kafka_stream_cert_expiry_timestamp_milliseconds: Displays the expiration timestamp of the current certificate in milliseconds.
  • kafka_stream_cert_days_remaining: This metric calculates the number of days left until a certificate expires from the current moment.
It is recommended that customers refer to the configuration methods in Monitoring & Alert via Prometheus▸ and utilize tools like Prometheus and CloudWatch to monitor certificate expiration times to mitigate risks.