Skip to Main Content

Mutal TLS (mTLS) Authentication

Refer to Overview▸, the AutoMQ data plane service is based on the Kafka API and provides mTLS identity authentication features. This document introduces how to configure mTLS identity authentication for an instance, along with examples of client access.

Enable MTLS:​

The mTLS protocol verifies clients by issuing a unique TLS certificate to each one. During communication, it authenticates the legitimacy of the certificate and conducts ACL access control checks on verified identities.

In the AutoMQ BYOC environment, enabling TLS transport encryption requires users to provide a CA and server certificate, ensuring that the certificates align with the domain name. Users must also regularly update certificates to prevent service disruptions due to expired certificates.

Step 1: Create a CA and Certificate​

If your organization cannot obtain certificates from a trusted CA, you can follow the document below to issue and maintain a private CA and certificates on your own. Additionally, it is recommended to use AWS Private CA for managing CA and certificates in a production environment.

  1. Configure the CA signing policy file ca.cnf.

[ ca ]
default_ca = CA_AutoMQ_Default

[ CA_AutoMQ_Default ]
default_days    = 3650
database        = index.txt
serial          = serial.txt
default_md      = sha256
copy_extensions = copy
unique_subject  = no
policy          = signing_policy

[ req ]
prompt             = no
distinguished_name = distinguished_name
x509_extensions    = extensions

[ distinguished_name ]
organizationName = <Replace with your Orgnaization Name>
commonName       = <Replace with your Common Name>

[ extensions ]
keyUsage         = critical,digitalSignature,nonRepudiation,keyEncipherment,keyCertSign
basicConstraints = critical,CA:true,pathlen:1

[ signing_policy ]
organizationName = supplied
commonName       = optional

  1. Generate the CA private key ca.key and set file access permissions.

openssl genrsa -out ca.key 2048
chmod 400 ca.key

  1. Generate the CA public certificate ca.crt.

openssl req -new -x509 -config ca.cnf -key ca.key -days 3650 -batch -out ca.crt

The commands above will generate the ca.crt file as the CA public certificate, which will need to be deployed to AutoMQ instances and Kafka clients subsequently.

  1. Create a Broker Certificate Configuration File broker.cnf .

In the AutoMQ BYOC environment, the Broker directly serves clients using IP addresses; therefore, domain name validation is currently not supported. It is advisable to keep the default value for the SAN information, and clients can subsequently disable domain name validation if needed.


[ req ]
prompt             = no
distinguished_name = distinguished_name
req_extensions     = extensions

[ distinguished_name ]
organizationName = <Replace with your Organization>

[ extensions ]
subjectAltName = @alt_names

[ alt_names ]
DNS.1 = localhost

  1. Generate the Broker Private Key broker.key .

openssl genrsa -out broker.key 2048

  1. Generate the Broker Certificate Signing Request CSR broker.csr .

openssl req -new -key broker.key -out broker.csr -nodes -config broker.cnf

  1. Sign the CSR using the previously generated CA private key to create the Broker Certificate broker.crt .

openssl x509 -req -CA ca.crt -CAkey ca.key -in broker.csr -out broker.crt -days 365 -CAcreateserial

The output file broker.crt is the signed Broker certificate. It's recommended to set it with read-only permissions. You will need the broker.crt, broker.key, and ca.crt files when creating the AutoMQ instance later.

Step 2: Create a Client Certificate​

  1. Refer to Manage Kafka ACLs▸** and navigate to the AutoMQ console to create an ACL user for the client, authorizing the necessary read and write permissions for Topic and Group, and record the ACL username**.

  2. Create a Client certificate configuration file client.cnf, and set the CN information to the ACL username.

Note:

When accessing an AutoMQ instance using the mTLS protocol, the server will, by default, map the client certificate's identity to the ACL user. The default mapping rule is set as RULE:.CN=([^,]+)./$1/.

For example, with client certificate identity information as: CN=user01,O=xxxx,OU=xxx,L=xxx,S=xxx,C=xxx

AutoMQ extracts the CN field and constructs Principal=User:user01.


[ req ]
prompt             = no
distinguished_name = distinguished_name
req_extensions     = extensions

[ distinguished_name ]
organizationName = <Replace with your Organization>
CN               = <Replace with your ACL User Name>

[ extensions ]
subjectAltName = @alt_names

[ alt_names ]
DNS.1 = localhost

  1. Generate Client private key client.key.

openssl genrsa -out client.key 2048

  1. Generate Client Certificate Signing Request (CSR) client.csr.

openssl req -new -key client.key -out client.csr -nodes -config client.cnf

  1. Use the previous CA private key to sign the CSR and generate the client certificate client.crt.

openssl x509 -req -CA ca.crt -CAkey ca.key -in client.csr -out client.crt -days 365 -CAcreateserial

The output file client.crt is the signed client certificate. The client application will need both client.crt and client.key files during connection.

Step 3: Server Configuration​

  1. For mTLS, click on Advanced Options >> Enable the following parameters when creating the instance. Refer to the interface diagram below:

    • TransitEncryption: Enable TLS Encryption.

    • Upload CA: Upload the CA certificate file obtained in the previous step ca.crt.

    • Upload Server Cert: Upload the Broker certificate file obtained in the previous step broker.crt.

    • Upload Private Key: Upload the key file for the Broker certificate obtained in the previous step broker.key.

  1. After enabling mTLS, navigate to the instance details page to view the corresponding connection point information.

Note: The SASL_SSL and mTLS protocols are only supported at the time of instance creation and do not currently support enabling transport encryption for existing instances.

Step 4: Client Configuration​

Kafka clients use the mTLS protocol to connect to the server. Refer to the Apache Kafka documentation for examples and configure the appropriate parameters.

  1. Convert the previously generated CA certificate into a JKS trustStore. Be sure to replace the parameters in the command below based on your actual situation.

    1. alias: Enter the alias of the CA certificate.

    2. file: Enter the CA certificate file obtained in the previous step.

    3. keystore: Enter the name of the JKS, which will need to be configured in the Kafka client later.

    4. storepass: Enter the access password for the JKS, which will need to be configured in the Kafka client later.


keytool -importcert -alias automq-ca -file ca.crt -keystore truststore.jks -storepass changeit

  1. Convert the previously generated client certificate into a PKCS12 keystore (including the client key). Be sure to replace the parameters in the command below according to your actual situation.

    1. in: Enter the client certificate file, client.crt, obtained in the previous step.

    2. inkey: Enter the client private key, client.key, obtained in the previous step.

    3. CAfile: Enter the CA certificate, which is the ca.crt obtained from the previous step.

    4. out: The name of the output keystore file.

    5. name: The alias for the output keystore.

    6. password: Set the password.


openssl pkcs12 -export -in client.crt -inkey client.key -chain -CAfile ca.crt -out client.p12 -name automq-client -password pass:changeit

  1. Add the following configuration information to the Kafka client configuration file.

# Configure SSL
security.protocol=SSL
# Set Trust Store Type to JKS
ssl.truststore.type=JKS
# Set the Location of Jks
ssl.truststore.location=/path/to/truststore.jks
# Set the Password of Jks
ssl.truststore.password=changeit
# Disable Ssl Hostname Validation，set the Algorithm to Empty
ssl.endpoint.identification.algorithm=


# Set the Client Keystore Type to PKCS12
ssl.keystore.type=PKCS12
# Set the Client Keystore Jks
ssl.keystore.location=/path/to/client.p12
# Set the Client Keystore Password
ssl.keystore.password=changeit
# Set the Client Key Password
ssl.key.password=changeit

Note:

When signing a private CA and certificates, it's not feasible to ensure they always match the IP of the AutoMQ cluster. As a result, the client's SSL configuration requires disabling hostname verification, which means the ssl.endpoint.identification.algorithm parameter should be set to an empty string.

Certificate Expiry Monitoring​

In a BYOC environment, TLS certificates are supplied by the users, so it's essential for users to monitor the certificates' validity periods to ensure renewal before expiration. The AutoMQ server provides the following metrics to monitor the server certificates' expiration time:

  • kafka_stream_cert_expiry_timestamp_milliseconds: Displays the expiration timestamp of the current certificate in milliseconds.

  • kafka_stream_cert_days_remaining: This metric calculates the number of days left until a certificate expires from the current moment.

It is recommended that customers refer to Monitoring & Alert via Prometheus▸ for configuration methods and utilize tools such as Prometheus and CloudWatch to monitor certificate expiration times and mitigate risks.