Skip to main content
Kafka Connect plugins are a collection of components used for streaming data between Kafka and other systems. They typically encapsulate connectors, transforms, and converters for specific data processing scenarios. The AutoMQ managed Kafka Connect service comes with several system plugins that have been officially tested and verified. At the same time, to meet diverse data integration needs, AutoMQ also supports users uploading and managing custom plugins.

​
View Plugin List

Users can view all available system plugins and custom plugins in the current environment through the plugin list. Procedure:
  1. Log in to the AutoMQ console.
  2. In the left navigation bar, click the Kafka Connect menu.
  3. You will be directed to the Plugins list page by default.
On this page, you can see all integrated system plugins, which can be directly used to create Connect tasks. If the system plugins cannot meet your specific needs, you can download plugins from the third-party community or develop and upload your own.

​
Upload Custom Plugin

When you need to use a plugin that is not officially provided, you can upload a custom plugin through the console. Procedure:
  1. On the Plugins list page, click the Upload Custom Plugin button in the upper right corner.
  2. In the pop-up dialog box, upload your prepared plugin package as prompted.
  3. The system will perform a pre-check and format validation on the uploaded plugin package.
  4. After the validation passes, click Confirm to complete the plugin upload.
Plugin Package Constraints:
  • Directory Structure: The plugin package must follow the standard Kafka Connect plugin directory structure of the community and be distributed in .zip format.
  • Size Limit: The uploaded .zip plugin package size must not exceed 200 MiB.

​
Delete Custom Plugin

When you no longer need a specific version of a custom plugin, you can choose to delete it to free up resources and simplify management. Procedure:
  1. On the Plugins list page, find the custom plugin you want to delete.
  2. Click the Delete button on the right side of the corresponding plugin.
  3. Complete the operation in the confirmation dialog box.
Please note the following constraints:
  • System Plugins: Cannot be deleted.
  • In-Use Plugins: If a plugin (or any of its versions) is being used by one or more Connect tasks, it cannot be deleted. You must first stop and delete the relevant tasks.

​
Update Plugin

When the community releases a new version of a plugin, or your own custom plugin has a new version, you can update it by uploading the new version. Procedure: The procedure for updating a plugin is the same as for uploading a custom plugin. The AutoMQ managed Kafka Connect service supports the coexistence of multiple versions of the same plugin. Different versions are independent of each other and do not interfere with each other. You can choose to use a specific version of the plugin when creating a Connect task as needed.