Skip to Main Content

Biweekly #6: Simplification of Self-Balancing Strategies and Observation of Stream Metadata

Community contributor list

New community contributors this week: @Dysprosium0626

https://github.com/AutoMQ/automq-for-kafka/pull/701

This PR fixes the ImageDowngradeTest unit test.

AutoMQ Dynamic Trunk Updates

AutoBalancing Simplified Default Strategy

https://github.com/AutoMQ/automq-for-kafka/pull/712

AutoBalancing uses an absolute value balance strategy by default, simplifying configuration without the need to set capacities. AutoBalancing ensures traffic balance across cluster nodes based on traffic sent and consumed.

Kafka-metadata-shell now supports observing S3Stream metadata

https://github.com/AutoMQ/automq-for-kafka/pull/716

You can now monitor S3Stream metadata with kafka-metadata-shell.sh!

Scale-in node data cleanup

https://github.com/AutoMQ/automq-for-kafka/pull/703

Normally, the StreamSetObject generated by a node gradually splits into StreamObject, with the expiration and deletion of StreamObject triggered by the stream's trim. However, after a node is scaled in, its StreamSetObject can no longer split into StreamObject and cannot be expired and deleted. Therefore, a regular scanning logic is added to the control plane to clear expired StreamSetObject.

EBS disaster recovery support

https://github.com/AutoMQ/automq-for-rocketmq/pull/899

https://github.com/AutoMQ/automq-for-kafka/pull/693

Added an EBS disaster recovery atomic interface. When an EC2 instance is mistakenly downsized or becomes unavailable, the EBS from that EC2 can be mounted to another surviving node, and the data can be uploaded to S3 through the EBS disaster recovery interface to complete the EBS disaster recovery process.