Performance Tuning for Broker
AutoMQ enhances server performance through methods like read/write efficiency optimization and resource isolation, ensuring stable and efficient operation.
Users should adjust parameters based on their hardware environment and business scenarios.
Memory-related Parameters
S3.stream.allocator.policy
Configure the memory allocation strategy for AutoMQ by selecting either POOLED_HEAP or POOLED_DIRECT. If memory resources are abundant (> 8 GiB), it is advisable to use POOLED_DIRECT to alleviate GC pressure.
S3.wal.cache.size
Manage the size of the Log Cache, which influences the maximum size of cached data during Tail Read (hot reads).
S3.block.cache.size
Control the size of the Block Cache; it influences the maximum size of data cached during Catch-up Read (catch-up read, cold read). In Catch-up Read scenarios, consider appropriately increasing this value.
S3.wal.upload.threshold
Manage the amount of data AutoMQ accumulates in the Log Cache before triggering an S3 upload. It is advisable not to exceed 1/3 of s3.wal.cache.size.
Recommended Parameter Values
When the memory allocation strategy is POOLED_HEAP, the sum of s3.wal.cache.size and s3.block.cache.size should be approximately 1/4 of MaxHeapSize.
When the memory allocation strategy is POOLED_DIRECT, the sum of s3.wal.cache.size and s3.block.cache.size should be about 1/2 of MaxDirectMemorySize (if MaxDirectMemorySize is large, it can be slightly increased).
| - | Small | Medium | Large |
|---|---|---|---|
| Physical Memory | 8 GiB | 16 GiB | 32 GiB |
| s3.stream.allocator.policy | POOLED_HEAP | POOLED_DIRECT | POOLED_DIRECT |
| -XX:MaxHeapSize | 6g | 6g | 12g |
| -XX:MaxDirectMemorySize | - | 6g | 14g |
| s3.wal.cache.size | 1,073,741,824 | 2,147,483,648 | 6,442,450,944 |
| s3.block.cache.size | 536,870,912 | 1,073,741,824 | 4,294,967,296 |
| s3.wal.upload.threshold | 357,913,941 | 536,870,912 | 536,870,912 |
Network-related Parameters
S3.network.baseline.bandwidth
The baseline for assessing local network bandwidth should be the network's stable capability to achieve both outbound and inbound bandwidth.
Since the server internally performs operations such as writing the WAL, S3, and Compaction, each data write from a client translates into three network writes by the server. By considering the business's read/write ratio, you can ultimately determine the maximum write load a single machine can manage.