Cross-AZ Replication Traffic Is a Pricing Model, Not Just an Architecture Detail

A platform engineer searching for "Kafka cross-AZ replication cost" wants two things at once: a number, and a way to trust the number. The number changes every month, and trusting it means tracing how the design turns into bytes and how the bytes turn into a bill.

In a multi-AZ Kafka cluster, replication is the feature that makes a broker loss survivable, and it is the feature that moves data across a cloud pricing boundary almost continuously. The mistake is not choosing multi-AZ. The error is recording "replication factor 3" as a durability knob and treating the resulting traffic as an unavoidable overhead that no one owns, models, or approves.

A more useful mental model treats replication traffic as part of the pricing model: a design-governed meter that charges per gigabyte for crossing a boundary. Once the cluster spans zones, leader placement, follower distribution, recovery, and consumer position all set the meter reading. Pricing the design means naming the mechanism, applying the arithmetic to a real topology, and walking the levers that change the price without pretending the availability objective does not exist.

Read the diagram as a meter: every cross-zone replica edge carries a per-gigabyte price, and the bill is the product of those edges, the write rate, and the period.

1A design choice that invoices itself monthly

Apache Kafka keeps partition replicas on broker-local disks. A producer writes to a partition's leader; the leader appends to its local log and replicates to the followers that are eligible for in-sync status. When a leader and one of its followers live in different Availability Zones, the replication path crosses a boundary that cloud providers bill as inter-zone data transfer.

Durability and billing come from the same design. When the leader has a remote follower, it has to move the new records. Charges accumulate continuously, not once when the architecture is drawn. The meter runs on the cross-zone portion of the ordinary replication path, on partition reassignments that move a leader across a zone, on consumer reads that cross a zone, and on the burst that follows a broker loss. Cost follows the topology and its behavior under load and failure, not the broker-hour invoice.

Architecture deserves attention here: reassignments and leader moves shift the multiplier, but they cannot remove the relationship between durable data, broker ownership, and replica movement. The replicas are the data, the data has to move, and the boundary crossing is the price of owning the data the way a broker-local design owns it.

2Where the traffic actually goes: racks, zones, regions

To find the cost, map the movement. Record each broker's rack awareness label and zone, the leaders and followers for representative partitions, and the zones consumers read from. Then mark every path that crosses a zone, region, VPC, or account boundary.

Replication traffic has a steady component and an unsteady component. The steady component is the leader-to-follower stream: for each partition, the eligible edges are the follower connections that cross a boundary, moving at approximately the in-sync write rate each follower must receive. The unsteady component includes follower catch-up after isolation or throttling, leadership changes, and the replica rebuilds that wait on the far side of a broker replacement.

Consumers add a second and often larger movement layer. A local producer does not guarantee a local consumer. An analytics group, a replay job, or a change data capture sink can pull a stream across a zone at a rate that exceeds the produce rate, and the charge lands on the platform even when another team owns the reader.

The map is the spec for the meter. Until the meter has been compared against the topology, an architecture review has not priced the design.

3The multiplication factor for your scale

Replication does not move the logical write rate. It is a multiple of that rate, and the multiple is a property of placement rather than a nameplate value from the configuration.

Define the meters with symbols rather than a single multiplier. Let W be the logical write rate, c_repl the number of eligible cross-zone replica edges averaged over the time window and leader placements, and T the period. Let R be the logical read rate, c_read the number of eligible cross-zone reader edges, and c_recovery the fraction of repaired bytes that cross a boundary:

replication_bytes = W × c_repl × T
consumer_traffic  = R × c_read × T
recovery_bytes    = repaired_bytes × c_recovery

Keep the steady, read, and recovery terms separate. Conflating them hides the read-side cost that no broker-side change can remove, and the recovery term spikes at the most difficult moment, when a broker is replaced in another zone. Replay paths add a retained-volume multiplier on top of the read term.

As an illustrative assumption, a topic that writes 1 TiB per day with c_repl equal to two moves roughly 60 TiB across zones over thirty days, before the provider's per-GiB rate applies. The arithmetic is deliberately simple. The point is that the meter reading depends on topology, not on topic count, and a cluster-level average can hide one hot topic whose leaders keep failing across the boundary, so the metric belongs at the topic and consumer-group level when the bill does not match the model.

A replication factor of three gives the local-disk model two duplication edges for written data. Whether both edges cross a charged boundary depends on the placement of the followers relative to the leader and on how the provider prices intra-zone versus inter-zone traffic. The multiplier is measured from the placement map, not read off the replication factor.

4Four escape hatches and their boundaries

There are four levers for steering cross-AZ replication spend. Each changes behavior differently, and each has a boundary that should be acknowledged before the lever is pulled.

• Placement hygiene. Correct rack awareness, deliberate leader distribution, and colocation of the largest readers with their leaders often remove a meaningful share of the total. It preserves workloads and needs no storage migration, but it requires keeping topology metadata and scheduling aligned, and it cannot remove the replica edges the durability target itself requires.

• Relaxing the durability target. A reduced replication factor or a single-AZ deployment lowers movement, and it must be read as a reliability decision. A single-AZ cluster removes the boundary and its failure isolation at the same time. Teams that choose this path should record the new recovery and availability objective, not the cost saving alone.

• Tiered storage. In the Apache Kafka sense, tiering moves older segments off broker disks while the active log remains broker-based. It reduces how much each broker holds and how much is resent during some repairs, but it does not remove the broker-to-broker replica path for the active write stream. It is a storage economics change more than a replication traffic change.

• Shared storage. Separating durable stream storage from broker-local disks lets a stream survive without every broker owning a copy. When the durable copy no longer depends on broker-to-broker replication, the replication meter for that path can be unwired. This change is the most invasive to validate because it touches durability, read paths, and recovery semantics together.

Read down the comparison: durability reduction removes traffic at the expense of the failure objective; tiered storage keeps the objective but removes little of the replication path; shared storage removes the replication path while keeping the objective, at the cost of validating a new storage model.

5Where shared storage changes the arithmetic

Reaching a replication meter of zero requires the stream's durable copy to stop being a broker's local disk. Shared Storage architecture makes that change by writing the stream to a shared object-storage layer and treating brokers as request-handling and compute nodes rather than the owners of the long-lived data.

AutoMQ implements this with S3Stream, which turns a shared object store into a stream storage layer with WAL storage in front of it so the write path keeps the latency and durability behavior a Kafka API expects. AutoMQ is 100% Kafka API compatible, so the change sits in the storage architecture rather than the application protocol.

In an applicable deployment, whether a follower sits in the same zone or another stops setting how many copies of every record the platform ships. A record is written once through WAL into the shared storage layer, whose durability comes from the object store rather than broker-to-broker replica edges. That is the basis for the Zero cross-AZ traffic claim for the shared storage architecture: what is removed is the replicated-bytes term, not every network packet in the system.

The honest comparison includes the replacement meters. Object storage charges for requests and stored volume, the WAL tier has its own write and durability behavior, and the read path with cache misses from the shared layer has to be priced against the cache strategy. The S3Stream overview and WAL storage pages describe those boundaries. The claim to test is not "no network bytes ever." It is whether the high-multiplier replication term disappears and whether the replacement terms price lower for the workload.

6When rebuilding for this line actually pays off

Do not redesign a platform because one invoice contains a spike. First prove the path: export the traffic map, check leader placement against rack labels, and determine whether the movement came from steady replication, a failed broker, or a replay campaign. Correct placement errors and re-run the accounting before drawing conclusions.

The rebuild decision is economic. Estimate the recurring replication term under expected growth, add the recovery term that carries the cost of broker replacement and the operational work of keeping placement healthy, and compare that against the engineering and migration cost of changing the storage model. Include the object-storage request economics that a shared-storage proof of concept must measure.

Architecture work becomes the reasonable option when the traffic survives topology hygiene, grows with every retained byte and reader, and has become a platform-level line rather than a single topic's quirk. The AutoMQ architecture overview is the starting point for testing the shared-storage claim against your zones, WAL characteristics (Regional EBS WAL or NFS WAL, depending on deployment), retention, and reader mix. End the review with a decision record that separates three quantities: the traffic removable by placement, the traffic accepted for availability, and the traffic that remains because the storage architecture creates it.

If the replication line is recurring and growing rather than a one-time anomaly, start an AutoMQ BYOC evaluation with the same availability objective and the same traffic map. Scope the evaluation to a few representative topics; the surprise invoice spans the platform.

7The meter and the map

Cross-AZ replication traffic is a price attached to a durability design. It can be the right price to pay, and many teams should keep paying it. It belongs in the architecture record next to the failure objective, not as a discovery in a monthly invoice.

The engineer who started with the search query leaves the trace with a way to read the number: find the eligible edges, model the steady, read, and recovery terms, try the four levers from least to most invasive, and compare the shared-storage design's replacement meters against the replication term it removes. The meter does not stop on its own. Someone has to change the map.

8References

• Apache Kafka design documentation — replication
• Apache Kafka broker configuration
• Apache Kafka tiered storage
• AWS data transfer pricing
• Google Cloud VPC network pricing
• Azure bandwidth pricing
• AutoMQ — 100% Kafka API compatibility
• AutoMQ architecture overview
• S3Stream shared streaming storage
• WAL storage
• Eliminate inter-zone traffic

9FAQ

9.1Does a three-AZ Kafka cluster always triple network traffic?

No. The multiplier is a property of the placement. Count how many cross-zone follower edges and reader edges exist in the actual topology, and treat recovery movement separately. A replication factor of three does not by itself mean three times the billed bytes.

9.2Is reducing the replication factor a legitimate cost lever?

It is a durability lever that also reduces movement. It is legitimate only if the resulting recovery and availability behavior remains acceptable, and that review should be recorded as a reliability decision rather than a cost win.

9.3What is the difference between tiered storage and shared storage here?

Tiered storage moves older segments off broker disks while the active log remains broker-based, so the broker-to-broker replication path stays for the write stream. Shared storage removes durable ownership from broker-local disks, which is what allows the replication meter for that path to be removed in an applicable deployment; it replaces that meter with object-storage request, WAL, and read-path costs that have to be measured.

9.4Does shared storage remove all cross-AZ network traffic?

No. It removes the replicated-bytes term for the design it applies to. Other network paths, including the WAL write to the shared layer, object-storage requests, and cache-read behavior, still need to be mapped against the provider's pricing boundaries.