Common Batches Just Got Better

Common batch mode was enhanced in SymmetricDS 3.11 to allow any group of nodes to share batches. Let’s look at what common batches are and why this is a big deal for database replication with more than a couple nodes.

Before Common

In the early history of SymmetricDS, it didn’t include the concept of common mode for batches. The router service would put changes into separate batches for each node that needed them. For example, if there were 4 nodes on the system, and the router service found 3 changed rows of data, it would write a new batch for each node and place the changes inside. The screenshot below shows an example, with batches 58 through 61.

Common Batches

Starting in version 3.0, it was optimized to route the change once into a common batch for all the nodes. There is still a separate batch entry to track the status for each node, but they share the same batch number and the same batch data. The screenshot below shows batch 63 is shared.

Sharing is Better

Why is sharing batches better? Because common batches are more efficient to route, extract, and purge. We can see the difference when looking at the runtime model. Changes are captured as “Data” and assigned to one or more “Batches”. The many-to-many relationship between the two is associated in “Data Events”. With common batches, the router service has less work to create batches and the purge service has less work to clean up later. The illustration below shows the difference when routing 3 changes to 4 nodes.

Limitations Lifted

Common mode has been around, but it was limited to channels with only “default” routers and uni-directional replication. In other words, it worked for a channel when sending one-way to all nodes, but not for any arbitrary group of nodes. With the new SymmetricDS 3.11 release, any group of nodes are eligible for a common batch. In other words, now common batch mode is always enabled! Routing to groups of nodes to subset data by chain, region, district, and any other combination now benefit from common mode sharing of batches.