Data Load Wizard in SymmetricDS 3.8

Data replication not only involves replicating changes moving forward but also the ability to load existing data between systems prior to replication being put in place.  Data loading was enhanced in SymmetricDS 3.8 to bring together all the features around this process into a new data load wizard.

Why I would need to load existing data

• Target database is empty and new tables and data need to be built.
• To get two databases initially in sync with each other.
• There was an outage and some data needs to be resent from another database.
• Some new tables were added to an existing replication configuration.
• The current replication is in error due to some parent tables that were not originally setup for replication.

Overview

The new data load wizard will walk users through 4 screens to properly construct a load request.

1. Selecting the source and target nodes.  
2. Selecting a full or partial load as well as a “before action” that occurs before loading the data.
3. Selecting the tables involved along with any conditional selection of data in each table.
4. Reviewing a summary of the request.

The load data wizard is located on the manage nodes screen through the “Load Data” button at the top of the table.

Selecting the source and target nodes

In this screen you can select multiple sources or multiple targets to load data between.   The nodes must be part of the same node group though in order to multi-select.  This achieves the same functionality that would have been performed using the “send initial load to” and “request initial load from” prior to 3.8.   

Full or Partial Load and Before Actions

Next you can choose to load all tables currently configured for replication (full initial load) or you can choose to select only a subset of tables (available on next screen).   At this point you also have the ability to choose an action to take before the load sends the data.   

Before Actions

• No action needed just send the data
• Create any tables that are not found on the target as part of this load
• Delete all data in the target tables prior to loading
• Truncate target tables that are part of the load prior to loading
• Execute a custom SQL script for each table.  Here you can use a variable %s to represent each table name

Partial Load Table Selection

If you selected a partial load you will see a screen to select tables next.   If you selected a full load you will be taken next to the summary screen and will bypass the table selection screen.   On the table selection screen you can check the tables you would like to load and you also can provide a where clause on how the source data should be gathered.  This allows you to load a specific table based on last 5 days for example if you have a time stamp column to provide in the where clause.   You do not need to provide the “where” keyword in this clause.   You also can provide a set of variables that will be resolved.

Variables available in where clause

• $(groupId)
• $(nodeId)
• $(externalId)

Summary

Finally you will see a summary screen of the load to review before it is created and queued up to process.