What’s New in SymmetricDS 3.10

The new release of SymmetricDS Pro 3.10 will help you:

• Initial load target database faster
• Monitor load status and estimate time to completion
• Recover automatically from foreign key and constraint errors
• Deploy loads and changes to Snowflake database

Outgoing Load Screen

The Outgoing Load screen is enhanced to open a tab of details on how the load is configured and its current progress. Each step of the load is shown, including progress to calculate the number of tables and rows before loading begins. Progress by table is shown across panels for pending, active, and completed. Within the active panel, each table is displayed with a progress bar and statistics broken down by extracting, sending, and loading. Overview progress is summarized with a pie chart of loaded versus remaining and an estimated time until completion. A time series chart shows performance of the load as both rows per second and megabytes per second.

Faster Initial Load

Initial and partial loads are faster and more resilient to constraint errors. The queuing step of the load now estimates row counts and batches using system views provided by each database platform, which is considerably faster than getting exact counts for large systems. Extraction queries no longer use SQL functions and format data within code to offload the database server. When multiple targets are loaded simultaneously, the extract service will extract common tables only once to improve overall performance. For Oracle and Tibero databases, tables with large objects (LOBs) receive two passes to optimize extraction for data under the 4k limit. These databases also have new loaders using SQL*Loader and tbLoader to bulk load into tables as fast as possible. Indexes and constraints on target tables are deferred until after the data load, which improves load times. If the user wants to leave constraints on tables instead, the load can automatically handle any constraint errors that occur due to timing of table extraction on a busy system undergoing changes.

Enhancements to Change Data Capture

The fallback operations that automatically resolve constraint violations are enhanced to handle more cases and to record an error only after attempting resolution. Complex chaining of exceptions are handled to get data back into sync, including primary key, unique key, parent missing, and child dependent violations. To maintain performance of the change capture subsystem, a more aggressive purge algorithm is used to make more frequent passes that reduce the size and overhead of runtime tables. If PostgreSQL 9.5 or newer is being used, insert performance is improved using the new “on conflict” option for detecting conflicts. For Oracle and Tibero, large object (LOB) routing and extraction automatically optimizes for data under the 4k limit.

Snowflake Database

Replicate data into the Snowflake database, a data warehouse based in the cloud for data storage and analytics. Use initial loads, partial loads, and change data capture from a source relational database to target data into Snowflake and keep it in sync. The bulk loader integrates with Snowflake managed storage, AWS S3, and Azure Storage Account to quickly populate the data warehouse.