What’s New in SymmetricDS 3.8

What’s New in SymmetricDS Pro 3.8

The new release of SymmetricDS 3.8 will help you:

• Improve performance of database replication
• Send and receive partial initial loads
• Efficiently use the network and withstand instability
• Monitor for problems and receive email notifications
• Sync data with VoltDB database
• Explore the database and compare tables for differences

Improved Performance

Performance is improved across the server by adding thread pools, reducing the number of operational queries, and caching of objects. Channels are assigned to a thread queue, which gives the user control of pushing or pulling data channels in parallel. Separate threads are used to extract batch data, transfer across the network, and load into the database. The initial load runs on a background thread, extracting batches at the same time they are sending, and large batches are broken into smaller batches using channel settings. Routing of batches is only done for channels with outstanding data, and gap detection is held in memory for faster processing. Transfers with the staging area were optimized for minimal parsing, and multi-homed nodes benefit from direct transfers between staging areas. The server startup and synchronize triggers times were improved.

Partial Initial Loads and Load Wizard

Initial loads and reloads were combined into a single, simplified wizard interface that allows sending or receiving data between any nodes on the network. The wizard allows the user to multi-select source and target nodes, which tables will be sent, how to subset rows, and options for altering schema or clearing data first. Data load use the same batching process as changes, with rows broken into multiple batches controlled by the channel settings.

Network Resilience

Keepalives are sent during pushing or pulling while waiting for batch data or acknowledgements to be sent, which prevents connection time outs. When a retrying a batch in error, the protocol sends a “retry” command to load the batch from staging instead of resending it across the network. Channels can be throttled to a maximum transfer rate to control priority and prevent saturating the network. The pull has a hybrid pull option that causes remote nodes to push a notification that they should be pulled for outstanding batches, which can give a central node control over how the network is used.

Monitoring and Notification

Monitors can be configured to run on specific nodes to record batch errors, unsent batches, unrouted data, CPU usage, disk space, and memory usage. Each monitor is assigned a threshold value when it should record an event. Monitor events can be synced to a central node for reporting. Notifications can be configured for specific nodes to write to the log or send an email when a monitor event is received. Both monitors and notifications use the extension API, which makes it easy to monitor for custom events.

Support for VoltDB

Replicate data with VoltDB, a high performance NewSQL database, to integrate data in a mixed environment with other databases. Capture changes, load data, and synchronize data with any of the other supported databases. Leverage a rich feature set, including guaranteed data delivery, filtered synchronization, data transformation, and offline mode.

SQL Explorer and DBCompare

The SQL Explorer screen for interactive SQL queries has new features, including auto-completion of SQL statements, cancellation of running queries, examination of trigger text, browsing of database metadata, and pinning of results tabs. It also supports DB Compare, a utility that can be run from command line or within SQL Explorer’s web interface. DB Compare can connect to source and target engines’ databases and compare the data in tables, reporting on different, missing, and extra rows. It can also write the SQL to bring the target tables back in-sync with the source table.