Data replication into Snowflake

SymmetricDS now supports loading data and changes into Snowflake.   Capture changes in your relational database and sync them in near real-time into a Snowflake cloud data warehouse.

Version 3.10 of SymmetricDS now has the ability to bulk load data into Snowflake as well as be configured to replicate changes from an existing relational database into Snowflake as the occur.

To begin setup a SymmetricDS node that will connect to an existing relational database. Through the quick configuration wizard provided with SymmetricDS you can configure all the tables on the source database that should be replicated. Once you have your source node in place and the appropriate tables to be replicated to Snowflake you are all set to begin the Snowflake configuration.

• Setup a database in SnowFlake

Create a new database in SnowFlake or use an existing one while setting up the SnowFlake node below.  In this example we will use a database named “JUMPMIND”.

• Choose a file stage option that will be used to bulk load data into SnowFlake

All 3 staging options are supported by SymmetricDS.  The staged area will be used to hold the data files used during the initial load process from an existing relational database into the SnowFlake database.

• Within SymmetricDS, add a node that will use the “SnowFlake (Load Only)” database dialect.

Using the Add Node Wizard in SymmetricDS a SnowFlake node can be configured.  You will need to provide the account name and database name in the URL text field as well as the user and password to connect to your SnowFlake environment.

• Next, provide the staging information setup in SnowFlake

Configure SymmetricDS to utilize bulk loading capabilities of existing data into Snowflake.  The selections and inputs on this panel should match what was setup in SnowFlake for storage options.  In this blog SnowFlake managed staging was used but AWS S3 and Azure Storage are also available.

Once you have configured the storage for the bulk loading complete the wizard and your SnowFlake node is ready.

• Starting the bulk load to SnowFlake

Using the Load Data Wizard you can send an initial load of data from your source relational database into SnowFlake.