Returns are one of the most common problems faced by retailers. Returns management software can …
Load testing of database replication finds the upper limit of how well the system can perform, and it provides assurance that replication will make it through times of peak usage. Let’s look at how to simulate production activity for SymmetricDS data replication in a lower environment so you can deploy with confidence.
Our load test will orchestrate many simulated SymmetricDS clients syncing data to an actual SymmetricDS server. We’ll start with a simple batch of data on the “heartbeat” channel that updates the sym_node_host table. The test framework allows additional batches of data on other channels to be added, so it can be customized to use actual data from an environment. The clients are simulated using The Grinder load testing framework.
Clone or download the SymmetricDS Load Test project from GitHub. It requires the Gradle build tool is installed on your system.
First, build the agent and install it to a directory. Each agent will run our tests in the background, using multiple threads and processes. Multiple agents can also be installed across different machines to scale out resources.
$ gradle assemble $ unzip build/distributions/symmetric-loadtest.zip -d ~/ $ mv ~/symmetric-loadtest ~/loadtest-agent
Second, build the console and install it to the directory. You only need one console to control multiple agents.
$ gradle assemble -Pconsole=true $ unzip build/distributions/symmetric-loadtest.zip -d ~/ $ mv ~/symmetric-loadtest ~/loadtest-console
Run the console, which opens a graphical application to control the agents and the load test.
$ cd ~/loadtest-console/bin $ ./symmetric-loadtest
Run each agent, which can be run as a background process. Repeat this step for each agent you installed.
$ cd ~/loadtest-agent/bin $ ./symmetric-loadtest
Each agent will connect to the console. Under the “Processes” tab of the console, you should see each agent listed there.
In the console, go to the “Script” tab. Here are the files included:
Edit the “loadtest.properties” file. Either specify the number of runs (0 means run forever) or how long the test should run. This example runs each test 50 times or for 2 minutes, which ever comes first:
Tell each agent how many threads to use and how often to run each test. Tests run simultaneously across all the threads. This example has each agent using 3 threads, and each thread is running tests every 10 seconds.
Configure unique node IDs for each agent to use. This example gives three node IDs to each of the three agents.
locations.agent.id.0.process.id.0=5,6,7 locations.agent.id.1.process.id.0=8,9,10 locations.agent.id.2.process.id.0=11,12,13
The SymmetricDS server needs these nodes to be configured in its database. Run SQL on its database to create each node and set its password. The “server.auth.token” property should match the node_password field of the sym_node_security table.
insert into sym_node (node_id, node_group_id, external_id, sync_enabled, created_at_node_id) values ('5', 'client', '5', 1, 'server'); insert into sym_node_security (node_id, node_password, registration_time, created_at_node_id) values ('5', 'test', current_timestamp, 'server');
Configure the test to target a SymmetricDS server. It needs to know the node ID and sync URL of the target server that the clients will connect to.
target.node.id=server server.url=http://localhost:31415 server.path=/sync/server-central
After configuring the test and saving the file, select Distribute -> Distribute Files from the menu to send the configuration out to all the agents. To perform the load testing, select Action -> Start Processes. Now switch to the “Results” tab to watch statistics.
As the database replication runs, the tests per second (TPS) and batches will increase. When a test runs, it may not successful sync a batch of data. The SymmetricDS server can reject data synchronization requests with a “too busy” response, when the number of requests exceeds its http.concurrent.workers.max parameter. The parameter can be adjusted to control an acceptable amount of load on the database.
We’ve simulated a load test for SymmetricDS database replication that included “heartbeat” batches of data. Now you can add more channels and batches to the test by configuring them in the “loadtest.properties” file. To add batch files from an actual environment, copy them from the SymmetricDS staging directory (under the “tmp” sub-directory) or save them from the Manage -> Outgoing Batches screen. You can also run your batch processes on the server’s database to simulate changes that the clients will pull and acknowledge. Now you have a working load test harness to build a simulated production load.