Skip to main content

Configure a Real-Time Sync Task

Overview

After preparing the Data Account, database environment, and Real-time Sync environment, you can create a Real-time Sync task and configure source and target data settings to complete task creation.

Prerequisites

Before creating a task, deploy the Real-time Sync environment, configure the Data Account, and complete the database environment preparation.

Create a Real-Time Sync Task

Configure a Connection Parameter Template in Advance

To ensure that the Real-time Sync task runs properly, you need to configure additional parameters for the target database through a connection parameter template.

|400

  1. Enter the template name.

  2. Select the database type. Currently, StarRocks and GuassDB are supported.

  3. Configure the parameter template.

  4. When the database type is StarRocks, the template typically includes the concurrency level, the IP address and port of the FE node, and the replica count. Example:

    {"parallelism":2,"loadUrl":"172.20.0.77:8030","replicationNum":1}
    1. When the database type is GuassDB, no parameters are required for now, so enter {}.
    2. Enter the template description.

Select the Data Source

Go to the Real-time Sync module and click Create Real-time Sync.

Select the Data Account for the source table, which must be created in advance, and then select the table name.

Select the Target Table

Sync Method

Full Sync + Incremental Sync

First synchronize all existing data, and then continuously synchronize new changes. When the task runs for the first time, it performs a complete full sync followed by incremental sync.

Incremental Sync Only

The task contains only the incremental phase. When the task runs for the first time, incremental synchronization begins from the time the user starts the task.

Configure Table and Field Mapping

  1. Set the target location where the data will be stored.

  2. Configure the mapping between target fields and source fields.

  3. Configure the time marker field. When the target field type is datetime, this field can be set as the time marker field. It records the actual insert and update time of the data in the database as a millisecond-level timestamp.

  4. Select Parameter Template.

    |300

Configure Task Information

Enter the task name and task description, then click Confirm to create the task.