Skip to main content

Database Output (Advanced)

1. Overview

1.1. Function Description

Output dataset is the result data after data flow processing, which can directly push the processed data to the database.

Multiple database output operators can be configured at any node in the ETL data flow, and different storage locations can be specified.

1.2. Prerequisites

At least 1 "Input Dataset" or "Database Input" is required in the data flow before "Database Output" can be configured.

2. Operation Steps

  1. Drag the Database Output operator from the data flow operator area into the right canvas editing area, and connect it using connection lines.

  2. Click the Database Output operator and select the update method.

    |350

Update MethodExplanation
Direct AppendRetain existing data in the target table and write new data
Full UpdateClear historical data from the target table before writing
Insert UpdateIf comparison fields match successfully in the target table, update data will be executed, otherwise data will be directly appended

  1. Select database type, data account, and target table name (table will be automatically created if it doesn't exist).

    |350

  2. Perform field mapping for the target table.

    |550

  3. If the user selects insert update, comparison fields need to be confirmed.

    |350

  4. If predecessor cleaning is needed for the target table, enter scripts in the predecessor SQL. Supports time macro parameters, global parameters, and workflow parameters. For workflow parameter creation methods, see Workflow Parameters.

    Usage example:

3. High-Performance Mode

The following databases support high-performance mode to improve data synchronization performance:

Database TypeNotes
StarRocksDirect append and full update support using high-performance model. When switching to high-performance mode, note the following:

1. HTTP port needs to be configured for StarRocks data account, otherwise task execution will report errors

2. When synchronized data length exceeds the target table field type length, it will automatically be set to null. It is recommended to plan field length in advance and create tables
GaussDBSimply enable the high-performance switch