Skip to main content

Dataset Operator Overview

1. Overview

1.1. Function Description

In a complete ETL task, input and output nodes ensure the correct flow and processing from source to destination, and are indispensable components of the ETL lifecycle.

In Guandata Smart ETL, input and output operators are collectively referred to as dataset operators, including Input Dataset and Output Dataset operators, representing the original and result datasets respectively.

Supports rapid integration of multi-source heterogeneous data (multi-input), and output at any node in the data flow (multi-output).

image.png

1.2. Usage Restrictions

  1. Smart ETL requires at least one Input Dataset operator, and at least one Input Dataset to configure an Output Dataset.
  2. Input datasets can come from file data, database datasets (excluding direct-connect databases and view datasets), and output datasets from other Smart ETL tasks.

2. Usage Instructions

  1. Drag the Input Dataset operator from the ETL canvas area to the right canvas editing area, and click the operator to upload the source data;
  2. Drag other operators for data processing, and connect the operators with lines;
  3. After data processing is complete, drag the Output Dataset operator to the right canvas editing area;
  4. Click the Output Dataset operator, name and select the storage location;
  5. Preview the output dataset result, and after confirming it is correct, save or run the task as needed in the upper right corner.

(If you select "Save, Run and Exit", the output dataset will be automatically generated after the ETL runs successfully.)

image.png

3. Learning Path

You can learn directly according to the following content:

Operator NameDescription
[Input Dataset](1-Input Dataset.md)Provides the data foundation for the first stage (extraction) of the ETL process, preparing for subsequent ETL data processing, and supports rapid integration of multi-source heterogeneous data (multi-input)
[Output Dataset](2-Output Dataset.md)The result data after ETL processing, which can be used for subsequent business analysis and report analysis, and supports output at any node (multi-output)