Skip to main content

Input Dataset

1. Overview

The input dataset is an important pre-operator (starting point) in the ETL process, providing the data foundation for the first stage (extraction) of the ETL process and preparing for subsequent ETL data processing.

Supports rapid integration of multi-source heterogeneous data (multi-input), enabling users to easily integrate data from different sources and structures.

2. Usage Guide

2.1. Operation Steps

  1. Drag the Input Dataset operator from the ETL operator area to the right canvas editing area;
  2. Click the Input Dataset operator and select the target dataset;
  3. Click OK to upload the dataset;
  4. Optionally configure preview rules for the input dataset;
  5. You can then add other operator nodes to form a complete data processing workflow.

image.png

2.2. Detailed Description

Note: Input datasets can come from file data, database datasets (excluding direct-connect databases and view datasets), and output datasets from other Smart ETL tasks.

Below is an example of uploading an Excel file dataset.

  1. Drag the Input Dataset operator from the ETL operator area to the right canvas editing area;

image.png

  1. Click the Input Dataset operator, enter the dataset name Simulated Data 6, then select Tile. Choose as needed: Tile/Directory;

image.png

  1. Select Excel type, find the Simulated Data 6 dataset, and then click OK. The entire folder path of the dataset is displayed to help users quickly identify the needed dataset;

image.png

Note: When switching between Tile and Directory during search, the entered search terms and dataset type filters will be cleared. After directory search, clicking into the next level folder or returning to the previous folder will also clear the search terms and dataset type filters.

In the Tile page, dataset information is displayed as read-only. To edit, please go to the Data Center.

Search ModeDescription
Tile SearchAll datasets under the root directory become peers. Enter search content to directly find the corresponding dataset. The right side of the window also shows the folder path and details of the selected dataset.
Directory SearchAll folders and datasets in the current directory become peers. Enter search content to find the corresponding folder and dataset. Clicking into the next level folder clears the search box. Searching again will search all folders and datasets in the current folder.
  1. After the dataset is imported successfully, the left configuration bar will display the dataset type, storage path, detailed field information, etc. as shown below:

image.png

  1. Optionally configure preview rules for the input dataset.
  • All data
  • Partial data - limit number of rows
  • Partial data - set filter conditions

Note: To improve ETL preview performance, it is recommended to select partial data. This only applies to preview data, not the final data.

image.png

For subsequent use of other data processing operators, see Getting Started.