Skip to main content

Output Dataset

1. Overview

1.1. Function Description

The output dataset is the result data after ETL processing, which can be used for subsequent business analysis and report analysis.

Multiple output dataset operators can be configured at any node in the ETL data flow, and different storage locations can be specified for ETL output datasets.

Users can also set acceleration fields for ETL output datasets. The system will shard the dataset according to these fields, thereby improving the speed of card queries using these datasets.

1.2. Prerequisites

At least one Input Dataset is required in Smart ETL to configure an Output Dataset.

2. Usage Guide

2.1. Operation Steps

  1. Drag the Output Dataset operator from the ETL operator area to the right canvas editing area and connect it with lines;
  2. Click the Output Dataset operator, name it, select the storage location, and preview the data result;
  3. Edit output dataset field comments as needed, and set acceleration fields and other configurations as needed;
  4. After the ETL task runs successfully, the system will automatically output the ETL dataset to Data Center - Datasets;
  5. You can then configure preview permissions as needed to manage the ETL output dataset.

image.png

2.2. Detailed Description

Note: During operation, ensure the connection line is solid. A dashed line indicates a broken link and you need to check the previous steps.

  1. Drag the Output Dataset operator from the ETL operator area to the right canvas editing area and connect it to the upstream operator;

image.png

  1. Click the Output Dataset operator, name it, select the storage location, and preview the data result;

image.png

  1. Click Edit Comment to modify output dataset field comments, then click OK to save. Output datasets automatically inherit input dataset field comments, which can be modified as needed;

image.png

  1. Click the icon above the Output Dataset operator (as shown in area 1 of the figure) to set acceleration fields. The system will shard the dataset according to these fields, making downstream card queries faster. It is recommended to set commonly used filter fields (such as date) as acceleration fields;

Note: Up to three acceleration fields are supported. Set acceleration fields can be viewed in the configuration details on the left.

image.png

  1. In the upper right corner of the page, select Save, Run and Exit to get the converted dataset, which is the ETL dataset. You can choose the save method as needed;

image.png

  1. For ETL datasets, administrators can go to Management Center > System Settings > Advanced Settings to set whether to allow visitors to preview datasets according to ETL owner permissions, etc.

    · Enabled: ETL visitors will preview datasets in ETL according to owner permissions;

    · Disabled: When previewing datasets in ETL, ETL owners, visitors, and system administrators will preview according to their own dataset permissions. ETL owners and system administrators preview according to owner permissions. Please operate with caution!

image.png