Filter Data Rows

Overview

Feature Description

By defining filter conditions, you can precisely filter a dataset to meet different business needs. This operator supports multiple filter rules, filtering out Null values, and setting trigger conditions for filtering.

Use Cases

In real business scenarios, source data often contains a large amount of dirty data, or analysis requires only a subset of the data. In these cases, the Filter Data Rows operator can be used.

Use Case	Business Example
Filter Invalid Data	E-commerce order data cleansing: when processing e-commerce order data, abnormal orders may exist due to system issues or malicious attacks. Filter Data Rows can exclude invalid order records and ensure that later analysis is based on valid orders.
Extract a Data Subset	Regional sales data extraction: from nationwide sales data, you may only care about performance in a specific region. Filter Data Rows can extract that regional data for detailed analysis and reporting. Quarterly financial report extraction: during financial data processing, data for a specific quarter can be extracted to generate a quarterly financial report.

User Guide

Steps

1. Drag the Filter Data Rows operator from the ETL operator area to the canvas on the right.
2. Click the Filter Data Rows operator and add one or more Filter Rules.
3. When configuring a filter rule, select the field, filter type, and related settings, optionally filter out Null values, and click OK.
4. Configure the trigger rule for the filter conditions.
5. Click OK and preview the data result.

Guandata BI supports four filter categories: Text Filter, Numeric Filter, Date Filter, and Boolean Filter. Different filter categories support different filter types.

• Text type: supports Select, Range, and Condition.

  • Select filter

    

  • Range filter

    

  • Condition filter

    

• Numeric type: supports Select and Range.

  • Select filter

    

  • Range filter

    

• Date type: supports Select and Range`.

  • Select filter

    

  • Range filter

    

• Boolean type: supports condition filter.

  

Filter Category	Supported Filter Types
Text Filter	Selection, Range, Condition
Numeric Filter	Selection, Range
Date Filter	Selection, Range
Boolean Filter	Condition

Filter Type	Description
Selection	Filters data using a default condition of Equals. Enter a comparison value such as a specific value, keyword text, or date to find matching rows.
Range	Filters data using conditions such as Greater Than, Between, or Equals Null. Enter comparison values such as specific numbers to find matching rows.
Condition	Filters data using conditions such as Equals, Contains, or Starts With. Enter comparison values such as keyword text to find matching rows.

In addition, for Range filters and Condition filters, filter rules can be configured not only against fixed values, but also between columns.

For example, if you want to filter orders that were shipped successfully on the same day the user placed the order, you can choose ORDER_DATE equals SHIP_DATE as a column comparison.

Notes

When multiple filter conditions are added, the trigger condition can be set to Match All Rules or Match Any Rule.

Explanation of ETL filter trigger conditions

For other data processing operators used later, see Getting Started.