Filter Rows

Feature Description

Filter Rows performs precise filtering on a dataset by defining filter conditions to meet different business requirements. It supports multiple filter rules, filtering out Null values, and setting trigger logic for the filters.

In real business scenarios, source data may contain large amounts of dirty data, or analysis may require only part of the dataset. In these cases, the Filter Rows operator can be used.

Use Case	Business Example
Filter invalid data	E-commerce order data cleanup: When processing order data, abnormal orders may exist due to system errors or malicious attacks. Filter Rows can exclude invalid order records so that subsequent analysis is based only on valid orders.
Extract a data subset	Regional sales extraction: For nationwide sales data, you may only care about the performance of a specific region. Filter Rows can extract data for that region for detailed analysis and reporting. Quarterly financial report extraction: In financial data processing, you can extract data for a specific quarter to generate a quarterly report.

Procedure

1. Drag the Filter Rows operator from the dataflow operator panel into the canvas on the right.

2. Click the Filter Rows operator and add Filter Rules. Multiple rules are supported.

3. When configuring a filter rule, choose the field, filter type, related options, and whether to exclude Null values, then click Confirm.

   

   Time macro parameters, global parameters, and workflow parameters are supported as comparison items.

   

Different filter types support different parameter behaviors.

Filter Category	Filter Type	Operators	Supports Time Macros and Parameters
Text	Selection	-	No
	Range	Less than, less than or equal to, greater than, greater than or equal to, equal to, not equal to, between	Yes. The comparison dropdown includes workflow parameters and global parameters, and the comparison value supports text-type workflow and global parameters.
		Equal to Null, not equal to Null	No
	Condition	Equal to, not equal to, contains, does not contain, starts with, ends with, does not start with, does not end with	Yes. The comparison dropdown includes workflow parameters and global parameters, and the comparison value supports text-type workflow and global parameters.
Numeric	Selection	-	No
	Range	Less than, less than or equal to, greater than, greater than or equal to, equal to, not equal to, between	Yes. The comparison dropdown includes workflow parameters and global parameters, and the comparison value supports text-type workflow and global parameters.
Date	Selection	-	No
	Range	Less than, less than or equal to, greater than, greater than or equal to, equal to, not equal to, between	Yes. The comparison dropdown includes time macros, workflow parameters, and global parameters, and the comparison value supports date-type workflow and global parameters.
		Equal to Null, not equal to Null, and relative date options such as yesterday and last week	No

4. Configure the trigger rule for the filter conditions.
5. Click Confirm, then preview the result.

Four data filter categories are supported: text, numeric, date, and boolean. Different categories support different filter types.

Filter Category	Supported Filter Types
Text	Selection, Range, Condition
Numeric	Selection, Range
Date	Selection, Range
Boolean	Condition

Filter Type	Description
Selection	Uses the default condition Equal to for filtering. Enter a comparison value such as a number, keyword, or specific date to find matching rows.
Range	Uses conditions such as Greater than, Between, or Equal to Null for filtering. Enter a comparison value such as a number to find matching rows.
Condition	Uses conditions such as Equal to, Contains, or Starts with for filtering. Enter a comparison value such as a keyword to find matching rows.

In addition, for Range and Condition filters, rules can be configured not only against fixed values but also between columns.

For example, to filter orders that were shipped on the same day they were placed, you can set Order Date equal to Ship Date as a comparison column.

Note: When multiple filter conditions are added, you can choose either Meet All Rules or Meet Any Rule as the trigger mode.