Filter Data Rows

1. Overview

1.1. Function Description

By defining filter conditions, you can precisely filter the dataset to meet different business needs. Supports adding multiple filter rules, filtering out Null values, and setting trigger conditions for filtering.

1.2. Application Scenarios

In actual business scenarios, source data often contains a lot of dirty data, or only part of the data is needed for analysis. The Filter Data Rows operator can be used in these cases.

Application Scenario	Business Example
Filter invalid data	E-commerce platform order data cleaning: When processing e-commerce order data, there may be abnormal orders due to system errors or malicious attacks. By filtering data rows, invalid order records can be excluded to ensure subsequent analysis is based on valid order data
Extract data subset	Regional sales data extraction: For nationwide sales data, you may only care about the performance of a specific region. By filtering data rows, you can extract sales data for that region for detailed analysis and reporting. Quarterly financial report data extraction: In financial data processing, extract data for a specific quarter to generate quarterly financial reports. By filtering data rows, you can select data for a specific quarter

2. Usage Guide

2.1. Operation Steps

1. Drag the Filter Data Rows operator from the ETL operator area to the right canvas editing area;
2. Click the Filter Data Rows operator and add filter rules (supports multiple rules);
3. When configuring filter rules, select the field, filter type and related configuration, filter out Null values, and click OK;
4. Configure the trigger rule for the filter condition;
5. Click OK and preview the data result.

Guandata BI supports four data types for filtering: text, numeric, date, and boolean. Different filter categories support different filter types.

1. Text type: supports selection, range, and condition filtering.

Selection type filtering

Range type filtering

Condition type filtering

2. Numeric type: supports selection and range filtering.

Selection type filtering

Range type filtering

3. Date type: supports selection and range filtering.

Selection type filtering

Range type filtering

4. Boolean type: supports condition filtering.

Filter Category	Supported Filter Types
Text Filter	Selection, Range, Condition
Numeric Filter	Selection, Range
Date Filter	Selection, Range
Boolean Filter	Condition

Filter Type	Description
Selection	Uses the default condition "equals" to filter data. Enter a comparison value (number, keyword, specific date) to find matching data rows
Range	Uses conditions such as "greater than", "range", "equals Null value" to filter data. Enter a comparison value (number) to find matching data rows
Condition	Uses conditions such as "equals", "contains", "starts with" to filter data. Enter a comparison value (keyword) to find matching data rows

In addition, in range and condition filters, filter rules can be set not only for fixed values but also for logic conditions between columns.

For example, to filter out orders where the user placed an order and shipped on the same day, you can select Order Date equals Shipping Date (compare columns).

Note: When adding multiple filter conditions, you can set the trigger condition to meet all rules or meet any rule.

[Explanation of trigger conditions for filtering data rows in ETL](../../../../11-FAQ/1-Data Processing/2-ETL FAQ.md).

2.2. Detailed Description

Below is an example of configuring East China Regional Sales Analysis.

Add Filter Rule

The upstream node is nationwide sales data.

1. Drag the Filter Data Rows operator from the ETL operator area to the right canvas editing area and connect it to the upstream node;
2. Click the Filter Data Rows operator, the left area becomes the current operator configuration area, and rename as needed, e.g., "East China Sales Analysis";
3. Click Add to open the filter
4. Rule editor popup.

Configure Filter Rule Editor

1. Set the filter rule:

• Select field: Region
• Type: Condition
• Operator: Contains
• Comparison: Fixed value
• Enter comparison value: East China
• Filter out Null values: Checked

Preview Data Result

1. Click OK and preview the data result.

For subsequent use of other data processing operators, see Getting Started.