8.4 数据质量模型调用
创建于 2022-10-21 / 最近更新于 2022-10-21 / 1776
字体：
[默认]
[大]
[更大]
支持通过API传入数据，并调起数据质量模型对该数据进行检测，并返回相应的检测结果。
1. 调起数据质量模型
接口定义
请求方式：POST
请求URL：$home_url/universe/public-api/data-quality/model/{modelId}/run
home_url需要填入对应环境的IP/域名，eg: http://u.guandata.com
请求参数：
| Name | Location | 类型 | 含义 | 必填 | 备注 |
| Content-Type | Header | String | 内容类型 | 是 | application/json;charset=utf-8 |
| token | Header | String | 登录用的Token | 是 | 调用登录校验API获取（8.1 登录校验） |
| modelId | 请求URL | Int | 待调用的质量模型ID | 是 | |
| data | Body | Json | 待检测的数据 |
接口调用示例
Header 示例
{
"Content-Type": "application/json; charset=utf-8",
"token": "ab7f1665-8dfc-4d8f-928d-c7a5aefc9332" /*登录校验接口获得的Token（参考8.1）*/
}
POST Body 示例
{
"data": [
{
"column1": "data1_1",
"column2": "data1_2"
},
{
"column1": "data2_1",
"column2": "data2_2"
}
]
}
Response示例
{
"code": 0,
"msg": "success",
"data": 407 // 运行记录id，用于后续接口查询
}
2. 查询数据质量模型运行结果
接口定义
请求方式：GET
请求URL：$home_url/universe/public-api/data-quality/model/{modelId}/record/{recordId}
home_url需要填入对应环境的IP/域名，eg: http://u.guandata.com
请求参数：
| Name | Location | 类型 | 含义 | 必填 | 备注 |
| Content-Type | Header | String | 内容类型 | 是 | application/json;charset=utf-8 |
| token | Header | String | 登录用的Token | 是 | 调用登录校验API获取（8.1 登录校验） |
| modelId | 请求URL | Int | 待调用的质量模型ID | 是 | |
| recordId | 请求URL | Int | 待查询的数据质量模型运行记录ID | 是 |
接口调用示例
Header 示例
{
"Content-Type": "application/json; charset=utf-8",
"token": "ab7f1665-8dfc-4d8f-928d-c7a5aefc9332" /*登录校验接口获得的Token（参考8.1）*/
}
Response示例
{
"code": 0,
"msg": "success",
"data": {
"id": 407,
"modelId": 9,
"ruleRecords": [
{
"datasetId": 98,
"datasetName": null,
"rules": [
{
"id": 923, //此id用于下一个接口查询异常数据
"ruleId": 12,
"modelRecordId": 407,
"datasetId": 98,
"datasetName": null,
"name": "duplication",
"type": "DUPLICATE_VALUE",
"tableLevel": false,
"conditions": [
{
"enableNullFilter": true,
"percentage": 98,
"primaryKeyList": [
"商品编码"
],
"value": 1,
"operator": "LT"
}
],
"duration": 11,
"status": "FAILURE",
"resultDetail": {
"total": 5,
"success": 3,
"failed": 2
},
"errorDetail": null,
"percentage": 74
},
{
"id": 924,
"ruleId": 32,
"modelRecordId": 407,
"datasetId": 98,
"datasetName": null,
"name": "字段检查",
"type": "COLUMN_CHECK",
"tableLevel": true,
"conditions": [
{
"columnName": "大类"
},
{
"columnName": "小类"
}
],
"duration": 1,
"status": "SUCCESS",
"resultDetail": {
"total": 5
},
"errorDetail": null,
"percentage": 100
},
{
"id": 925,
"ruleId": 46,
"modelRecordId": 407,
"datasetId": 98,
"datasetName": null,
"name": "新建规则",
"type": "ROW_COUNT",
"tableLevel": true,
"conditions": [
{
"value": 1233123,
"operator": "EQ"
}
],
"duration": 1,
"status": "FAILURE",
"resultDetail": {
"total": 5
},
"errorDetail": null,
"percentage": 100
}
]
}
],
"startTime": "2021-05-24T16:36:51.000+0800",
"endTime": "2021-05-24T16:37:03.000+0800",
"datasetSnapshots": [
{
"dataType": "API",
"id": 98,
"taskInstanceId": null,
"name": null,
"nodeName": null,
"filePath": "file:///data-quality/d3f887a6-1abd-45ee-85b9-aecb925eb6c9/task_result_data.avro",
"updateTime": "2021-05-24T16:36:50.431+0800",
"colCount": 9,
"rowCount": 5
}
],
"duration": 12,
"status": "FAILURE",
"processInstanceId": null,
"runType": "API"
}
}
3. 查询数据质量规则异常数据
接口定义
请求方式：GET
请求URL：$home_url/universe/public-api/data-quality/rule-record/{ruleRecordId}/preview
home_url需要填入对应环境的IP/域名，eg: http://u.guandata.com
请求参数：
| Name | Location | 类型 | 含义 | 必填 | 备注 |
| Content-Type | Header | String | 内容类型 | 是 | application/json;charset=utf-8 |
| token | Header | String | 登录用的Token | 是 | 调用登录校验API获取（8.1 登录校验） |
| ruleRecordId | 请求URL | Int | 待调用的质量模型ID | 是 | |
| limit | 请求URL | Int | 记录条数，默认返回全部 | 否 |
接口调用示例
Header 示例
{
"Content-Type": "application/json; charset=utf-8",
"token": "ab7f1665-8dfc-4d8f-928d-c7a5aefc9332" /*登录校验接口获得的Token（参考8.1）*/
}
Response示例
{
"code": 0,
"msg": "success",
"data": {
"columns": [
{
"name": "商品编码",
"type": "LONG",
"comment": null
},
{
"name": "大类",
"type": "STRING",
"comment": null
},
{
"name": "中类",
"type": "STRING",
"comment": null
},
{
"name": "小类",
"type": "STRING",
"comment": null
},
{
"name": "品牌",
"type": "STRING",
"comment": null
},
{
"name": "英文名",
"type": "STRING",
"comment": null
},
{
"name": "商品名",
"type": "STRING",
"comment": null
},
{
"name": "售价",
"type": "DOUBLE",
"comment": null
},
{
"name": "测试",
"type": "STRING",
"comment": null
}
],
"preview": [
[
"114577",
"美发护理",
"护法/润发",
"护发素/乳
",
"海飞丝",
"",
"海飞丝护发素丝源复活组合头皮净化530ml",
"99.8",
null
],
[
"114577",
"美发护理",
"护法/润发",
"护发素/乳
",
"海飞丝",
"",
"海飞丝护发素丝源复活组合头皮净化530ml",
"99.8",
null
]
],
"rowCount": null,
"status": null,
"message": null
}
}
4. 查询数据质量模型详情
接口定义
请求方式：GET
请求URL：$home_url/universe/public-api/data-quality/model/{modelId}
home_url需要填入对应环境的IP/域名，eg: http://u.guandata.com
请求参数：
| Name | Location | 类型 | 含义 | 必填 | 备注 |
| Content-Type | Header | String | 内容类型 | 是 | application/json;charset=utf-8 |
| token | Header | String | 登录用的Token | 是 | 调用登录校验API获取（8.1 登录校验） |
| modelId | 请求URL | Int | 待调用的质量模型ID | 是 |
接口调用示例
Header 示例
{
"Content-Type": "application/json; charset=utf-8",
"token": "ab7f1665-8dfc-4d8f-928d-c7a5aefc9332" /*登录校验接口获得的Token（参考8.1）*/
}
Response示例
{
"code": 0,
"msg": "success",
"data": {
"id": 9,
"name": "测试重复检测1",
"projectId": 1,
"description": "描述",
"rules": [
{
"id": 12,
"modelId": 9,
"datasetId": 98,
"name": "duplication",
"type": "DUPLICATE_VALUE",
"tableLevel": false,
"conditions": [
{
"enableNullFilter": true,
"percentage": 98,
"primaryKeyList": [
"商品编码"
],
"value": 1,
"operator": "LT"
}
],
"percentage": 74,
"enableNullFilter": true,
"del": false
},
{
"id": 32,
"modelId": 9,
"datasetId": 98,
"name": "字段检查",
"type": "COLUMN_CHECK",
"tableLevel": true,
"conditions": [
{
"columnName": "大类"
},
{
"columnName": "小类"
}
],
"percentage": 100,
"enableNullFilter": false,
"del": false
}
],
"ruleCount": null,
"enable": true,
"relatedDatasetList": [
{
"dataType": "DATASET",
"id": 98,
"taskInstanceId": null,
"name": "数据集0826",
"nodeName": null,
"updateTime": "2021-07-12T06:28:39.000+0800",
"colCount": 9,
"rowCount": 146,
"fileFullPath": null
}
],
"crontab": "0 53 8,20 ? * SUN *",
"warningType": "FAILURE",
"alertConfigs": [
{
"alertType": "EMAIL",
"receivers": "zhoulina@guandata.com",
"receiversCc": "",
"warningGroupId": 2,
"webHookUrl": null
}
],
"createTime": "2020-09-11T11:32:53.000+0800",
"updateTime": "2020-12-08T14:20:52.000+0800",
"lastRunningTime": null,
"duration": null,
"status": null,
"createUserId": 1,
"createUserName": "shuaihao",
"modifyUserId": 3,
"del": false
}
}
1 人点赞过
