1 Star 0 Fork 174

langyamorenren / Qualitis

forked from WeBank / Qualitis 
加入 Gitee
与超过 800 万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README.md

License

English | 中文

Overview

Qualitis is a data quality management platform that supports quality verification, notification, and management for various datasource. It is used to solve various data quality problems caused by data processing.

Based on Spring Boot, Qualitis submits quality model task to Linkis platform. It provides functions such as data quality model construction, data quality model execution, data quality verification, reports of data quality generation and so on.

At the same time, Qualitis provides enterprise-level features of financial-level resource isolation, management and access control. It is also guaranteed working well under high-concurrency, high-performance and high-availability scenarios.

Features

  • Define Data Quality Model
    Supports data quality model below:
    1.Model of single table.
    2.Model of multi-table.
    3.Custom model.

    At the same time, Qualitis presets multiple data quality verification templates, including common checks such as null check, blank check, number check, enumeration check and so on, which simplifies data quality model definition.

  • Data Quality Model Scheduling
    Supports data quality model scheduling.

  • Data Quality Report
    Supports generating data quality reports.

  • Log Management Supports management of data quality task.

  • Abnormal Data Management
    Supports abnormal data storage, which can quickly locate problems.

  • Workflow
    Supports workflow
    Workflow needs DataSphereStudio.

  • Administrator Console
    Administrator console provided.
    And it also supports personnel management, access control management, privilege control management, metadata management and so on.

Compared with similar systems

Documents

Quick Deploy
User Manual
Architecture Design

Click here to view more documents

Architecture

Road Map

1. Support Verification in Report Data

  • Support verification in report data in workflow

2. Support Verification in Multi-Storage

  • Support data quality verification between Hive, HDFS, and MySQL
  • Support data quality verification between kafka

3. Support generating data quality report

Supports generating data quality reports with optional latitude.

4. Support intelligent discovery of data quality problems


Communication

If you desire immediate response, please kindly raise issues to us or scan the below QR code by WeChat and QQ to join our group:

License

Linkis is under the Apache 2.0 license. See the LICENSE file for details.

Tips

Because the front-end code of Qualitis is FES, a front-end framework developed by WeBank, but the FES framework is not currently open source, so the front-end code of Qualitis is not open source for the time being. We will promote Qualitis's front-end open source as soon as possible. We are very sorry about this.

You can download the source code and unzip web/app/src/main/resources/static/static.zip to get the compiled front-end code, or use it directly by downloading the release.

仓库评论 ( 0 )

你可以在登录后,发表评论

简介

Qualitis是一个支持多种异构数据源的质量校验、通知、管理服务的数据质量管理平台,用于解决业务系统运行、数据中心建设及数据治理过程中的各种数据质量问题。 Qualitis基于Spring Boot,依赖于Linkis进行数据计算,提供数据质量模型构建,数据质量模型执行,数据质量任务管理,异常数据发现保存以及数据质量报表生成等功能。并提供了金融级数据质量模型资源隔离,资源管控,权限隔离等企业特性,具备高并发,高性能,高可用的大数据质量管理能力。 展开 收起
Java
Apache-2.0
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
Java
1
https://gitee.com/langyamorenren/Qualitis.git
git@gitee.com:langyamorenren/Qualitis.git
langyamorenren
Qualitis
Qualitis
master

搜索帮助

103111 552b83b3 1850385 103110 ed87a847 1850385