1 Star 0 Fork 449

cgb-middleware / Dinky

forked from DataLinkDC / Dinky 
加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
Apache-2.0

Dinky

License Total Lines CN doc EN doc

Stargazers over time

简介

实时即未来,Dlink 为 Apache Flink 而生,让 Flink SQL 纵享丝滑,并致力于实时计算平台建设。

Dinky 基于 Apache Flink 实现 Dlink ,增强 Flink 的应用与体验,探索流式数仓。即站在巨人肩膀上创新与实践,Dinky 在未来批流一体的发展趋势下潜力无限。

最后,Dinky 的发展皆归功于 Apache Flink 等其他优秀的开源项目的指导与成果。

特点

一个 开箱即用易扩展 ,以 Apache Flink 为基础,连接 OLAP数据湖 等众多框架的 一站式 实时计算平台,致力于 流批一体湖仓一体 的建设与实践。

其主要目标如下:

  • 可视化交互式 FlinkSQL 和 SQL 的数据开发平台:自动提示补全、语法高亮、调试执行、语法校验、语句美化、全局变量等
  • 支持全面的多版本的 FlinkSQL 作业提交方式:Local、Standalone、Yarn Session、Yarn Per-Job、Yarn Application、Kubernetes Session、Kubernetes Application
  • 支持 Apache Flink 所有的 Connector、UDF、CDC等
  • 支持 FlinkSQL 语法增强:兼容 Apache Flink SQL、表值聚合函数、全局变量、CDC多源合并、执行环境、语句合并、共享会话等
  • 支持易扩展的 SQL 作业提交方式:ClickHouse、Doris、Hive、Mysql、Oracle、Phoenix、PostgreSql、SqlServer 等
  • 支持 FlinkCDC (Source 合并)整库实时入仓入湖
  • 支持实时调试预览 Table 和 ChangeLog 数据及图形展示
  • 支持语法逻辑检查、作业执行计划、字段级血缘分析等
  • 支持 Flink 元数据、数据源元数据查询及管理
  • 支持实时任务运维:作业上线下线、作业信息、集群信息、作业快照、异常信息、作业日志、数据地图、即席查询、历史版本、报警记录等
  • 支持作为多版本 FlinkSQL Server 的能力以及 OpenApi
  • 支持易扩展的实时作业报警及报警组:钉钉、微信企业号等
  • 支持完全托管的 SavePoint 启动机制:最近一次、最早一次、指定一次等
  • 支持多种资源管理:集群实例、集群配置、Jar、数据源、报警组、报警实例、文档、用户、系统配置等
  • 更多隐藏功能等待小伙伴们探索

原理

dinky_principle

精彩瞬间

FlinkSQL Studio

flinksqlstudio

实时调试预览

selectpreview

语法和逻辑检查

checksql

JobPlan

jobplan

字段级血缘分析

lineage

BI 展示

charts

元数据查询

metadata

实时任务监控

monitor

实时作业信息

jobinfo

数据地图

datamap

数据源注册

datasource

功能

详见 功能

近期计划

  • 多租户及命名空间
  • 全局血缘与影响分析
  • 统一元数据管理
  • Flink 元数据持久化
  • 多版本 Flink-Client Server
  • 整库千表同步

参与贡献

欢迎您为社区贡献自己的力量,共建共赢,贡献流程请参考: [参与贡献]

如何部署

详见 编译安装

如何升级到最新

由于功能较多,所以 bug 及优化点较多,强烈建议你使用或升级到最新版本。 替换最新 Dinky 所有依赖包 ,执行 sql 目录下的 dlink_history.sql 中的部分升级语句,依据是通过版本号与日期来判断从何处开始执行,请不要直接执行全部 sql。

感谢

站在巨人的肩膀上,Dinky 才得以诞生。对此我们对使用的所有开源软件及其社区表示衷心的感谢!我们也希望自己不仅是开源的受益者,也能成为开源的贡献者,也希望对开源有同样热情和信念的伙伴加入进来,一起为开源献出一份力!致谢列表如下:

Apache Flink

Apache Dolphinscheduler

Ant-Design-Pro

Mybatis Plus

Monaco Editor

SpringBoot

感谢 JetBrains 提供的免费开源 License 赞助。

JetBrains

获得帮助

1.提交 issue

2.进入微信用户社区群(推荐,添加微信号 wenmo_ai 邀请进群)和QQ用户社区群(543709668)交流,申请备注 “ Dinky + 企业名 + 职位”,不写不批

3.关注微信公众号获取相关内容的文章(最新消息获取建议关注):DataLink数据中台

4.关注 bilibili UP 主(是文末呀)获取最新视频教学

5.访问 GithubPages官网 网址,阅读最新文档手册

版权

请参考 LICENSE 文件。

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

简介

Dinky 是一个开箱即用的一站式实时计算平台,以 Apache Flink 为基础,连接 OLAP 和数据湖等众多框架,致力于流批一体和湖仓一体的建设与实践。 展开 收起
Java
Apache-2.0
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
Java
1
https://gitee.com/cgb-middleware/Dinky.git
git@gitee.com:cgb-middleware/Dinky.git
cgb-middleware
Dinky
Dinky
dev

搜索帮助