同步操作将从 袋鼠云/chunjun 强制同步,此操作会覆盖自 Fork 仓库以来所做的任何修改,且无法恢复!!!
确定后同步将在后台操作,完成时将刷新页面,请耐心等待。
English | 中文
We are recruiting Big data platform development engineers.If you want more information about the position, please add WeChat ID [ysqwhiletrue] or email your resume to sishu@dtstack.com.
We use DingTalk to communicate,You can search the group number [30537511] or scan the QR code below to join the communication group
Chunjun is a data synchronization tool based on Flink. Chunjun can collect static data, such as MySQL, HDFS, etc, as well as real-time changing data, such as MySQL binlog, Kafka, etc. At the same time, Chunjun is also a computing framework that supports all the syntax and features of native FlinkSql , And provide a large number of cases. Chunjun currently includes the following features:
Most plugins support concurrent reading and writing of data, which can greatly improve the speed of reading and writing;
Some plug-ins support the function of failure recovery, which can restore tasks from the failed location and save running time; Failure Recovery
The source plugin for relational databases supports interval polling. It can continuously collect changing data; Interval Polling
Some databases support opening Kerberos security authentication; Kerberos
Limit the reading speed of source plugins and reduce the impact on business databases;
Save the dirty data when writing data;
Limit the maximum number of dirty data;
Multiple running modes: Local,Standalone,Yarn Session,Yarn Per;
Synchronization tasks support transformer operations that execute flinksql syntax;
sql task support is shared with flinkSql's own connectors;
The following databases are currently supported:
Database Type | Source | Sink | Lookup | |
---|---|---|---|---|
Batch Synchronization | MySQL | doc | doc | doc |
TiDB | reference mysql | reference mysql | ||
Oracle | doc | doc | doc | |
Doris | doc | |||
SqlServer | doc | doc | doc | |
PostgreSQL | doc | doc | doc | |
DB2 | doc | doc | doc | |
ClickHouse | doc | doc | doc | |
Greenplum | doc | doc | ||
KingBase | doc | doc | ||
MongoDB | doc | doc | doc | |
SAP HANA | doc | doc | ||
ElasticSearch7 | doc | doc | doc | |
FTP | doc | doc | ||
HDFS | doc | doc | ||
Stream | doc | doc | ||
Redis | doc | doc | ||
Hive | doc | |||
Solr | doc | doc | ||
File | doc | |||
StarRocks | doc | |||
Stream Synchronization | Kafka | doc | doc | |
EMQX | doc | doc | ||
MySQL Binlog | doc | |||
Oracle LogMiner | doc | |||
Sqlserver CDC | doc | |||
Postgres CDC | doc |
Please click Quick Start
Please click General Configuration
Please click Statistics Metric
Please click Iceberg
Please click Kerberos
Please click Questions
Please click Contribution
Chunjun is under the Apache 2.0 license. See the LICENSE file for details.
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。