代码拉取完成,页面将自动刷新
同步操作将从 sxfad/porter 强制同步,此操作会覆盖自 Fork 仓库以来所做的任何修改,且无法恢复!!!
确定后同步将在后台操作,完成时将刷新页面,请耐心等待。
compatible v0.3.2+,include v0.3.2
Spring-boot configuration file, configurable running environment (spring.profiles.active), etc. For details, please refer to the spring-boot official documentation.
The above parameters can also be specified via the startup.sh script, for example: startup.sh --logging.level.root=debug
Spring-boot configuration file, configuration files for different runtime environments.
Task configuration file
Tasks for different environments (${env}) are configured in different subdirectories of the config/tasks directory.
spring-boot configuration parameters,default : logging.file=${app.home}/logs/data-porter.log.
${app.home} is specified in the startup script and refers to the root directory of datas-porter.
Used to specify the task node number, which is unique in the distributed environment. Used to self-describe and report heartbeats on zookeeper to implement distributed locks.
eg.
porter.id=100
Whether to upload statistics, including logs, TPS indicators.
Type : Boolean
Distributed cluster implementation, currently only supports zookeeper
Specify the implementation strategy of the distributed cluster.
eg.
porter.cluster.strategy=ZOOKEEPER
Cluster connection parameters.
eg.
porter.cluster.client.url=127.0.0.1:2181
Cluster connection timeout
eg.
porter.cluster.client.sessionTimeout=overtime time, in milliseconds.
Alarm policy driver, currently only supports mail.
Specify the alarm mode
eg.
porter.alert.strategy=EMAIL
Same content notification receiving frequency.
eg.
porter.alert.frequencyOfSeconds=60
porter.alert.client.host=smtp server
porter.alert.client.username=mail address
porter.alert.client.password=password
porter.alert.client.smtpAuth=true
porter.alert.client.smtpStarttlsEnable=true
porter.alert.client.smtpStarttlsRequired=false
Global alarm informer
Type :ArrayList
porter.alert.receiver[index].realName=name
porter.alert.receiver[index].email=mail address
porter.alert.receiver[index].phone=phone number
Configured when multiple task targets and sources share a database connection pool to save database link resources.
Type : Map
Source type
Type : enum
Optional parameter : JDBC
Multiple separated by ","
eg.
porter.source.Named name.url=jdbc link
account name
eg.
porter.source.命名名字.userName=account name
password
eg.
porter.source.Named name.password=password
Connection error retries
Type : Int
Type : Enum
Optional parameters : MYSQL、 ORACLE
Type : Boolean
When the target table has no primary key, the default all fields are primary keys.
default : true
Node task
Type : ArrayList
Task number
Type : String
eg.
porter.task[index].taskId=1
Current task alarm informer
Task alert will be notified : porter.task[index].receiver + porter.alert.receiver
Type :ArrayList
porter.task[index].receiver[index].realName=name
porter.task[index].receiver[index].email=mail address
porter.task[index].receiver[index].phone=phone number
Task consumption source configuration
Type : DataConsumerConfig
Consumer plugin
Type : String
Optional parameter : CanalFetch、KafkaFetch
Message converter
Type : String
Optional parameter : canalRow(1.0 add)、oggJson
Consumer data source
Type : Map
CanalFetch:(1.0 add)
porter.task[index].consumer.source.sourceType=CANAL
porter.task[index].consumer.source.slaveId=mysql slaveId
porter.task[index].consumer.source.address=ip:port
porter.task[index].consumer.source.database=database
porter.task[index].consumer.source.username=account
porter.task[index].consumer.source.password=password
porter.task[index].consumer.source.filter=Subscription form regular
KafkaFetch:
porter.task[0].consumer.source.sourceType=KAFKA
porter.task[0].consumer.source.servers=ip:port,ip:port
porter.task[0].consumer.source.topics=topic
porter.task[0].consumer.source.group=Consumer group
porter.task[0].consumer.source.autoCommit=true|false
Metadata data source
Type : Map
When the configuration is not done, the data between the source and the target will not be compared.(1.1 new rule)
Form 1:
porter.task[index].consumer.metaSource.sourceName=public data source name
Form 2:
porter.task[index].consumer.metaSource.dbType
porter.task[index].consumer.metaSource.url
porter.task[index].consumer.metaSource.userName
porter.task[index].consumer.metaSource.password
porter.task[index].consumer.metaSource.maxWait
porter.task[index].consumer.metaSource.minPoolSize
porter.task[index].consumer.metaSource.maxPoolSize
porter.task[index].consumer.metaSource.initialPoolSize
porter.task[index].consumer.metaSource.connectionErrorRetryAttempts
Custom sync data data extractor
Format : package.className
class path、jar path、Source content
Empty query notification interval, in seconds.
default : 3600
Empty query notification time threshold, in seconds
-1 does not take effect, default 3600
Task loader configuration
Type : DataLoaderConfig
Target loader plugin
Type : Enum
Optional parameter : JdbcBatch、JdbcSingle
Target data source
Type : Map
Form 1:
porter.task[index].loader.source.sourceName=public data source name
Form 2:
porter.task[index].loader.source.dbType
porter.task[index].loader.source.url
porter.task[index].loader.sourceuserName
porter.task[index].loader.source.password
porter.task[index].loader.source.maxWait
porter.task[index].loader.source.minPoolSize
porter.task[index].loader.source.maxPoolSize
porter.task[index].loader.source.initialPoolSize
porter.task[index].loader.source.connectionErrorRetryAttempts
The target end update fails to insert the switch parameter, default enabled.
Type : Boolean
Source and destination schema mapping, used to handle the inconsistency between source and destination naming.
Type : List
porter.task[index].mapper[index].schema=source schema,target schema
porter.task[index].mapper[index].table=source table name, target table name
Data synchronization result check function is not enabled if it is not configured or configured incorrectly.
porter.task[index].mapper[subscript].updateDate=The source table automatically updates the time field, and the target table automatically updates the time field.
Field mapping, no need to configure.
porter.task[index].mapper[subscript].column.Source field name = target field name
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。