代码拉取完成，页面将自动刷新
redis-migrate-tool is a convenient and useful tool for migrating data between redis.
Please install automake, libtool, autoconf and bzip2 at first.
To build redis-migrate-tool:
$ cd redis-migrate-tool
$ autoreconf -fvi
$ ./configure
$ make
$ src/redis-migrate-tool -h
src/redis-migrate-tool -c rmt.conf -o log -d
Before run this tool, make sure your source redis machines have enough memory allowed at least one redis generate rdb file.
If your source machines have large enough memory allowed all the redis generate rdb files at one time, you can set 'source_safe: false' in the rmt.conf.
The following commands are not supported to be propagated to the target redis group, because the keys in those commands maybe cross different target redis nodes:
RENAME,RENAMENX,RPOPLPUSH,BRPOPLPUSH,FLUSHALL,FLUSHDB,BITOP,MOVE,GEORADIUS,GEORADIUSBYMEMBER,EVAL,EVALSHA,SCRIPT,PFMERGE
Config file has three parts: source, target and common.
filter supported glob-style patterns:
h?llo matches hello, hallo and hxllo
h*llo matches hllo and heeeello
h[ae]llo matches hello and hallo, but not hillo
h[^e]llo matches hallo, hbllo, ... but not hello
h[a-b]llo matches hallo and hbllo
Use \ to escape special characters if you want to match them verbatim.
For example, the configuration file shown below is to migrate data from single to twemproxy.
[source]
type: single
servers:
- 127.0.0.1:6379
- 127.0.0.1:6380
- 127.0.0.1:6381
- 127.0.0.1:6382
[target]
type: twemproxy
hash: fnv1a_64
hash_tag: "{}"
distribution: ketama
servers:
- 127.0.0.1:6380:1 server1
- 127.0.0.1:6381:1 server2
- 127.0.0.1:6382:1 server3
- 127.0.0.1:6383:1 server4
[common]
listen: 0.0.0.0:8888
threads: 2
step: 1
mbuf_size: 1024
source_safe: true
Migrate data from twemproxy to redis cluster.
[source]
type: twemproxy
hash: fnv1a_64
hash_tag: "{}"
distribution: ketama
servers:
- 127.0.0.1:6379
- 127.0.0.1:6380
- 127.0.0.1:6381
- 127.0.0.1:6382
[target]
type: redis cluster
servers:
- 127.0.0.1:7379
[common]
listen: 0.0.0.0:8888
step: 1
mbuf_size: 512
Migrate data from a redis cluster to another redis cluster with key filter(key's prefix is "abc").
[source]
type: redis cluster
servers:
- 127.0.0.1:8379
[target]
type: redis cluster
servers:
- 127.0.0.1:7379
[common]
listen: 0.0.0.0:8888
filter: abc*
Load data from rdb file to redis cluster.
[source]
type: rdb file
servers:
- /data/redis/dump1.rdb
- /data/redis/dump2.rdb
[target]
type: redis cluster
servers:
- 127.0.0.1:7379
[common]
listen: 0.0.0.0:8888
step: 2
mbuf_size: 512
source_safe: false
Just save rdb file from redis cluster.
[source]
type: redis cluster
servers:
- 127.0.0.1:7379
[target]
type: rdb file
[common]
listen: 0.0.0.0:8888
source_safe: true
Load data from aof file to redis cluster.
[source]
type: aof file
servers:
- /data/redis/appendonly1.aof
- /data/redis/appendonly2.aof
[target]
type: redis cluster
servers:
- 127.0.0.1:7379
[common]
listen: 0.0.0.0:8888
step: 2
You can use redis-cli to connect with redis-migrate-tool. The listening address and port can be setted at common config.
For example, you try the info command:
$redis-cli -h 127.0.0.1 -p 8888
127.0.0.1:8888> info
# Server
version:0.1.0
os:Linux 2.6.32-573.12.1.el6.x86_64 x86_64
multiplexing_api:epoll
gcc_version:4.4.7
process_id:9199
tcp_port:8888
uptime_in_seconds:1662
uptime_in_days:0
config_file:/ect/rmt.conf
# Clients
connected_clients:1
max_clients_limit:100
total_connections_received:3
# Memory
mem_allocator:jemalloc-4.0.4
# Group
source_nodes_count:32
target_nodes_count:48
# Stats
all_rdb_received:1
all_rdb_parsed:1
all_aof_loaded:0
rdb_received_count:32
rdb_parsed_count:32
aof_loaded_count:0
total_msgs_recv:7753587
total_msgs_sent:7753587
total_net_input_bytes:234636318
total_net_output_bytes:255384129
total_net_input_bytes_human:223.77M
total_net_output_bytes_human:243.55M
total_mbufs_inqueue:0
total_msgs_outqueue:0
127.0.0.1:8888>
info command response instruction:
The command behavior is the following:
Parameter:
For example, you try the shutdown command:
$redis-cli -h 127.0.0.1 -p 8888
127.0.0.1:8888> shutdown
OK
After migrate the data, you can use redis_check command to check data in the source group and target group.
Try the redis_check command:
$src/redis-migrate-tool -c rmt.conf -o log -C redis_check
Check job is running...
Checked keys: 1000
Inconsistent value keys: 0
Inconsistent expire keys : 0
Other check error keys: 0
Checked OK keys: 1000
All keys checked OK!
Check job finished, used 1.041s
If you want check more keys, try the follow:
$src/redis-migrate-tool -c rmt.conf -o log -C "redis_check 200000"
Check job is running...
Checked keys: 200000
Inconsistent value keys: 0
Inconsistent expire keys : 0
Other check error keys: 0
Checked OK keys: 200000
All keys checked OK!
Check job finished, used 11.962s
Try the redis_testinsert command:
$src/redis-migrate-tool -c rmt.conf -o log -C "redis_testinsert"
Test insert job is running...
Insert string keys: 200
Insert list keys : 200
Insert set keys : 200
Insert zset keys : 200
Insert hash keys : 200
Insert total keys : 1000
Correct inserted keys: 1000
Test insert job finished, used 0.525s
If you want insert more keys, try the follow:
$src/redis-migrate-tool -c rmt.conf -o log -C "redis_testinsert 30000"
Test insert job is running...
Insert string keys: 6000
Insert list keys : 6000
Insert set keys : 6000
Insert zset keys : 6000
Insert hash keys : 6000
Insert total keys : 30000
Correct inserted keys: 30000
Test insert job finished, used 15.486s
If you want insert only string type keys, try the follow:
$src/redis-migrate-tool -c rmt.conf -o log -C "redis_testinsert string"
Test insert job is running...
Insert string keys: 1000
Insert list keys : 0
Insert set keys : 0
Insert zset keys : 0
Insert hash keys : 0
Insert total keys : 1000
Correct inserted keys: 1000
Test insert job finished, used 0.024s
If you want insert some special type keys, try the follow:
$src/redis-migrate-tool -c rmt.conf -o log -C "redis_testinsert string|set|list 10000"
Test insert job is running...
Insert string keys: 3336
Insert list keys : 3336
Insert set keys : 3328
Insert zset keys : 0
Insert hash keys : 0
Insert total keys : 10000
Correct inserted keys: 10000
Test insert job finished, used 5.539s
Copyright © 2016 VIPSHOP Inc.
Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0
此处可能存在不合适展示的内容，页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容，可点击提交进行申诉，我们将尽快为您处理。
1. 开源生态
2. 协作、人、软件
3. 评估模型