liangqi
/
RedisShake
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
redis-shake工具
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
212 Commits
1 Branch
0 Tags
25 MiB
 
 
 
Tag: Branch: Tree: 9ae8e7f2f0
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '9ae8e7f2f0'
${ noResults }
Go to file
vinllen 9ae8e7f2f0
update .gitignore 		6 years ago
conf 1.source address supports cluster 2.target address supports several proxies to write data in a round-robin way. 6 years ago
src 1.source address supports cluster 2.target address supports several proxies to write data in a round-robin way. 6 years ago
.gitignore update .gitignore 6 years ago
ChangeLog 1.source address supports cluster 2.target address supports several proxies to write data in a round-robin way. 6 years ago
Dockerfile Create Dockerfile 6 years ago
MIT-LICENSE.txt initial release 6 years ago
Makefile initial release 6 years ago
README.md update .gitignore 6 years ago
build.sh polish 6 years ago
wandoujia_license.txt initial release 6 years ago

README.md

RedisShake is mainly used to synchronize data from one redis database to another.
Thanks to the Douyu's WSD team for the support.

Redis-Shake

Redis-shake is developed and maintained by NoSQL Team in Alibaba-Cloud Database department.
Redis-shake has made some improvements based on redis-port, including bug fixes, performance improvements and feature enhancements.

Main Functions

The type can be one of the followings:

  • decode: Decode dumped payload to human readable format (hex-encoding).
  • restore: Restore RDB file to target redis.
  • dump: Dump RDB file from souce redis.
  • sync: Sync data from source redis to target redis by sync or psync command. Including full synchronization and incremental synchronization.
  • rump: Sync data from source redis to target redis by scan command. Only support full synchronization. Plus, RedisShake also supports fetching data from given keys in the input file when scan command is not supported on the source side.

Please check out the conf/redis-shake.conf to see the detailed parameters description.

Configuration

Redis-shake has several parameters in the configuration(conf/redis-shake.conf) that maybe confusing, if this is your first time using, just configure the source.address and target.address parameters.

Verification

User can use RedisFullCheck to verify correctness.

Metric

Redis-shake offers metrics through restful api and log file.

  • restful api: curl 127.0.0.1:9320/metric.
  • log: the metric info will be printed in the log periodically if enable. m

Redis Type

Both the source and target type can be single redis, master-slave architecture redis, redis cluster and codis. Although the architecture patterns of different vendors are different for the cluster architecture, we still support different cloud vendors like alibaba-cloud, tencent-cloud and so on.
If the target is open source redis cluster, redis-shake uses redis-go-cluster driver to write data, otherwise, redigo driver is used to insert data in round robin way.
If the source is redis cluster, redis-shake launches multiple goroutines for parallel pull. User can control the concurrency.
The "move slot" operations must be disabled on the source side.

Code branch rules

Version rules: a.b.c.

  • a: major version
  • b: minor version. even number means stable version.
  • c: bugfix version
branch name rules
master master branch, do not allowed push code. store the latest stable version. develop branch will merge into this branch once new version created.
develop(main branch) develop branch. all the bellowing branches fork from this.
feature-* new feature branch. forked from develop branch and then merge back after finish developing, testing, and code review.
bugfix-* bugfix branch. forked from develop branch and then merge back after finish developing, testing, and code review.
improve-* improvement branch. forked from develop branch and then merge back after finish developing, testing, and code review.

Tag rules:
Add tag when releasing: "release-v{version}-{date}". for example: "release-v1.0.2-20180628"

Usage

You can use the binary in the release package.
You can also build redis-shake yourself according to the following steps, the go and govendor must be installed before compile:

  • git clone https://github.com/alibaba/RedisShake.git
  • cd RedisShake
  • export GOPATH=`pwd`
  • cd src/sync
  • govendor sync #please note: must install govendor first and then pull all dependencies: go get -u github.com/kardianos/govendor
  • cd ../../ && ./build.sh
  • ./bin/redis-shake -type=$(type_must_be_sync_dump_restore_or_decode) -conf=conf/redis-shake.conf #please note: user must modify collector.conf first to match needs.

Shake series tool

We also provide some tools for synchronization in Shake series.

Plus, we have a WeChat group so that users can join and discuss, but the group user number is limited. So please add my WeChat number: vinllen_xingge first, and I will add you to this group.