You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
85 lines
4.6 KiB
85 lines
4.6 KiB
RedisShake is mainly used to synchronize data from one redis database to another.<br>
|
|
Thanks to the Douyu's WSD team for the support. <br>
|
|
|
|
* [中文文档](https://yq.aliyun.com/articles/691794)
|
|
|
|
# Redis-Shake
|
|
---
|
|
Redis-shake is developed and maintained by NoSQL Team in Alibaba-Cloud Database department.<br>
|
|
Redis-shake has made some improvements based on [redis-port](https://github.com/CodisLabs/redis-port), including bug fixes, performance improvements and feature enhancements.<br>
|
|
|
|
# Main Functions
|
|
---
|
|
The type can be one of the followings:<br>
|
|
|
|
* **decode**: Decode dumped payload to human readable format (hex-encoding).
|
|
* **restore**: Restore RDB file to target redis.
|
|
* **dump**: Dump RDB file from souce redis.
|
|
* **sync**: Sync data from source redis to target redis by `sync` or `psync` command. Including full synchronization and incremental synchronization.
|
|
* **rump**: Sync data from source redis to target redis by `scan` command. Only support full synchronization. Plus, RedisShake also supports fetching data from given keys in the input file when `scan` command is not supported on the source side.
|
|
|
|
Please check out the `conf/redis-shake.conf` to see the detailed parameters description.<br>
|
|
|
|
# Configuration
|
|
Redis-shake has several parameters in the configuration(`conf/redis-shake.conf`) that maybe confusing, if this is your first time using, just configure the `source.address` and `target.address` parameters.
|
|
|
|
# Verification
|
|
---
|
|
User can use [RedisFullCheck](https://github.com/alibaba/RedisFullCheck) to verify correctness.<br>
|
|
|
|
# Metric
|
|
---
|
|
Redis-shake offers metrics through restful api and log file.<br>
|
|
|
|
* restful api: `curl 127.0.0.1:9320/metric`.
|
|
* log: the metric info will be printed in the log periodically if enable.
|
|
m
|
|
|
|
# Redis Type
|
|
---
|
|
Both the source and target type can be single redis, master-slave architecture redis, redis cluster and codis. Although the architecture patterns of different vendors are different for the cluster architecture, we still support different cloud vendors like alibaba-cloud, tencent-cloud and so on.<br>
|
|
If the target is open source redis cluster, redis-shake uses [redis-go-cluster](https://github.com/chasex/redis-go-cluster) driver to write data, otherwise, [redigo](https://github.com/garyburd/redigo) driver is used to insert data in round robin way.<br>
|
|
If the source is redis cluster, redis-shake launches multiple goroutines for parallel pull. User can control the concurrency.<br>
|
|
The "move slot" operations must be disabled on the source side.<br>
|
|
|
|
# Code branch rules
|
|
---
|
|
Version rules: a.b.c.
|
|
|
|
* a: major version
|
|
* b: minor version. even number means stable version.
|
|
* c: bugfix version
|
|
|
|
| branch name | rules |
|
|
| - | :- |
|
|
| master | master branch, do not allowed push code. store the latest stable version. develop branch will merge into this branch once new version created.|
|
|
| **develop**(main branch) | develop branch. all the bellowing branches fork from this. |
|
|
| feature-\* | new feature branch. forked from develop branch and then merge back after finish developing, testing, and code review. |
|
|
| bugfix-\* | bugfix branch. forked from develop branch and then merge back after finish developing, testing, and code review. |
|
|
| improve-\* | improvement branch. forked from develop branch and then merge back after finish developing, testing, and code review. |
|
|
|
|
Tag rules:<br>
|
|
Add tag when releasing: "release-v{version}-{date}". for example: "release-v1.0.2-20180628"
|
|
|
|
# Usage
|
|
---
|
|
Run `./bin/redis-shake.darwin64` or `redis-shake.linux64` which is built in OSX and Linux respectively.<br>
|
|
Or you can build redis-shake yourself according to the following steps:
|
|
* git clone https://github.com/alibaba/RedisShake.git
|
|
* cd RedisShake
|
|
* export GOPATH=\`pwd\`/../..
|
|
* cd src/sync
|
|
* govendor sync #please note: must install govendor first and then pull all dependencies: `go get -u github.com/kardianos/govendor`
|
|
* cd ../../ && ./build.sh
|
|
* ./bin/redis-shake -type=$(type_must_be_sync_dump_restore_or_decode) -conf=conf/redis-shake.conf #please note: user must modify collector.conf first to match needs.
|
|
|
|
# Shake series tool
|
|
---
|
|
We also provide some tools for synchronization in Shake series.<br>
|
|
|
|
* [MongoShake](https://github.com/aliyun/MongoShake): mongodb data synchronization tool.
|
|
* [RedisShake](https://github.com/aliyun/RedisShake): redis data synchronization tool.
|
|
* [RedisFullCheck](https://github.com/aliyun/RedisFullCheck): redis data synchronization verification tool.
|
|
|
|
Plus, we have a WeChat group so that users can join and discuss, but the group user number is limited. So please add my WeChat number: `vinllen_xingge` first, and I will add you to this group.<br>
|
|
|
|
|