Install Redis Cluster

在 Ubuntu 上安裝 Redis cluster 過程紀錄.

setting Ubuntu

1sudo apt-get update
2
3# 北京時區+8
4sudo dpkg-reconfigure tzdata

配置 ulimit 來解除 Linux 系統的最大進程數和最大文件打開數限制, *代表針對所有用戶,noproc 是代表最大進程數,nofile 是代表最大文件打開數.

1# 編輯檔案 /etc/security/limits.conf
2
3* soft noproc 65535
4* hard noproc 65535
5* soft nofile 65535
6* hard nofile 65535

編輯檔案 /etc/pam.d/common-session 讓 Ubuntu 使用 pam_limits.so 模組.

1# required pam_limits.so at common-session file
2sudo vim /etc/pam.d/common-session
3
4session required pam_limits.so
1# required pam_limits.so at common-session-noninteractive file
2sudo vim /etc/pam.d/common-session-noninteractive
3
4session required pam_limits.so

開啟指定的 port , 讓 Redis cluster 可以做網路的溝通.

 1# redis集群不僅需要開通redis客戶端連接的端口,而且需要開通集群總線端口
 2# 集群總線端口為redis客戶端連接的端口+ 10000
 3# 如redis端口為6379
 4# 則集群總線端口為16379,故,所有服務器的點需要開通redis的客戶端連接端口和集群總線端口
 5
 6sudo ufw allow 6379
 7sudo ufw allow 16379
 8sudo ufw allow 6380
 9sudo ufw allow 16380
10
11# 啟動防火牆
12sudo ufw enable
13
14# 刪除防火牆用
15# sudo ufw delete allow 6379,6380/tcp

重新啟動, 並確認上面的設定有再重開機後有啟用.

1sudo reboot

Install Redis cluster

方法一: 透過套件安裝工具安裝

1# 安裝套件
2sudo apt-get install redis-server redis-tools ruby
3
4# 確認版本
5redis-server --version
6Redis server v=3.0.6 sha=00000000:0 malloc=jemalloc-3.6.0 bits=64 build=687a2a319020fa42
7
8# 安裝完成後,應該就會看到process再跑
9sudo ss -ln | grep 6379

方法二: 下載原始碼編譯

 1# install build base
 2sudo apt-get update
 3sudo apt-get install build-essential ruby
 4
 5wget http://download.redis.io/releases/redis-4.0.11.tar.gz
 6tar xzf redis-4.0.11.tar.gz
 7cd redis-4.0.11
 8sudo make && make install
 9
10cd utils
11sudo ./install_server.sh

setting Redis configuration

配置 Redis 服務, 有三台 host, 每台主機上有兩個 Redis 服務, 下面為一台主機的設定:

1# 先將服務停止
2sudo systemctl stop redis
1# 複製redis的設定檔案
2sudo cp /etc/redis/redis.conf /etc/redis/redis_6379.conf
3sudo cp /etc/redis/redis.conf /etc/redis/redis_6380.conf
 1sudo vim /etc/redis/redis_6379.conf /etc/redis/redis_6380.conf
 2# modified content
 3pidfile /var/run/redis/redi_6379.pid # 依據master,slave起 redi_6379.pid,redi_6380.pid
 4port 6379 # 依據master,slave起 6379,6380兩個port
 5# bind 0.0.0.0 # 取消綁定網段
 6protected-mode no
 7
 8logfile /var/log/redis/redis_6379.log # redis_6379.log,redis_6380.log
 9dbfilename dump.rdb # 統一dump.rdb
10dir /www/redis_6379 # 變更目錄redis_6379,redis_6380
11
12appendonly yes # 啟用持久化
13
14cluster-enabled yes # 啟用Redis Cluster
15cluster-config-file nodes.conf # 指定Redis Cluster Config存放檔案,nodes_6379.conf,nodes_6380.conf
16cluster-node-timeout 5000 # Cluster Node TimeOut,超過視為Fail Node
17
18repl-diskless-sync no
19appendfsync no
20hash-max-ziplist-value 1024
 1# 若有變更目錄記得建立,可以不用複製原目錄 sudo cp -r /var/lib/redis /www/
 2sudo mkdir -p /www/redis_6379
 3sudo chown -R redis:redis /www/redis_6379
 4sudo chown redis:redis /etc/redis/redis_6379.conf
 5
 6sudo mkdir -p /www/redis_6380
 7sudo chown -R redis:redis /www/redis_6380
 8sudo chown redis:redis /etc/redis/redis_6380.conf
 9
10# 關閉deamon
11sudo systemctl stop redis
12sudo systemctl disable redis
13# sudo systemctl start redis
14
15# 記得起動服務
16sudo redis-server /etc/redis/redis_6379.conf # 若systemctl遇到異常,採用systemctl啟動不了,採用此方式啟動
17sudo redis-server /etc/redis/redis_6380.conf

所有 host 都完成上述步驟之後, 在其中一部建立 Redis cluster

1sudo gem install redis
2sudo gem install redis -v 3.3.3 # 指定版本
 1sudo find / -name redis-trib.rb
 2
 3# --replicas 1  表示每個Master帶有一個Slave
 4sudo ./redis-trib.rb create --replicas 0 10.0.0.17:6379 10.0.0.11:6379 10.0.0.16:6379 # 這是Master,Master,Master的架構
 5
 6sudo ./redis-trib.rb create --replicas 1 10.0.0.5:6379 10.0.0.5:6380 10.0.0.11:6379 10.0.0.11:6380 10.0.0.16:6379 10.0.0.16:6380 # 這是Master,Slave,Master,Slave,Master,Slave的架構
 7
 8# 若被中斷,遇到ERR Slot 0 is already busy (Redis::CommandError),對所有node執行下面命令
 9# 127.0.0.1:6379> FLUSHALL
10# 127.0.0.1:6380> CLUSTER RESET SOFT
11
12# 添加節點
13# ruby ./redis-trib.rb add-node 192.168.3.15:6390 192.168.3.12:6390
14# 分配槽空間,根據嚮導分配對應的槽給心得節點
15# ruby ./redis-trib.rb reshard 192.168.3.15:6390
16
17# 確認群集狀態
18redis-cli -c
19
20127.0.0.1:6379> CLUSTER NODES
21a1c717fe107 10.0.0.16:6380 slave fa1685991b3 0 1515476872739 6 connected
22fa1685991b3 10.0.0.16:6379 master - 0 1515476872238 5 connected 10923-16383
23e356bc7bc46 10.0.0.11:6379 master - 0 1515476873241 3 connected 5461-10922
24e3423dd62fa 10.0.0.17:6379 myself,master - 0 0 1 connected 0-5460
2563a93b44330 10.0.0.11:6380 slave e3423dd62fa 0 1515476873241 4 connected
26590885b4164 10.0.0.17:6380 slave e356bc7bc46 0 1515476873742 3 connected
1# automation start redis service at reboot
2sudo vim /etc/rc.local
3
4sudo redis-server /etc/redis/redis_6379.conf # append content
5sudo redis-server /etc/redis/redis_6380.conf # append content

執行測試

 1redis-cli -c -p 6379
 2127.0.0.1:6379> set foo bar
 3OK
 4127.0.0.1:6379> get goo
 5-> Redirected to slot [6310] located at 10.0.0.11:6379
 6(nil)
 710.0.0.11:6379> get foo
 8-> Redirected to slot [12182] located at 10.0.0.16:6379
 9"bar"
1010.0.0.16:6379> set hello world
11-> Redirected to slot [866] located at 10.0.0.17:6379
12OK
1310.0.0.17:6379> get hello world
14(error) ERR wrong number of arguments for 'get' command
1510.0.0.17:6379> get hello
16"world"
1710.0.0.17:6379>

測試完畢後可以查閱 log 確認有沒有異常, 若看到警告 warning 可以做排除.

17439:M 19 Jun 14:11:51.373 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
2
3sudo echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf
4sudo sysctl vm.overcommit_memory=1

一主二從三衛兵

1sudo apt-get update
2
3//北京時區+8
4sudo dpkg-reconfigure tzdata
1sudo apt-get install redis-server redis-sentinel
2
3//安裝完成後,應該就會看到process再跑
4sudo ss -ln | grep 6379
comments powered by Disqus