Install Redis Cluster
在 Ubuntu 上安裝 Redis cluster 過程紀錄.
setting Ubuntu
1sudo apt-get update
2
3# 北京時區+8
4sudo dpkg-reconfigure tzdata
配置 ulimit 來解除 Linux 系統的最大進程數和最大文件打開數限制, *代表針對所有用戶,noproc 是代表最大進程數,nofile 是代表最大文件打開數.
1# 編輯檔案 /etc/security/limits.conf
2
3* soft noproc 65535
4* hard noproc 65535
5* soft nofile 65535
6* hard nofile 65535
編輯檔案 /etc/pam.d/common-session
讓 Ubuntu 使用 pam_limits.so
模組.
1# required pam_limits.so at common-session file
2sudo vim /etc/pam.d/common-session
3
4session required pam_limits.so
1# required pam_limits.so at common-session-noninteractive file
2sudo vim /etc/pam.d/common-session-noninteractive
3
4session required pam_limits.so
開啟指定的 port , 讓 Redis cluster 可以做網路的溝通.
1# redis集群不僅需要開通redis客戶端連接的端口,而且需要開通集群總線端口
2# 集群總線端口為redis客戶端連接的端口+ 10000
3# 如redis端口為6379
4# 則集群總線端口為16379,故,所有服務器的點需要開通redis的客戶端連接端口和集群總線端口
5
6sudo ufw allow 6379
7sudo ufw allow 16379
8sudo ufw allow 6380
9sudo ufw allow 16380
10
11# 啟動防火牆
12sudo ufw enable
13
14# 刪除防火牆用
15# sudo ufw delete allow 6379,6380/tcp
重新啟動, 並確認上面的設定有再重開機後有啟用.
1sudo reboot
Install Redis cluster
方法一: 透過套件安裝工具安裝
1# 安裝套件
2sudo apt-get install redis-server redis-tools ruby
3
4# 確認版本
5redis-server --version
6Redis server v=3.0.6 sha=00000000:0 malloc=jemalloc-3.6.0 bits=64 build=687a2a319020fa42
7
8# 安裝完成後,應該就會看到process再跑
9sudo ss -ln | grep 6379
方法二: 下載原始碼編譯
1# install build base
2sudo apt-get update
3sudo apt-get install build-essential ruby
4
5wget http://download.redis.io/releases/redis-4.0.11.tar.gz
6tar xzf redis-4.0.11.tar.gz
7cd redis-4.0.11
8sudo make && make install
9
10cd utils
11sudo ./install_server.sh
setting Redis configuration
配置 Redis 服務, 有三台 host, 每台主機上有兩個 Redis 服務, 下面為一台主機的設定:
1# 先將服務停止
2sudo systemctl stop redis
1# 複製redis的設定檔案
2sudo cp /etc/redis/redis.conf /etc/redis/redis_6379.conf
3sudo cp /etc/redis/redis.conf /etc/redis/redis_6380.conf
1sudo vim /etc/redis/redis_6379.conf /etc/redis/redis_6380.conf
2# modified content
3pidfile /var/run/redis/redi_6379.pid # 依據master,slave起 redi_6379.pid,redi_6380.pid
4port 6379 # 依據master,slave起 6379,6380兩個port
5# bind 0.0.0.0 # 取消綁定網段
6protected-mode no
7
8logfile /var/log/redis/redis_6379.log # redis_6379.log,redis_6380.log
9dbfilename dump.rdb # 統一dump.rdb
10dir /www/redis_6379 # 變更目錄redis_6379,redis_6380
11
12appendonly yes # 啟用持久化
13
14cluster-enabled yes # 啟用Redis Cluster
15cluster-config-file nodes.conf # 指定Redis Cluster Config存放檔案,nodes_6379.conf,nodes_6380.conf
16cluster-node-timeout 5000 # Cluster Node TimeOut,超過視為Fail Node
17
18repl-diskless-sync no
19appendfsync no
20hash-max-ziplist-value 1024
1# 若有變更目錄記得建立,可以不用複製原目錄 sudo cp -r /var/lib/redis /www/
2sudo mkdir -p /www/redis_6379
3sudo chown -R redis:redis /www/redis_6379
4sudo chown redis:redis /etc/redis/redis_6379.conf
5
6sudo mkdir -p /www/redis_6380
7sudo chown -R redis:redis /www/redis_6380
8sudo chown redis:redis /etc/redis/redis_6380.conf
9
10# 關閉deamon
11sudo systemctl stop redis
12sudo systemctl disable redis
13# sudo systemctl start redis
14
15# 記得起動服務
16sudo redis-server /etc/redis/redis_6379.conf # 若systemctl遇到異常,採用systemctl啟動不了,採用此方式啟動
17sudo redis-server /etc/redis/redis_6380.conf
所有 host 都完成上述步驟之後, 在其中一部建立 Redis cluster
1sudo gem install redis
2sudo gem install redis -v 3.3.3 # 指定版本
1sudo find / -name redis-trib.rb
2
3# --replicas 1 表示每個Master帶有一個Slave
4sudo ./redis-trib.rb create --replicas 0 10.0.0.17:6379 10.0.0.11:6379 10.0.0.16:6379 # 這是Master,Master,Master的架構
5
6sudo ./redis-trib.rb create --replicas 1 10.0.0.5:6379 10.0.0.5:6380 10.0.0.11:6379 10.0.0.11:6380 10.0.0.16:6379 10.0.0.16:6380 # 這是Master,Slave,Master,Slave,Master,Slave的架構
7
8# 若被中斷,遇到ERR Slot 0 is already busy (Redis::CommandError),對所有node執行下面命令
9# 127.0.0.1:6379> FLUSHALL
10# 127.0.0.1:6380> CLUSTER RESET SOFT
11
12# 添加節點
13# ruby ./redis-trib.rb add-node 192.168.3.15:6390 192.168.3.12:6390
14# 分配槽空間,根據嚮導分配對應的槽給心得節點
15# ruby ./redis-trib.rb reshard 192.168.3.15:6390
16
17# 確認群集狀態
18redis-cli -c
19
20127.0.0.1:6379> CLUSTER NODES
21a1c717fe107 10.0.0.16:6380 slave fa1685991b3 0 1515476872739 6 connected
22fa1685991b3 10.0.0.16:6379 master - 0 1515476872238 5 connected 10923-16383
23e356bc7bc46 10.0.0.11:6379 master - 0 1515476873241 3 connected 5461-10922
24e3423dd62fa 10.0.0.17:6379 myself,master - 0 0 1 connected 0-5460
2563a93b44330 10.0.0.11:6380 slave e3423dd62fa 0 1515476873241 4 connected
26590885b4164 10.0.0.17:6380 slave e356bc7bc46 0 1515476873742 3 connected
1# automation start redis service at reboot
2sudo vim /etc/rc.local
3
4sudo redis-server /etc/redis/redis_6379.conf # append content
5sudo redis-server /etc/redis/redis_6380.conf # append content
執行測試
1redis-cli -c -p 6379
2127.0.0.1:6379> set foo bar
3OK
4127.0.0.1:6379> get goo
5-> Redirected to slot [6310] located at 10.0.0.11:6379
6(nil)
710.0.0.11:6379> get foo
8-> Redirected to slot [12182] located at 10.0.0.16:6379
9"bar"
1010.0.0.16:6379> set hello world
11-> Redirected to slot [866] located at 10.0.0.17:6379
12OK
1310.0.0.17:6379> get hello world
14(error) ERR wrong number of arguments for 'get' command
1510.0.0.17:6379> get hello
16"world"
1710.0.0.17:6379>
測試完畢後可以查閱 log 確認有沒有異常, 若看到警告 warning 可以做排除.
17439:M 19 Jun 14:11:51.373 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
2
3sudo echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf
4sudo sysctl vm.overcommit_memory=1
一主二從三衛兵
1sudo apt-get update
2
3//北京時區+8
4sudo dpkg-reconfigure tzdata
1sudo apt-get install redis-server redis-sentinel
2
3//安裝完成後,應該就會看到process再跑
4sudo ss -ln | grep 6379