Contents
环境说明
系统 | 主机名 | ip | redis版本 |
---|---|---|---|
Ubuntu 18.04.4 LTS | redis01 | 172.40.11.18,172.40.11.21(vip) | redis5.0.5 |
Ubuntu 18.04.4 LTS | redis02 | 172.40.11.19 | redis5.0.5 |
Ubuntu 18.04.4 LTS | redis03 | 172.40.11.20 | redis5.0.5 |
1 安装redis服务
1.1 配置基础环境
apt install make
apt install make-guile
apt install gcc
1.2 优化操作系统
关闭内存大页
echo never |sudo tee /sys/kernel/mm/transparent_hugepage/defrag
echo never |sudo tee /sys/kernel/mm/transparent_hugepage/enabled
sed -i '/^exit/i\echo never > /sys/kernel/mm/transparent_hugepage/defrag' /etc/rc.local
sed -i '/^exit/i\echo never > /sys/kernel/mm/transparent_hugepage/enabled' /etc/rc.local
最大打开文件数
ulimit -SHn 65535
echo "ulimit -SHn 65535" >> /etc/rc.local
echo "* soft nofile 65535" >>/etc/security/limits.conf
echo "* hard nofile 65535" >>/etc/security/limits.conf
echo "* soft nproc 65535" >>/etc/security/limits.conf
echo "* hard nproc 65535" >>/etc/security/limits.conf
TCP积压值过小优化
echo "net.core.somaxconn = 1024" > /etc/sysctl.d/redis.conf
sysctl -p /etc/sysctl.d/redis.conf #刷新使其生效
net.core.somaxconn = 1024
内存使用策略优化
echo "vm.overcommit_memory = 1" >> /etc/sysctl.d/redis.conf
sysctl -p /etc/sysctl.d/redis.conf #刷新使其生效
vm.overcommit_memory = 1
1.3 编译安装redis
wget http://download.redis.io/releases/redis-5.0.5.tar.gz
tar zxf redis-5.0.5.tar.gz
cd redis-5.0.5/
make && make install PREFIX=/usr/local/redis
1.4 配置redis
[root@redis redis-5.0.5]# mkdir -p /usr/local/redis/{bin,conf,data} #创建目录
[root@redis redis-5.0.5]# cp src/redis* /usr/local/redis/bin/ #拷贝命令
[root@redis redis-5.0.5]# cp redis.conf /usr/local/redis/conf/ # 拷贝配置文件
[root@redis redis-5.0.5]# cd /usr/local/redis/bin/ #进入命令目录下,删除不必要的文件
[root@redis bin]# rm -rf *.c
[root@redis bin]# rm -rf *.h
[root@redis bin]# rm -rf *.o
[root@redis bin]# cd ../conf/
[root@redis conf]# cp redis.conf{,.bak} #备份配置文件
[root@redis conf]# egrep -v '^$|^#' redis.conf.bak > redis.conf
#过滤注释及空行以便生成新的配置文件
bind 0.0.0.0
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
pidfile /var/run/redis_6379.pid
loglevel notice
logfile "/usr/local/redis/logs/redis.log"
databases 16
requirepass "111111"
always-show-logo yes
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir /usr/local/redis/data
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
replica-priority 100
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
stream-node-max-bytes 4096
stream-node-max-entries 100
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
dynamic-hz yes
aof-rewrite-incremental-fsync yes
rdb-save-incremental-fsync yes
1.5 创建redis启动脚本
vim /etc/systemd/system/redis.service
[Unit]
Description=redis-server
After=network.target
[Service]
Type=forking
ExecStart=/usr/local/redis/bin/redis-server /usr/local/redis/conf/redis.conf
PrivateTmp=true
[Install]
WantedBy=multi-user.target
启动redis
systemctl daemon-reload
systemctl start redis.service
2 配置redis主从
主从复制的安全性
对于数据比较重要的节点,主节点会通过设置requirepass参数进行设置密码,这时,所有的client访问都需要使用auth进行认证,因此,在配置主从时,需要配置slave节点的masterauth参数与master的requirepass参数一致,这样从节点才可以正确的连接到主节点并发起复制流程。
slave设置只读
默认情况下,slave使用slave-read-only=yes配置为只读模式。由于复制只能从master到slave,对于slave的任何修改master都无法感知,修改slave节点数据会造成主从数据不一致。因此建议线上不要修改从节点的只读模式。
在从的redis.conf中添加如下
replicaof 172.40.11.18 6379 #启动主从复制
requirepass "111111"
masterauth "111111"
slave-read-only yes
查看主从信息
127.0.0.1:6379> KEYS *
(error) NOAUTH Authentication required.
127.0.0.1:6379> auth 111111
OK
127.0.0.1:6379> INFO replication
# Replication
role:slave
master_host:172.40.11.18
master_port:6379
master_link_status:up
master_last_io_seconds_ago:5
master_sync_in_progress:0
slave_repl_offset:308
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:4349917446005cf2e4eff8dd06821bbfbcbcf7c1
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:308
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:127
repl_backlog_histlen:182
主从手动切换
先在slave执行如下命令
SLAVEOF NO ONE
然后在slave执行如下命令
slaveof 172.40.11.19 6379 #指定新的主
在原来的主上也执行这个命令
3 配置Sentinel哨兵模式
Redis的哨兵机制存在的意义就是当主从架构中,master发生宕机,无需人工干预,自动实现故障转移。 Redis的Sentinel系统用于管理多个Redis示例,该系统执行以下三个任务: 1.监控(Monitoring): Sentinel 会不断地检查你的master和slave是否运作正常。 2.提醒(Notification): 当被监控的某个 Redis 服务器出现问题时, Sentinel 可以通过 API 向管理员或者其他应用程序发送通知。 3.自动故障迁移(Automatic failover): 当一个主服务器不能正常工作时, Sentinel 会开始一次自动故障迁移操作, 它会将失效master的其中一个slave升级为新的master,并让失效master的其他slave改为复制新的master;当客户端试图连接失效的master时,集群也会向客户端返回新master的地址, 使得集群可以使用新master代替失效服务器。
配置Sentinel哨兵模式
egrep -v "^#|^$" sentinel.conf.bak > sentinel.conf
bind 0.0.0.0
port 26379
daemonize yes
pidfile /var/run/redis-sentinel.pid
logfile "/usr/local/redis/logs/redis-sentinel.log"
dir /usr/local/redis/data/
sentinel monitor mymaster 172.40.11.18 6379 2
sentinel auth-pass mymaster "111111"
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000
sentinel deny-scripts-reconfig yes
参数说明
配置文件修改说明:
//端口默认为26379
port:26379
//设置为后台启动
daemonize:yes
//关闭保护模式,可以外部访问
protected-mode:no
//PID文件路径
pidfile "/var/run/redis-sentinel.pid"
//日志文件路径
logfile "/usr/local/redis/logs/sentinel.log"
//数据存放路径
dir "/usr/local/redis/data"
//指定主机IP地址和端口,并且指定当有2台哨兵认为主机挂了,则对主机进行容灾切换
sentinel monitor mymaster 172.30.93.122 6379 2
//redis主从密码,这里也需要设置
# sentinel auth-pass mymaster redis2021
//这里设置了主机多少秒无响应,则认为挂了
sentinel down-after-milliseconds mymaster 3000
//主备切换时,最多有多少个slave同时对新的master进行同步,这里设置为默认的1
snetinel parallel-syncs mymaster 1
//故障转移的超时时间,这里设置为三分钟
sentinel failover-timeout mymaster 180000
//避免了一个简单的安全问题,客户端可以将脚本设置为任何内容并触发故障转移以便执行程序
sentinel deny-scripts-reconfig yes
配置启动文件
cat /etc/systemd/system/redis-sentinel.service
[Unit]
Description=Redis sentinel Server
After=network.target
[Service]
Type=forking
PIDFile=/var/run/redis-sentinel.pid
ExecStart=/usr/local/redis/bin/redis-sentinel /usr/local/redis/conf/sentinel.conf --sentinel
ExecStop=/usr/local/redis/bin/redis-cli -p 26379 shutdown
Restart=on-failure
[Install]
WantedBy=multi-user.target
启动
systemctl daemon-reload
systemctl start redis-sentinel.service
查看进程
root@gpu04:~# netstat -tnlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:8130 0.0.0.0:* LISTEN 1840/sshd
tcp 0 0 0.0.0.0:26379 0.0.0.0:* LISTEN 4276/redis-sentinel
tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 3861/redis-server 0
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 1348/systemd-resolv
tcp6 0 0 :::8130 :::* LISTEN 1840/sshd
tcp6 0 0 :::26379 :::* LISTEN 4276/redis-sentinel
查看sentinel启动后配置文件的变化
port 26379
daemonize yes
pidfile "/var/run/redis-sentinel.pid"
logfile "/data/dev/sdb1/redis/logs/redis-sentinel.log"
dir "/data/dev/sdb1/redis/data"
sentinel myid 48f805205d4aa11f890b0c6c6d4d417e16d03aff
sentinel deny-scripts-reconfig yes
sentinel monitor mymaster 172.40.11.18 6379 2
sentinel auth-pass mymaster 5c32WQaCbt77
sentinel config-epoch mymaster 0
sentinel leader-epoch mymaster 0
# Generated by CONFIG REWRITE
protected-mode no
sentinel known-replica mymaster 172.40.11.19 6379
sentinel known-replica mymaster 172.40.11.20 6379
sentinel known-sentinel mymaster 172.40.11.19 26379 34092a0b7862caf72345d160d3304311636df390
sentinel known-sentinel mymaster 172.40.11.18 26379 1f8f6e1690e69955ff19c582c50e135db0e2b3a8
sentinel current-epoch 0
登录到哨兵监听的26379端口
redis-cli -p 26379
127.0.0.1:26379> sentinel master mymaster # 查看mymaster的主节点状态以及相关的统计信息
1) "name"
2) "mymaster"
3) "ip"
4) "172.40.11.18"
5) "port"
6) "6379"
7) "runid"
8) "54c54b520fdd61e03eb72af9b322fc5370cf86df"
9) "flags"
10) "master"
11) "link-pending-commands"
12) "0"
13) "link-refcount"
14) "1"
15) "last-ping-sent"
16) "0"
17) "last-ok-ping-reply"
18) "380"
19) "last-ping-reply"
20) "380"
21) "down-after-milliseconds"
22) "30000"
23) "info-refresh"
24) "957"
25) "role-reported"
26) "master"
27) "role-reported-time"
28) "292118"
29) "config-epoch"
30) "0"
31) "num-slaves"
32) "2"
33) "num-other-sentinels"
34) "2"
35) "quorum"
36) "2"
37) "failover-timeout"
38) "180000"
39) "parallel-syncs"
40) "1"
验证故障转移效果
systemctl stop redis.service(master)
查看日志
tail -1000f /usr/local/redis/logs/redis-sentinel.log
5169:X 21 May 2021 14:40:47.131 # +sdown master mymaster 172.40.11.18 6379
5169:X 21 May 2021 14:40:47.197 # +odown master mymaster 172.40.11.18 6379 #quorum 2/2
5169:X 21 May 2021 14:40:47.197 # +new-epoch 1
5169:X 21 May 2021 14:40:47.197 # +try-failover master mymaster 172.40.11.18 6379
5169:X 21 May 2021 14:40:47.198 # +vote-for-leader 1f8f6e1690e69955ff19c582c50e135db0e2b3a8 1
5169:X 21 May 2021 14:40:47.207 # 34092a0b7862caf72345d160d3304311636df390 voted for 1f8f6e1690e69955ff19c582c50e135db0e2b3a8 1
5169:X 21 May 2021 14:40:47.208 # 48f805205d4aa11f890b0c6c6d4d417e16d03aff voted for 1f8f6e1690e69955ff19c582c50e135db0e2b3a8 1
5169:X 21 May 2021 14:40:47.253 # +elected-leader master mymaster 172.40.11.18 6379
5169:X 21 May 2021 14:40:47.253 # +failover-state-select-slave master mymaster 172.40.11.18 6379
5169:X 21 May 2021 14:40:47.353 # +selected-slave slave 172.40.11.20:6379 172.40.11.20 6379 @ mymaster 172.40.11.18 6379
5169:X 21 May 2021 14:40:47.353 * +failover-state-send-slaveof-noone slave 172.40.11.20:6379 172.40.11.20 6379 @ mymaster 172.40.11.18 6379
5169:X 21 May 2021 14:40:47.412 * +failover-state-wait-promotion slave 172.40.11.20:6379 172.40.11.20 6379 @ mymaster 172.40.11.18 6379
5169:X 21 May 2021 14:40:47.414 # +promoted-slave slave 172.40.11.20:6379 172.40.11.20 6379 @ mymaster 172.40.11.18 6379
5169:X 21 May 2021 14:40:47.414 # +failover-state-reconf-slaves master mymaster 172.40.11.18 6379
5169:X 21 May 2021 14:40:47.488 * +slave-reconf-sent slave 172.40.11.19:6379 172.40.11.19 6379 @ mymaster 172.40.11.18 6379
5169:X 21 May 2021 14:40:48.283 # -odown master mymaster 172.40.11.18 6379
5169:X 21 May 2021 14:40:48.446 * +slave-reconf-inprog slave 172.40.11.19:6379 172.40.11.19 6379 @ mymaster 172.40.11.18 6379
5169:X 21 May 2021 14:40:48.446 * +slave-reconf-done slave 172.40.11.19:6379 172.40.11.19 6379 @ mymaster 172.40.11.18 6379
5169:X 21 May 2021 14:40:48.508 # +failover-end master mymaster 172.40.11.18 6379
5169:X 21 May 2021 14:40:48.508 # +switch-master mymaster 172.40.11.18 6379 172.40.11.20 6379
5169:X 21 May 2021 14:40:48.508 * +slave slave 172.40.11.19:6379 172.40.11.19 6379 @ mymaster 172.40.11.20 6379
5169:X 21 May 2021 14:40:48.508 * +slave slave 172.40.11.18:6379 172.40.11.18 6379 @ mymaster 172.40.11.20 6379
5169:X 21 May 2021 14:41:18.542 # +sdown slave 172.40.11.18:6379 172.40.11.18 6379 @ mymaster 172.40.11.20 6379
查看redis
redis-cli
127.0.0.1:6379> AUTH 111111
OK
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:172.40.11.20
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:179416
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:c86766de43f666ce09c0786726f18129f4f05abe
master_replid2:2e991169f1d5ed75dadbabcf12b3bb73e817a779
master_repl_offset:179416
second_repl_offset:79893
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:179416
查看Sentinel配置
port 26379
daemonize yes
pidfile "/var/run/redis-sentinel.pid"
logfile "/usr/local/redis/logs/redis-sentinel.log"
dir "/usr/local/redis/data"
sentinel myid 1f8f6e1690e69955ff19c582c50e135db0e2b3a8
sentinel deny-scripts-reconfig yes
sentinel monitor mymaster 172.40.11.20 26379 2
sentinel auth-pass mymaster 5c32WQaCbt77
sentinel config-epoch mymaster 1
sentinel leader-epoch mymaster 1
# Generated by CONFIG REWRITE
protected-mode no
sentinel known-replica mymaster 172.40.11.19 6379
sentinel known-replica mymaster 172.40.11.18 6379
sentinel known-sentinel mymaster 172.40.11.19 26379 34092a0b7862caf72345d160d3304311636df390
sentinel known-sentinel mymaster 172.40.11.20 26379 48f805205d4aa11f890b0c6c6d4d417e16d03aff
sentinel current-epoch 1
由上可见
sentinel模式已生效
sentinel维护命令
sentinel master mymaster # 查看mymaster的主节点状态以及相关的统计信息
127.0.0.1:26379> sentinel slaves mymaster # 查看指定的从节点状态及相关统计信息
127.0.0.1:26379> sentinel get-master-addr-by-name mymaster # 返回主节点的IP和端口
127.0.0.1:26379> sentinel failover mymaster # 对指定进行强制故障转移(他会将master角色自动转移到当前任意一个slave,没有和其他sentinel节点协商),当故障转移完成之后,其他的sentinel节点按照故障转移的结果更新自身配置。
127.0.0.1:26379> sentinel ckquorum mymaster # 检测当前主节点的哨兵是否到达quorum的个数。
127.0.0.1:26379> sentinel flushconfig # 将sentinel节点的配置信息强制写道磁盘上
127.0.0.1:26379> sentinel remove mymaster # 取消当前sentinel节点对于指定主节点的监控
4 redis漂移VIP
这里可以使用redis sentinel的一个参数client-reconfig-script,这个参数配置执行脚本,sentinel在做failover的时候会执行这个脚本,并且传递6个参数<master-name>、 <role>、 <state>、 <from-ip>、 <from-port>、 <to-ip> 、<to-port>,其中<to-ip>是新主redis的IP地址,可以在这个脚本里做VIP漂移操作。
sentinel client-reconfig-script mymaster /opt/redis/failover.sh
脚本内容
#########################################################################
# File Name: /opt/redis/failover.sh
#########################################################################
# 一定要用root用户运行
#!/bin/bash
MY_IP='172.40.11.18' # 每个Server本身的IP
VIP='172.40.11.21' # VIP
NETMASK='24' # Netmask
INTERFACE='bond0' # 接口
while true;do
MASTER_IP=$(redis-cli -h 127.0.0.1 -p 26379 info sentinel | grep address | cut -d, -f3 | cut -d= -f2 | cut -d: -f1)
if [ ${MASTER_IP} = ${MY_IP} ]; then
ip -4 addr | grep $VIP || (/sbin/ip addr add ${VIP}/${NETMASK} dev ${INTERFACE};/sbin/arping -q -c 3 -A ${VIP} -I ${INTERFACE})
else
ip -4 addr | grep $VIP && /sbin/ip addr del ${VIP}/${NETMASK} dev ${INTERFACE}
fi
sleep 10;
done
添加执行权限chmod 755 /opt/redis/failover.sh
需要注意的是,当你第一次使用脚本的时候,要在主节点上手动创建VIP
ip addr add 172.40.11.21/24 dev bond0
arping -q -c 3 -A 172.40.11.21 -I em1
总结:
使用redis主从 + 哨兵(sentinel)+ 漂移VIP的方案搭建了一个redis高可用系统,但这个系统保证的是单个redis实例的高可用,所以适合业务比较小的应用。如果业务比较大,并发量比较高,建议搭建redis集群,比如官方redis cluster,还有开源的codings集群。
5 redis持久化
Redis 4.0 开始支持 RDB 和 AOF 的混合持久化(默认关闭,可以通过配置项 aof-use-rdb-preamble 开启)。
如果把混合持久化打开,AOF 重写的时候就直接把 RDB 的内容写到 AOF 文件开头。这样做的好处是可以结合 RDB 和 AOF 的优点, 快速加载同时避免丢失过多的数据。当然缺点也是有的, AOF 里面的 RDB 部分就是压缩格式不再是 AOF 格式,可读性较差。
- 我的微信
- 这是我的微信扫一扫
-
- 我的微信公众号
- 我的微信公众号扫一扫
-