内核配置（Optional）

为了集群的稳定性和兼容性，生产环境的内核最好升级到 4.18 版本以上，本示例将升级到 4.19 版本。

1. 下载离线包

cd /root
wget http://193.49.22.109/elrepo/kernel/el7/x86_64/RPMS/kernel-ml-4.19.12-1.el7.elrepo.x86_64.rpm
wget http://193.49.22.109/elrepo/kernel/el7/x86_64/RPMS/kernel-ml-devel-4.19.12-1.el7.elrepo.x86_64.rpm

2. 安装内核

yum localinstall -y kernel-ml*

3. 更改内核启动顺序

grub2-set-default 0 && grub2-mkconfig -o /etc/grub2.cfg
grubby --args="user_namespace.enable=1" --update-kernel="$(grubby --default-kernel)"

检查默认内核是不是 4.19：

# grubby --default-kernel
/boot/vmlinuz-4.19.12-1.el7.elrepo.x86_64

节点重启，然后检查内核是不是 4.19：

# uname -a
Linux k8s-master01 4.19.12-1.el7.elrepo.x86_64 #1 SMP Fri Dec 21 11:06:36 EST 2018 x86_64 x86_64 x86_64 GNU/Linux

4. 安装 ipvsadm 和 ipset

yum install ipvsadm ipset sysstat conntrack libseccomp -y

所有节点配置 ipvs 模块，在内核 4.19+ 版本 nf_conntrack_ipv4 已经改为 nf_conntrack，4.18 以下版本使用 nf_conntrack_ipv4 即可：

echo 'ip_vs
ip_vs_lc
ip_vs_wlc
ip_vs_rr
ip_vs_wrr
ip_vs_lblc
ip_vs_lblcr
ip_vs_dh
ip_vs_sh
ip_vs_fo
ip_vs_nq
ip_vs_sed
ip_vs_ftp
ip_vs_sh
nf_conntrack # 4.18改为nf_conntrack_ipv4
ip_tables
ip_set
xt_set
ipt_set
ipt_rpfilter
ipt_REJECT
ipip' >> /etc/modules-load.d/ipvs.conf

cat /etc/modules-load.d/ipvs.conf
systemctl enable --now systemd-modules-load.service

5. 修改内核参数

开启一些 K8s 集群中必需的内核参数：

echo 'net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
fs.may_detach_mounts = 1
net.ipv4.conf.all.route_localnet = 1
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_watches=89100
fs.file-max=52706963
fs.nr_open=52706963
net.netfilter.nf_conntrack_max=2310720

net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.tcp_keepalive_intvl =15
net.ipv4.tcp_max_tw_buckets = 36000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_max_orphans = 327680
net.ipv4.tcp_orphan_retries = 3
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.ip_conntrack_max = 65536
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_timestamps = 0
net.core.somaxconn = 16384' > /etc/sysctl.d/k8s.conf

cat /etc/sysctl.d/k8s.conf
sysctl --system

配置完内核后，重启服务器，保证重启后内核依旧加载：

lsmod | grep --color=auto -e ip_vs -e nf_conntrack