flannel cni容器部署

发表于   |   更新于 | 分类于 | | 阅读次数: | 总览

flannel cni 配置

        flannel 以 DaemonSet 的形式运行在 Kubernetes 集群中。 由于我们的 etcd 集群启用了 TLS 认证，为了从 flannel 容器中能访问 etcd，我们先把 etcd 的 TLS 证书信息保存到 Kubernetes 的Secret 中。

        之前是吧flannel以服务的形式部署在node节点上，这种模式每次新增节点的时候都要去部署一次flannel，显得比较麻烦，以pod形式部署，每次新增节点后，kubernetes会自动的部署flannel。

        删除原有的网络组件，采用容器方式进行部署。

kube-controller-manager修改

        启动参数增加如下两项

1
2
--allocate-node-cidrs=true
--cluster-cidr=10.244.0.0/16

参数:

  • 1.是否应在云提供商上分配和设置Pod的CIDR
  • 2.集群中Pod的CIDR范围。要求–allocate-node-cidrs为true

kube-controller-manager参数详细说明

kubelet 修改

        kubelet修改两个配置文件，kubelet-config.yaml 文件已更新

kubelet.service启动文件

1
2
3
--network-plugin=cni
--cni-conf-dir=/etc/cni/net.d
--cni-bin-dir=/opt/cni/bin

kubele参数详细说明

kube-proxy 修改子网

  • kube-proxy-config.yaml
    1
    clusterCIDR: 10.244.0.0/16

        k8s集群安装文档已经更新，node节点安装文档已经更新。

        首先停止node 的flananel、 kubelet、kube-proxy。在删除/etc/cni/net.d/、/run/flannel/subnet.env、/var/lib/cni/flannel 的历史文件，重启网卡，除了flannel不启动，其他均启动。全新安装不需要这么操作，flannel证书也不需要复制。二进制安装参考

下载cni插件

        cni 插件下载，解压后，放在各个节点的 /opt/cni/bin 下。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
ls -l /opt/cni/bin/
total 70144
-rwxr-xr-x 1 root root  4159253 Nov 26 16:32 bandwidth
-rwxr-xr-x 1 root root  4628074 Nov 26 16:32 bridge
-rwxr-xr-x 1 root root 12124236 Nov 26 16:32 dhcp
-rwxr-xr-x 1 root root  5894275 Nov 26 16:32 firewall
-rwxr-xr-x 1 root root  3069556 Nov 26 16:32 flannel
-rwxr-xr-x 1 root root  4113837 Nov 26 16:32 host-device
-rwxr-xr-x 1 root root  3614305 Nov 26 16:32 host-local
-rwxr-xr-x 1 root root  4275320 Nov 26 16:32 ipvlan
-rwxr-xr-x 1 root root  3209373 Nov 26 16:32 loopback
-rwxr-xr-x 1 root root  4346248 Nov 26 16:32 macvlan
-rwxr-xr-x 1 root root  3895553 Nov 26 16:32 portmap
-rwxr-xr-x 1 root root  4546828 Nov 26 16:32 ptp
-rwxr-xr-x 1 root root  3392736 Nov 26 16:32 sbr
-rwxr-xr-x 1 root root  2885430 Nov 26 16:32 static
-rwxr-xr-x 1 root root  3356497 Nov 26 16:32 tuning
-rwxr-xr-x 1 root root  4275168 Nov 26 16:32 vlan

flannel 部署

1
2
3
wget https://raw.githubusercontent.com/coreos/flannel/v0.11.0/Documentation/kube-flannel.yml

kubectl apply -f ./kube-flannel.yml

查看flannel安装情况

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# kubectl get po,svc -o wide
NAME                 READY   STATUS    RESTARTS   AGE   IP           NODE            NOMINATED NODE   READINESS GATES
pod/nginx-ds-b9wgm   1/1     Running   0          43h   10.244.6.2   172.21.17.41    <none>           <none>
pod/nginx-ds-dd9mb   1/1     Running   0          43h   10.244.1.3   172.21.17.34    <none>           <none>
pod/nginx-ds-lcrn5   1/1     Running   0          43h   10.244.4.2   172.21.16.231   <none>           <none>
pod/nginx-ds-n52vr   1/1     Running   0          43h   10.244.2.2   172.21.17.40    <none>           <none>
pod/nginx-ds-twdxm   1/1     Running   0          43h   10.244.5.2   172.21.16.204   <none>           <none>
pod/nginx-ds-z72mx   1/1     Running   0          43h   10.244.3.2   172.21.17.38    <none>           <none>

NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE   SELECTOR
service/kubernetes   ClusterIP   10.254.0.1      <none>        443/TCP        44h   <none>
service/nginx-ds     NodePort    10.254.230.34   <none>        80:31286/TCP   43h   app=nginx-ds

# kubectl get po,svc -o wide -n kube-system
NAME                                              READY   STATUS    RESTARTS   AGE     IP              NODE            NOMINATED NODE   READINESS GATES
pod/coredns-5579b8778b-dlk4c                      1/1     Running   4          43h     10.244.1.2      172.21.17.34    <none>           <none>
pod/kube-flannel-ds-amd64-2t8rn                   1/1     Running   0          44h     172.21.17.41    172.21.17.41    <none>           <none>
pod/kube-flannel-ds-amd64-9sb8v                   1/1     Running   0          44h     172.21.16.231   172.21.16.231   <none>           <none>
pod/kube-flannel-ds-amd64-npl5k                   1/1     Running   0          44h     172.21.17.34    172.21.17.34    <none>           <none>
pod/kube-flannel-ds-amd64-r5n8r                   1/1     Running   0          44h     172.21.16.204   172.21.16.204   <none>           <none>
pod/kube-flannel-ds-amd64-wpfxw                   1/1     Running   0          44h     172.21.17.40    172.21.17.40    <none>           <none>
pod/kube-flannel-ds-amd64-x2v8z                   1/1     Running   0          44h     172.21.17.38    172.21.17.38    <none>           <none>
pod/kubernetes-dashboard-65dfbf6f4f-sc92h         1/1     Running   0          21h     10.244.6.10     172.21.17.41    <none>           <none>
pod/metrics-server-v0.3.4-5cd6d6b55f-psq2z        2/2     Running   0          3h20m   172.21.17.34    172.21.17.34    <none>           <none>
pod/traefik-ingress-controller-56b6d5b864-srnmm   1/1     Running   0          155m    172.21.17.41    172.21.17.41    <none>           <none>

NAME                              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                  AGE     SELECTOR
service/kube-dns                  ClusterIP   10.254.0.2       <none>        53/UDP,53/TCP,9153/TCP   43h     k8s-app=kube-dns
service/kubernetes-dashboard      NodePort    10.254.139.196   <none>        443:31417/TCP            21h     k8s-app=kubernetes-dashboard
service/metrics-server            ClusterIP   10.254.201.191   <none>        443/TCP                  3h56m   k8s-app=metrics-server
service/traefik-ingress-service   ClusterIP   None             <none>        80/TCP,8080/TCP          154m    k8s-app=traefik-ingress-lb
service/traefik-web-ui            ClusterIP   10.254.78.17     <none>        80/TCP                   23h     k8s-app=traefik-ingress-lb

查看节点的网络

  • node-01

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    # ifconfig
    cni0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
            inet 10.244.6.1  netmask 255.255.255.0  broadcast 10.244.6.255
            ether 42:56:73:73:c5:69  txqueuelen 1000  (Ethernet)
            RX packets 121976  bytes 48185929 (45.9 MiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 143012  bytes 112170228 (106.9 MiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            inet 172.21.17.41  netmask 255.255.240.0  broadcast 172.21.31.255
            ether fa:16:3e:a0:b1:af  txqueuelen 1000  (Ethernet)
            RX packets 1878425  bytes 997129342 (950.9 MiB)
            RX errors 0  dropped 29  overruns 0  frame 0
            TX packets 1014950  bytes 378435170 (360.9 MiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
            inet 10.244.6.0  netmask 255.255.255.255  broadcast 0.0.0.0
            ether 32:83:f7:e6:de:77  txqueuelen 0  (Ethernet)
            RX packets 207502  bytes 36217464 (34.5 MiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 242424  bytes 246992876 (235.5 MiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
            inet 127.0.0.1  netmask 255.0.0.0
            loop  txqueuelen 1  (Local Loopback)
            RX packets 15889  bytes 7196484 (6.8 MiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 15889  bytes 7196484 (6.8 MiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    vethb26f5960: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
            ether a6:a8:18:c3:f0:16  txqueuelen 0  (Ethernet)
            RX packets 23  bytes 2075 (2.0 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 49  bytes 4330 (4.2 KiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

  • node-02

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    # ifconfig
    cni0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
            inet 10.244.4.1  netmask 255.255.255.0  broadcast 10.244.4.255
            ether 82:14:33:5c:b0:b7  txqueuelen 1000  (Ethernet)
            RX packets 150228  bytes 747318135 (712.6 MiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 214837  bytes 209342880 (199.6 MiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            inet 172.21.16.231  netmask 255.255.240.0  broadcast 172.21.31.255
            ether fa:16:3e:d8:23:fe  txqueuelen 1000  (Ethernet)
            RX packets 10290258  bytes 3436274673 (3.2 GiB)
            RX errors 0  dropped 19  overruns 0  frame 0
            TX packets 9490095  bytes 3777352983 (3.5 GiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
            inet 10.244.4.0  netmask 255.255.255.255  broadcast 0.0.0.0
            ether ee:a3:ff:37:3a:61  txqueuelen 0  (Ethernet)
            RX packets 280851  bytes 31096456 (29.6 MiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 262931  bytes 968429571 (923.5 MiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
            inet 127.0.0.1  netmask 255.0.0.0
            loop  txqueuelen 1  (Local Loopback)
            RX packets 653134  bytes 200562497 (191.2 MiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 653134  bytes 200562497 (191.2 MiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    veth8aa15f82: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
            ether fa:23:1e:b0:65:20  txqueuelen 0  (Ethernet)
            RX packets 49191  bytes 4622232 (4.4 MiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 53647  bytes 19011709 (18.1 MiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
坚持原创技术分享，您的支持将鼓励我继续创作！
0%