【Kubernetes存储篇】StorageClass存储类动态生成PV详解

2023-12-07 10:49 由だā简ゑ箪ャ发表于 #其他

一、StorageClass存储类理论
StorageClass的作用主要有以下几个方面：

动态存储卷分配：StorageClass可以根据定义的属性动态地创建存储卷，无需手动创建和管理存储卷。
存储卷的属性管理：StorageClass可以定义存储卷的属性，如存储类型、存储容量、访问模式等，从而更好地满足应用程序的存储需求。
存储资源的管理：StorageClass可以将存储资源进行分类管理，方便开发者根据应用程序的需求进行选择。
每个 StorageClass 都有一个供应商（Provisioner），用来决定使用哪个卷插件制备 PV。该字段必须指定，官网提供供应商如下表：

本文章以NFS为例，要想使用NFS，我们需要一个nfs-client的自动装载程序，称之为provisioner，这个程序会使用我们已经配置好的NFS服务器自动创建持久卷，也就是自动帮我们创建PV。

二、案例：Storageclass存储类实战演示
1、搭建NFS服务端
注意：K8S集群所有Node节点都需要安装 nfs-utils 包

yum -y install nfs-utils
mkdir /data/nfs_pro -p
vim /etc/exports
/data/test  *(rw,sync,no_root_squash)

加载生效 && 启动NFS服务

exportfs -arv
systemctl enable nfs --now

2、搭建NFS供应商(provisioner)

第一步：创建运行nfs-provisioner需要使用的SA账号

[root@Master testv2]# more 00-nfs-provisioner.yaml 
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-nginx

执行 YAML 文件 && 查看创建的 SA 账号：

kubectl apply -f 00-nfs-provisioner.yaml
kubectl get sa nfs-nginx

kubectl get sa nfs-nginx
NAME        SECRETS   AGE
nfs-nginx   1         18h

第二步：针对SA账号进行授权：

kubectl create clusterrolebinding nfs-nginx --clusterrole=cluster-admin --serviceaccount=default:nfs-nginx

第三步：安装01-nfs-deployment.yaml程序 YAML 如下：

cat 01-nfs-deployment.yaml 
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nfs-nginx
spec:
  selector:
    matchLabels:
       app: nfs-nginx
  replicas: 3
  strategy:        # 更新策略
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-nginx
    spec:
      serviceAccount: nfs-nginx   # 指定SA账号
      containers:
        - name: nfs-nginx
          image: registry.cn-beijing.aliyuncs.com/mydlq/nfs-subdir-external-provisioner:v4.0.0
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: example.com/nfs-nginx     # NFS供应商名称
            - name: NFS_SERVER
              value: 172.16.42.5         # NFS服务端地址
            - name: NFS_PATH            
              value: /data/test/      # NFS共享目录
      volumes:
        - name: nfs-client-root
          nfs:
            server: 172.16.42.5          # NFS服务端地址
            path: /data/test/          # NFS共享目录

执行YAML 文件 && 查看 Pod状态：

kubectl apply -f  01-nfs-deployment.yaml 
deployment.apps/nfs-nginx unchanged

kubectl get pods 
NAME                        READY   STATUS    RESTARTS   AGE
nfs-nginx-97f96446d-6ng62   1/1     Running   0          3s
nfs-nginx-97f96446d-9kf6x   1/1     Running   0          3s
nfs-nginx-97f96446d-cbbz9   1/1     Running   0          3s

3、创建StorageClass存储类

cat 02-nfs-storageclass.yaml 
---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: nfs-nginx
provisioner: example.com/nfs-nginx   # 指定NFS供应商名称,和上面对应上

注意：provisioner处写的 example.com/nfs应该跟安装01-nfs-deployment.yaml时候的env下的PROVISIONER_NAME的value值保持一致。

执行YAML文件 && 查看storageclass 状态：

kubectl apply -f 02-nfs-storageclass.yaml 
kubectl get sc nfs-nginx

kubectl get sc nfs-nginx
NAME        PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-nginx   example.com/nfs-nginx   Delete          Immediate           false                  18h

4、创建PVC，通过StorageClass动态生成PV

cat 03-nfs-pvc.yaml 
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: nfs-nginx
spec:
  accessModes:  ["ReadWriteMany"]
  resources:
    requests:
      storage: 6Gi
  storageClassName:  nfs-nginx   # 指定使用storageclass的名称,来自动生产PV

执行YAML 文件 && 查看是否自动生成PV

kubectl apply -f 03-nfs-pvc.yaml
kubectl get pvc

[root@Master testv2]# kubectl apply -f 03-nfs-pvc.yaml 
persistentvolumeclaim/nfs-nginx unchanged
[root@Master testv2]# kubectl get pvc nfs-nginx
NAME        STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
nfs-nginx   Bound    pvc-770e8d47-5cbf-4115-87ea-bbcc8551b993   6Gi        RWX            nfs-nginx      18h

如上图已经自动创建PV，并绑定上PVC了

5、创建Pod挂载PVC

 cat 04-nfs-pod-demo.yaml 
---
apiVersion: v1
kind: Pod
metadata:
  name: nfs-nginx
  labels:
    type: nfs-v1
spec:
  volumes:
  - persistentVolumeClaim: 
      claimName: nfs-nginx   # 指定PVC
    name: nfs-nginx                   # 卷名称
  containers:
  - name: nfs-nginx-demo
    image: nginx:latest
    imagePullPolicy: IfNotPresent
    volumeMounts:
    - name: nfs-nginx                # 指定上面卷名称
      mountPath: /usr/share/nginx/html  # 容器挂载目录

执行 YAML 文件 && 查看Pod状态：

kubectl apply -f 04-nfs-pod-demo.yaml
ubectl get pods nfs-nginx -o wide --show-labels

[root@Master testv2]# kubectl apply -f 04-nfs-pod-demo.yaml 
pod/nfs-nginx configured
[root@Master testv2]# kubectl get pods nfs-nginx -o wide --show-labels
NAME        READY   STATUS    RESTARTS   AGE   IP            NODE    NOMINATED NODE   READINESS GATES   LABELS
nfs-nginx   1/1     Running   0          18h   10.244.1.99   node1   <none>           <none>            type=nfs-v1

在 PVC 绑定宿主机目录，创建 index.html 文件

echo "storageclass demo successd...." > /data/test/default-storageclass-pvc-demo-pvc-6b213e25-12a2-4ffd-a1ff-af23ec531233/index.html

获取Pod IP访问网站：

kubectl get pods nfs-nginx -o wide

[root@Master testv2]# kubectl get pods nfs-nginx -o wide
NAME        READY   STATUS    RESTARTS   AGE   IP            NODE    NOMINATED NODE   READINESS GATES
nfs-nginx   1/1     Running   0          18h   10.244.1.99   node1   <none>           <none>
[root@Master testv2]# curl 10.244.1.99
storageclass demo successd....

三、步骤总结
1、搭建NFS服务端

2、搭建NFS供应商，指定NFS服务端IP地址及共享目录

3、创建StorageClass资源，指定使用NFS供应商

4、创建PVC，使用storageClassName 自动指定使用StorageClass

5、创建Pod，使用PVC
-------------------------------------------------------------------------------------

Dockerfile 镜像构建

1、准备 tomcat jdk 包

apache-tomcat-10.1.16.tar.gz 
dockerfile  
jdk-11.0.21_linux-x64_bin.tar.gz

# Centos7 image
# VERSION 2
# Author:  sixmillions

# 指定基于的容器镜像
FROM centos:7

# 维护者信息
MAINTAINER sixmillions

# 镜像的操作指令
# 设置时间
RUN /bin/cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
  && echo 'Asia/Shanghai' >/etc/timezone
# 指定jdk1.8
add jdk-11.0.21_linux-x64_bin.tar.gz /usr/local/
add apache-tomcat-10.1.16.tar.gz /usr/local/
# RUN rm -f /usr/local/apache-tomcat-8.5.34/webapps/ROOT/favicon.ico
RUN mkdir -p /data &&  mv /usr/local/apache-tomcat-10.1.16 /usr/local/tomcat
env JAVA_HOME=/usr/local/jdk-11.0.21
env PATH=${JAVA_HOME}/bin:$PATH
env LC_ALL=en_US.UTF-8
# 一会我们做pv存储,直接将webapps目录映射出来.
# ADD helloworld.war /usr/local/apache-tomcat-8.5.34/webapps/
# RUN chmod +x /usr/local/apache-tomcat-7.0.68/bin/*
expose 8080

# 容器启动时执行指令
ENTRYPOINT [ "/usr/local/tomcat/bin/catalina.sh", "run" ]

打包基础镜像

docker build -t helloworldtomcat8 .

[root@Master dockerfile]# docker images hellowoldtomcat8
REPOSITORY         TAG       IMAGE ID       CREATED        SIZE
hellowoldtomcat8   latest    e13125a33897   18 hours ago   520MB

3. 上传到私有仓库

将打包好的镜像上传到私有仓库,方便其他节点获取

master节点

# 改个名字
docker tag  hellowoldtomcat8 101.43.xxx.xxx:8888/test/helloworldtomcat8:latest

# 上传
docker push 101.43.196.155:8888/test/helloworldtomcat8

node节点

下载镜像

docker pull 101.43.196.155:8888/test/helloworldtomcat8

四、使用 StorageClass 创建 tomcat logs webapp 存储

1、创建 tomcat-logs-pvc

cat 03-nfs-tomcat-logs-pvc.yaml 
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: nfs-tomcat-logs
spec:
  accessModes:  ["ReadWriteMany"]
  resources:
    requests:
      storage: 7Gi
  storageClassName:  nfs-nginx   # 指定使用storageclass的名称,来自动生产PV

查看tomcat-logs-pvc 创建

kubectl get pvc nfs-tomcat-logs
NAME              STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
nfs-tomcat-logs   Bound    pvc-618a8eaf-e798-4b03-b2c6-9b146eb6d792   7Gi        RWX            nfs-nginx      17h

1、创建 tomcat-webapp-pvc

cat 03-nfs-tomcat-webapp-pvc.yaml 
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: nfs-tomcat-webapp
spec:
  accessModes:  ["ReadWriteMany"]
  resources:
    requests:
      storage: 7Gi
  storageClassName:  nfs-nginx   # 指定使用storageclass的名称,来自动生产PV

查看tomcat-webapp pvc 创建

[root@Master tomcat]# kubectl apply -f 03-nfs-tomcat-webapp-pvc.yaml 
persistentvolumeclaim/nfs-tomcat-webapp unchanged
[root@Master tomcat]# kubectl get pvc nfs-tomcat-webapp
NAME                STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
nfs-tomcat-webapp   Bound    pvc-0b0f9b13-1bf0-46ff-8ce5-a95ec8a6dedd   7Gi        RWX            nfs-nginx      17h

创建 tomcat demo

cat 04-nfs-pod-demo.yaml 
apiVersion: apps/v1
kind: DaemonSet
metadata: 
  name: tomcattest
spec:
  selector:
    matchLabels:
      app: tomcattest
  template: 
    metadata: 
      labels: 
        app: tomcattest
    spec: 
      containers: 
        - name: tomcattest
          image: 101.43.196.155:8888/test/helloworldtomcat8
          imagePullPolicy: Always
          ports: 
            - containerPort: 8080
          volumeMounts:
            - name: logs
              mountPath: /usr/local/tomcat/logs
            - name: webapps
              mountPath: /usr/local/tomcat/webapps
      volumes:
        - name: logs
          persistentVolumeClaim:
            claimName: nfs-tomcat-logs
        - name: webapps
          persistentVolumeClaim:
            claimName: nfs-tomcat-webapp

查看创建结果

[root@Master tomcat]# kubectl apply -f 05-nfs-pod-demo.yaml 
daemonset.apps/tomcattest unchanged
[root@Master tomcat]# kubectl get pods 
NAME                        READY   STATUS    RESTARTS   AGE
tomcattest-8rcbl            1/1     Running   0          17h
tomcattest-9mv7p            1/1     Running   0          17h
tomcattest-qwgmm            1/1     Running   0          17h

2.

构建service 编写yaml文件

cat 06-tomcat-svc.yaml 
apiVersion: v1
kind: Service
metadata:
  name: tomcattest-service
spec:
  type: NodePort
  ports:
  - port: 8080
    name: tomcat
    targetPort: 8080
    nodePort: 32012
    protocol: TCP
  selector:
    app: tomcattest

构建Service

kubectl apply -f 06-tomcat-svc.yaml 
service/tomcattest-service unchanged

[root@Master tomcat]# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.16.42.3  netmask 255.255.255.0  broadcast 172.16.42.255
        inet6 fe80::216:3eff:fe08:a149  prefixlen 64  scopeid 0x20<link>
        ether 00:16:3e:08:a1:49  txqueuelen 1000  (Ethernet)
        RX packets 26297104  bytes 8933567314 (8.3 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 20506729  bytes 14593454145 (13.5 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@Master tomcat]# curl 172.16.42.3:32012
<!doctype html><html lang="en"><head><title>HTTP Status 404 – Not Found</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 404 – Not Found</h1><hr class="line" /><p><b>Type</b> Status Report</p><p><b>Description</b> The origin server did not find a current representation for the target resource or is not willing to disclose that one exists.</p><hr class="line" /><h3>Apache Tomcat/10.1.16</h3></body></html>[root@Master tomcat]#

测试

增加一个界面在映射文件

root@Node2 helloworld]# pwd
/data/test/default-nfs-tomcat-webapp-pvc-0b0f9b13-1bf0-46ff-8ce5-a95ec8a6dedd/helloworld
[root@Node2 helloworld]# cat index.html 
k8s-homework-successful!

[root@Master tomcat]# curl 172.16.42.3:32012/helloworld/index.html
k8s-homework-successful!