Nutanix File Storage Class

วิธีการ config ให้ Nutanix CSI เชื่อมต่อไปยัง File Service (NUS) ทำให้สามารถสร้าง persistent volume บน File Share ได้ ขั้นตอนการสร้าง Storage Class ทำได้ตามตัวอย่างดังนี้

สร้าง secret เพื่อเชื่อมต่อไปยัง Nutanix File server ผ่าน Prism Central

YAML
apiVersion: v1
kind: Secret
metadata:
 name: pc-files-secret
 namespace: ntnx-system
stringData:
# Provide Nutanix File Server credentials which is a REST API user created on File server UI separated by colon in "files-key:".
  key: "<<PC IP Address>>:9440:<<PC userid>>:<<password>>"
  files-key: "<<File Server IP>>:<<REST API userid>>:<<password>>"

ตัวอย่าง secret ที่สร้างจากข้อมูลที่ได้จาก lab

YAML
apiVersion: v1
kind: Secret
metadata:
 name: nutanix-csi-credentials-files
 namespace: ntnx-system
stringData:
# Provide Nutanix File Server credentials which is a REST API user created on File server UI separated by colon in "files-key:".
  key: "192.168.10.25:9440:admin:P@ssw0rd123!"
  files-key: "nas.nutanix.poc:tao:nutanix/4u"

สร้าง Storage Class ด้วยข้อมูลดังนี้

YAML
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
    name: nfs-pc-sc
provisioner: csi.nutanix.com
parameters:
    csi.storage.k8s.io/node-publish-secret-name: pc-files-secret
    csi.storage.k8s.io/node-publish-secret-namespace: ntnx-system
    csi.storage.k8s.io/controller-expand-secret-name: pc-files-secret
    csi.storage.k8s.io/controller-expand-secret-namespace: ntnx-system
    dynamicProv: ENABLED
    nfsServer: <<IP address of file server>>
    nfsServerName: <<file server name>>
    csi.storage.k8s.io/provisioner-secret-name: pc-files-secret
    csi.storage.k8s.io/provisioner-secret-namespace: ntnx-system
    storageType: NutanixFiles
    squashType: root-squash
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true

ตัวอย่างการสร้าง file storage class ด้วยการใช้ข้อมูลจาก lab

YAML
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
    name: nutanix-files
provisioner: csi.nutanix.com
parameters:
    csi.storage.k8s.io/node-publish-secret-name: nutanix-csi-credentials-files
    csi.storage.k8s.io/node-publish-secret-namespace: ntnx-system
    csi.storage.k8s.io/controller-expand-secret-name: nutanix-csi-credentials-files
    csi.storage.k8s.io/controller-expand-secret-namespace: ntnx-system
    dynamicProv: ENABLED
    nfsServer: nas.nutanix.poc
    nfsServerName: nas
    csi.storage.k8s.io/provisioner-secret-name: nutanix-csi-credentials-files
    csi.storage.k8s.io/provisioner-secret-namespace: ntnx-system
    storageType: NutanixFiles
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true

deploy workload เพื่อทดสอบสร้าง file volume ทำการเขียนและอ่านไฟล์จาก file volume

YAML
---
# 1. PVC
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-fileshare-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: nutanix-files   # Change this
  resources:
    requests:
      storage: 1Gi
---
# 2. Writer Pod - writes a file
apiVersion: v1
kind: Pod
metadata:
  name: test-writer
  namespace: default
spec:
  containers:
  - name: writer
    image: busybox
    command: ["/bin/sh", "-c"]
    args:
      - |
        echo "Hello from writer pod at $(date)" > /data/testfile.txt
        echo "Write successful!"
        cat /data/testfile.txt
        sleep 3600
    volumeMounts:
    - name: shared-vol
      mountPath: /data
  volumes:
  - name: shared-vol
    persistentVolumeClaim:
      claimName: test-fileshare-pvc
---
# 3. Reader Pod - reads the same file (proves RWX works)
apiVersion: v1
kind: Pod
metadata:
  name: test-reader
  namespace: default
spec:
  containers:
  - name: reader
    image: busybox
    command: ["/bin/sh", "-c"]
    args:
      - |
        echo "Waiting for file..."
        until [ -f /data/testfile.txt ]; do sleep 2; done
        echo "Read successful!"
        cat /data/testfile.txt
        sleep 3600
    volumeMounts:
    - name: shared-vol
      mountPath: /data
  volumes:
  - name: shared-vol
    persistentVolumeClaim:
      claimName: test-fileshare-pvc

script สำหรับทดสอบ

Shell
# 1. Check available storage classes
kubectl get storageclass
# 2. Deploy (replace <your-storage-class> first)
kubectl apply -f test-fileshare.yaml
# 3. Check PVC is bound
kubectl get pvc test-fileshare-pvc
# 4. Check both pods are running
kubectl get pods test-writer test-reader
# 5. Verify writer wrote the file
kubectl logs test-writer
# 6. Verify reader can read the same file
kubectl logs test-reader
# 7. Extra test - write from reader pod too
kubectl exec test-reader -- sh -c 'echo "Hello from reader" >> /data/testfile.txt'
kubectl exec test-writer -- cat /data/testfile.txt

ข้อควรระวัง

CIS hardening จะทำการ disable/masks rpcbind ทำให้ Storage Class ไม่สามารถ mount volume NFSv3 บน worker node ได้ โดยมีทางเลือกดังนี้

Option 1 แนะนำให้ใช้ NFSv4 เนื่องจาก NFSv4 ไม่ต้องการ rpcbind หรือ rpc-statd เนื่องจากสามารถทำกระบวนการ locking file ได้ด้วยตัวเอง รวมทั้งเป็นไปตามข้อกำหนดของ CIS hardening

Shell
# Test NFSv4 mount
mount -t nfs -o vers=4 files.ntnxlab.local:/test /mnt/test

สำหรับ Nutanix CSI, ทำการ force ให้ใช้ NFSv4 ได้ตามตัวอย่าง

YAML
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nutanix-files-nfsv4
provisioner: csi.nutanix.com
parameters:
  nfsServerName: nas
  storageType: NutanixFiles
  mountOptions: "vers=4"
mountOptions:
  - vers=4
  - nolock

Option 2 ทำการ Unmask rpcbind ซึ่งจะมีผลต่อความปลอดภัย โดยจะต้องทำการ unmask ในทุกๆ kubernetes node

Shell
systemctl unmask rpcbind.socket
systemctl unmask rpcbind.service
systemctl enable rpcbind --now

Option 3 ทำการ mount volume ด้วย option nolock ในกรณีที่ CSI support ด้วยวิธีนี้จะข้าม rpc-statd แต่ disable file locking

YAML
mountOptions:
  - nolock
  - vers=3

note – install wordpress โดยใช้ file volume

YAML
apiVersion: v1
kind: Secret
metadata:
  name: mysql-pass
type: Opaque
stringData:
  password: nutanix
---
apiVersion: v1
kind: Service
metadata:
  name: wordpress-mysql
  labels:
    app: wordpress
spec:
  ports:
    - port: 3306
  selector:
    app: wordpress
    tier: mysql
  clusterIP: None
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim
  labels:
    app: wordpress
spec:
  storageClassName: nutanix-files
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: wordpress-mysql
  labels:
    app: wordpress
spec:
  selector:
    matchLabels:
      app: wordpress
      tier: mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: wordpress
        tier: mysql
    spec:
      containers:
        - image: https://10.38.16.169/docker.io/mysql:latest
          name: mysql
          env:
            - name: MYSQL_ROOT_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: mysql-pass
                  key: password
            - name: MYSQL_DATABASE
              value: wordpress
            - name: MYSQL_USER
              value: wordpress
            - name: MYSQL_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: mysql-pass
                  key: password
          ports:
            - containerPort: 3306
              name: mysql
          volumeMounts:
            - name: mysql-persistent-storage
              mountPath: /var/lib/mysql
      volumes:
        - name: mysql-persistent-storage
          persistentVolumeClaim:
            claimName: mysql-pv-claim
---
apiVersion: v1
kind: Service
metadata:
  name: wordpress
  labels:
    app: wordpress
spec:
  ports:
    - port: 80
  selector:
    app: wordpress
    tier: frontend
  type: ClusterIP
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: wp-pv-claim
  labels:
    app: wordpress
spec:
  storageClassName: nutanix-files
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: wordpress
  labels:
    app: wordpress
spec:
  selector:
    matchLabels:
      app: wordpress
      tier: frontend
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: wordpress
        tier: frontend
    spec:
      containers:
        - image: wordpress:latest
          name: wordpress
          env:
            - name: WORDPRESS_DB_HOST
              value: wordpress-mysql
            - name: WORDPRESS_DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: mysql-pass
                  key: password
            - name: WORDPRESS_DB_USER
              value: wordpress
            - name: WORDPRESS_DB_NAME
              value: wordpress
            - name: WORDPRESS_DEBUG
              value: "1"  
          ports:
            - containerPort: 80
              name: wordpress
          volumeMounts:
            - name: wordpress-persistent-storage
              mountPath: /var/www/html
      volumes:
        - name: wordpress-persistent-storage
          persistentVolumeClaim:
            claimName: wp-pv-claim
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: wordpress-ingress
spec:
  ingressClassName: kommander-traefik
  rules:
  - host: wordpress.10.55.42.19.sslip.io
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: wordpress
            port:
              number: 80