Integrate HashiCorp Vault with Kubernetes deployment

โดยหลักๆ แล้วจะมีวิธีการ integrate อยู่สองแบบคือ

  1. Vault Agent Injector เป็นวิธีการที่นิยมที่สุดด้วยการใช้ Annottions เพื่อให้ Vault Agent สร้าง sidecar container ทำการ inject credential ให้กับ pod
  2. Vault CSI Provider ใช้ Secret Store CSI Driver เพื่อ mount Vault secret ในรูปแบบ volume ให้กับ pod

ตัวอย่างนี้จะใช้วิธีแรกคือ Vault Agent Injector ที่เราได้ทำการ setup แล้วใน Install Hashicorp Vault to Kubernetes และใช้ตัวอย่างของ application testapp-cd โดยที่ branch main จะใช้ password จาก secret

เราจะเปลี่ยนให้เรียก password จาก vault ซึ่งต้องเพิ่ม annotation ตามตัวอย่างใน branch dev testapp-cd

เริ่มแรกต้องสร้าง service account สำหรับ vault agent เรียกไปยัง vault server

kubectl create sa vault-auth -n vault

กำหนดสิทธิให้กับ service account ด้วย cluster rolebinding เพื่อให้ review token ได้

kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: vault-auth-tokenreview-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:auth-delegator
subjects:
- kind: ServiceAccount
  name: vault-auth
  namespace: vault
EOF

สร้าง service account ภายใต้ namespace ของ application 

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
  name: octopus-app-sa
  namespace: default
EOF

ทำการ enable kubernetes auth method จาก vault cli

export VAULT_ADDR="http://10.55.39.59:8200"
[nutanix@nkp-boot ~]$ vault login hvs.j5wj9UsLG6lQqNS3nsdyBZN6
[nutanix@nkp-boot ~]$ vault auth enable kubernetes
Success! Enabled kubernetes auth method at: kubernetes/

สร้างข้อมูลที่จำเป็นสำหรับ kubernetes auth method

K8S_HOST=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')
K8S_CA_CERT=$(kubectl config view --raw --minify -o jsonpath='{.clusters[0].cluster.certificate-authority-data}' | base64 -d)
TOKEN_REVIEWER_JWT=$(kubectl create token vault-auth -n vault 2>/dev/null || \
  kubectl get secret $(kubectl get sa vault-auth -n vault -o jsonpath='{.secrets[0].name}') -n vault -o jsonpath='{.data.token}' | base64 -d)

ทำการ configure kubernetes auth method

vault write auth/kubernetes/config \
  kubernetes_host="${K8S_HOST}" \
  kubernetes_ca_cert="${K8S_CA_CERT}" \
  token_reviewer_jwt="${TOKEN_REVIEWER_JWT}"

สร้าง read policy ให้กับ application ให้ read ที่ secret path ได้

vault policy write octopus-app-policy - <<POLICY
path "secret/data/octopus/mongodb" {
  capabilities = ["read"]
}
POLICY

สร้าง vault role binding ให้กับ service account ของ application และ policy ที่สร้างขึ้น

vault write auth/kubernetes/role/octopus-app \
  bound_service_account_names=octopus-app-sa \
  bound_service_account_namespaces=default \
  policies=octopus-app-policy \
  ttl=1h

สร้าง secret บน vault server

vault kv put secret/octopus/mongodb \
  DB_USER="admin" \
  DB_PASSWORD="password123"

ทำสอบ deploy app โดย clone code จาก testapp-cd แล้วเปลี่ยนไปที่ branch dev

[nutanix@nkp-boot ~]$ git clone https://gitlab.com/pkhamdee/testapp-cd.git
Cloning into 'testapp-cd'...
remote: Enumerating objects: 189, done.
remote: Counting objects: 100% (189/189), done.
remote: Compressing objects: 100% (86/86), done.
remote: Total 189 (delta 97), reused 179 (delta 93), pack-reused 0 (from 0)
Receiving objects: 100% (189/189), 21.81 KiB | 21.81 MiB/s, done.
Resolving deltas: 100% (97/97), done.
[nutanix@nkp-boot ~]$ cd testapp-cd/
[nutanix@nkp-boot testapp-cd]$ git checkout dev
branch 'dev' set up to track 'origin/dev'.
Switched to a new branch 'dev'
[nutanix@nkp-boot testapp-cd]$ cd k8s-yamls
[nutanix@nkp-boot k8s-yamls]$ ls
app-deployment.yaml  app-service.yaml  argocd  configmap.yaml  mongodb-deployment.yaml  mongodb-service.yaml  route.yaml  secret.yaml
[nutanix@nkp-boot k8s-yamls]$ k apply -f .
deployment.apps/octopus-app created
service/octopus-app created
configmap/octopus-exam-config created
deployment.apps/mongodb created
service/mongodb created
Warning: annotation "kubernetes.io/ingress.class" is deprecated, please use 'spec.ingressClassName' instead
ingress.networking.k8s.io/octopus-app-route created
secret/octopusexam-secret created

ตรวจสอบ vault agent ว่าทำงานถูกต้องจากการ describe ที่ pod

[nutanix@nkp-boot ~]$ kubectl describe pod octopus-app-6dcf45487d-dnpbs
Name:             octopus-app-6dcf45487d-dnpbs
Namespace:        default
Priority:         0
Service Account:  octopus-app-sa
Node:             workload01-md-0-s5zcv-ntnvg-xrxpm/10.55.39.109
Start Time:       Fri, 27 Feb 2026 06:15:57 +0000
Labels:           app=octopus-app
                  pod-template-hash=6dcf45487d
Annotations:      vault.hashicorp.com/agent-inject: true
                  vault.hashicorp.com/agent-inject-secret-db-creds: secret/data/octopus/mongodb
                  vault.hashicorp.com/agent-inject-status: injected
                  vault.hashicorp.com/agent-inject-template-db-creds:
                    {{- with secret "secret/data/octopus/mongodb" -}}
                    export DB_USER="{{ .Data.data.DB_USER }}"
                    export DB_PASSWORD="{{ .Data.data.DB_PASSWORD }}"
                    {{- end -}}
                  vault.hashicorp.com/role: octopus-app
Status:           Running
IP:               192.168.1.45
IPs:
  IP:           192.168.1.45
Controlled By:  ReplicaSet/octopus-app-6dcf45487d
Init Containers:
  wait-for-mongodb:
    Container ID:  containerd://0dc59ec4b9b7894eedbbe48ed3a6ec3bff7dcd75b5408720fb0c89bd9d5abc7f
    Image:         busybox:1.35
    Image ID:      docker.io/library/busybox@sha256:98ad9d1a2be345201bb0709b0d38655eb1b370145c7d94ca1fe9c421f76e245a
    Port:          <none>
    Host Port:     <none>
    Command:
      sh
      -c
      until nc -z mongodb 27017; do echo waiting for mongodb; sleep 2; done;
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 27 Feb 2026 06:15:58 +0000
      Finished:     Fri, 27 Feb 2026 06:16:21 +0000
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-gpwm5 (ro)
      /vault/secrets from vault-secrets (rw)
  vault-agent-init:
    Container ID:  containerd://3c34e6174111c0cd278f1a251c48c01a339e625982c18af05b6fa91040eb9107
    Image:         hashicorp/vault:1.21.2
    Image ID:      docker.io/hashicorp/vault@sha256:eb0ba6836e8d4699b7a1e8ca70d8433f7b87dcd067e6d82dff237d3ed2600ea0
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/sh
      -ec
    Args:
      echo ${VAULT_CONFIG?} | base64 -d > /home/vault/config.json && vault agent -config=/home/vault/config.json
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 27 Feb 2026 06:16:22 +0000
      Finished:     Fri, 27 Feb 2026 06:16:22 +0000
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     500m
      memory:  128Mi
    Requests:
      cpu:     250m
      memory:  64Mi
    Environment:
      NAMESPACE:         default (v1:metadata.namespace)
      HOST_IP:            (v1:status.hostIP)
      POD_IP:             (v1:status.podIP)
      VAULT_LOG_LEVEL:   info
      VAULT_LOG_FORMAT:  standard
      VAULT_CONFIG:      eyJhdXRvX2F1dGgiOnsibWV0aG9kIjp7InR5cGUiOiJrdWJlcm5ldGVzIiwibW91bnRfcGF0aCI6ImF1dGgva3ViZXJuZXRlcyIsImNvbmZpZyI6eyJyb2xlIjoib2N0b3B1cy1hcHAiLCJ0b2tlbl9wYXRoIjoiL3Zhci9ydW4vc2VjcmV0cy9rdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3Rva2VuIn19LCJzaW5rIjpbeyJ0eXBlIjoiZmlsZSIsImNvbmZpZyI6eyJwYXRoIjoiL2hvbWUvdmF1bHQvLnZhdWx0LXRva2VuIn19XX0sImV4aXRfYWZ0ZXJfYXV0aCI6dHJ1ZSwicGlkX2ZpbGUiOiIvaG9tZS92YXVsdC8ucGlkIiwidmF1bHQiOnsiYWRkcmVzcyI6Imh0dHA6Ly92YXVsdC52YXVsdC5zdmM6ODIwMCJ9LCJ0ZW1wbGF0ZSI6W3siZGVzdGluYXRpb24iOiIvdmF1bHQvc2VjcmV0cy9kYi1jcmVkcyIsImNvbnRlbnRzIjoie3stIHdpdGggc2VjcmV0IFwic2VjcmV0L2RhdGEvb2N0b3B1cy9tb25nb2RiXCIgLX19XG5leHBvcnQgREJfVVNFUj1cInt7IC5EYXRhLmRhdGEuREJfVVNFUiB9fVwiXG5leHBvcnQgREJfUEFTU1dPUkQ9XCJ7eyAuRGF0YS5kYXRhLkRCX1BBU1NXT1JEIH19XCJcbnt7LSBlbmQgLX19XG4iLCJsZWZ0X2RlbGltaXRlciI6Int7IiwicmlnaHRfZGVsaW1pdGVyIjoifX0ifV0sInRlbXBsYXRlX2NvbmZpZyI6eyJleGl0X29uX3JldHJ5X2ZhaWx1cmUiOnRydWV9fQ==
    Mounts:
      /home/vault from home-init (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-gpwm5 (ro)
      /vault/secrets from vault-secrets (rw)
Containers:
  octopus-nodejs-container:
    Container ID:  containerd://214f0a43c0b1f52b8972fe5a7ef80bb63d4cff6bf79a95df609042c1a453d8aa
    Image:         pkhamdee/testapp:c0d742f
    Image ID:      docker.io/pkhamdee/testapp@sha256:f4d4a5ef8a19853ceef1be3ecec90643bd74ace7b9f0b04c0f22ac2a1a5a1624
    Port:          3000/TCP
    Host Port:     0/TCP
    Command:
      /bin/sh
      -c
    Args:
      source /vault/secrets/db-creds && node app.js
    State:          Running
      Started:      Fri, 27 Feb 2026 06:16:28 +0000
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     500m
      memory:  256Mi
    Requests:
      cpu:     500m
      memory:  256Mi
    Environment:
      DB_HOST:  <set to the key 'DB_HOST' of config map 'octopus-exam-config'>  Optional: false
      DB_PORT:  <set to the key 'DB_PORT' of config map 'octopus-exam-config'>  Optional: false
      DB_NAME:  <set to the key 'DB_NAME' of config map 'octopus-exam-config'>  Optional: false
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-gpwm5 (ro)
      /vault/secrets from vault-secrets (rw)
  vault-agent:
    Container ID:  containerd://2d68c01152a9fcd0c98ac6322209f6eb578b3cb4343861664a0a2da7be720320
    Image:         hashicorp/vault:1.21.2
    Image ID:      docker.io/hashicorp/vault@sha256:eb0ba6836e8d4699b7a1e8ca70d8433f7b87dcd067e6d82dff237d3ed2600ea0
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/sh
      -ec
    Args:
      echo ${VAULT_CONFIG?} | base64 -d > /home/vault/config.json && vault agent -config=/home/vault/config.json
    State:          Running
      Started:      Fri, 27 Feb 2026 06:16:28 +0000
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     500m
      memory:  128Mi
    Requests:
      cpu:     250m
      memory:  64Mi
    Environment:
      NAMESPACE:         default (v1:metadata.namespace)
      HOST_IP:            (v1:status.hostIP)
      POD_IP:             (v1:status.podIP)
      VAULT_LOG_LEVEL:   info
      VAULT_LOG_FORMAT:  standard
      VAULT_CONFIG:      eyJhdXRvX2F1dGgiOnsibWV0aG9kIjp7InR5cGUiOiJrdWJlcm5ldGVzIiwibW91bnRfcGF0aCI6ImF1dGgva3ViZXJuZXRlcyIsImNvbmZpZyI6eyJyb2xlIjoib2N0b3B1cy1hcHAiLCJ0b2tlbl9wYXRoIjoiL3Zhci9ydW4vc2VjcmV0cy9rdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3Rva2VuIn19LCJzaW5rIjpbeyJ0eXBlIjoiZmlsZSIsImNvbmZpZyI6eyJwYXRoIjoiL2hvbWUvdmF1bHQvLnZhdWx0LXRva2VuIn19XX0sImV4aXRfYWZ0ZXJfYXV0aCI6ZmFsc2UsInBpZF9maWxlIjoiL2hvbWUvdmF1bHQvLnBpZCIsInZhdWx0Ijp7ImFkZHJlc3MiOiJodHRwOi8vdmF1bHQudmF1bHQuc3ZjOjgyMDAifSwidGVtcGxhdGUiOlt7ImRlc3RpbmF0aW9uIjoiL3ZhdWx0L3NlY3JldHMvZGItY3JlZHMiLCJjb250ZW50cyI6Int7LSB3aXRoIHNlY3JldCBcInNlY3JldC9kYXRhL29jdG9wdXMvbW9uZ29kYlwiIC19fVxuZXhwb3J0IERCX1VTRVI9XCJ7eyAuRGF0YS5kYXRhLkRCX1VTRVIgfX1cIlxuZXhwb3J0IERCX1BBU1NXT1JEPVwie3sgLkRhdGEuZGF0YS5EQl9QQVNTV09SRCB9fVwiXG57ey0gZW5kIC19fVxuIiwibGVmdF9kZWxpbWl0ZXIiOiJ7eyIsInJpZ2h0X2RlbGltaXRlciI6In19In1dLCJ0ZW1wbGF0ZV9jb25maWciOnsiZXhpdF9vbl9yZXRyeV9mYWlsdXJlIjp0cnVlfX0=
    Mounts:
      /home/vault from home-sidecar (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-gpwm5 (ro)
      /vault/secrets from vault-secrets (rw)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True
  Initialized                 True
  Ready                       True
  ContainersReady             True
  PodScheduled                True
Volumes:
  kube-api-access-gpwm5:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
  home-init:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     Memory
    SizeLimit:  <unset>
  home-sidecar:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     Memory
    SizeLimit:  <unset>
  vault-secrets:
    Type:        EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:      Memory
    SizeLimit:   <unset>
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  43m   default-scheduler  Successfully assigned default/octopus-app-6dcf45487d-dnpbs to workload01-md-0-s5zcv-ntnvg-xrxpm
  Normal  Pulling    43m   kubelet            Pulling image "busybox:1.35"
  Normal  Pulled     43m   kubelet            Successfully pulled image "busybox:1.35" in 1.178s (1.178s including waiting). Image size: 2159953 bytes.
  Normal  Created    43m   kubelet            Created container: wait-for-mongodb
  Normal  Started    43m   kubelet            Started container wait-for-mongodb
  Normal  Pulled     42m   kubelet            Container image "hashicorp/vault:1.21.2" already present on machine
  Normal  Created    42m   kubelet            Created container: vault-agent-init
  Normal  Started    42m   kubelet            Started container vault-agent-init
  Normal  Pulling    42m   kubelet            Pulling image "pkhamdee/testapp:c0d742f"
  Normal  Pulled     42m   kubelet            Successfully pulled image "pkhamdee/testapp:c0d742f" in 5.156s (5.156s including waiting). Image size: 55585478 bytes.
  Normal  Created    42m   kubelet            Created container: octopus-nodejs-container
  Normal  Started    42m   kubelet            Started container octopus-nodejs-container
  Normal  Pulled     42m   kubelet            Container image "hashicorp/vault:1.21.2" already present on machine
  Normal  Created    42m   kubelet            Created container: vault-agent
  Normal  Started    42m   kubelet            Started container vault-agent