Pods are ephemeral. Their storage dies with them. Databases need persistent storage. PersistentVolumes (PVs) are cluster storage resources. PersistentVolumeClaims (PVCs) are requests for storage. StorageClasses enable dynamic provisioning. In this lesson, you'll master persistent storage for stateful applications.

1. Learning Objectives

By the end of this lesson, you will be able to:

  • Understand PersistentVolumes (PV) and PersistentVolumeClaims (PVC)
  • Create static PVs for pre-provisioned storage
  • Use StorageClasses for dynamic provisioning
  • Configure StatefulSets with persistent storage
  • Backup and restore persistent volumes
  • Troubleshoot storage issues

2. Why This Matters

Real-world scenario: Your PostgreSQL database pod dies. Without persistent storage, all your data is gone. With PersistentVolumes, the data survives and attaches to the new pod. Your database can even be rescheduled to another node without data loss.

3. Core Concepts

Static PersistentVolumes

# persistent-volume.yaml - Static PV
apiVersion: v1
kind: PersistentVolume
metadata:
  name: postgres-pv
  labels:
    type: local
    storage: ssd
spec:
  capacity:
    storage: 10Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce    # Can be mounted by single pod
    # - ReadOnlyMany    # Read-only by many pods
    # - ReadWriteMany   # Read-write by many pods
  persistentVolumeReclaimPolicy: Retain
  # Reclaim policies: Retain, Recycle, Delete
  storageClassName: standard
  hostPath:
    path: /mnt/data/postgres  # For local testing only!
  # For production, use cloud storage:
  # awsElasticBlockStore:
  #   volumeID: vol-123456
  #   fsType: ext4
  # gcePersistentDisk:
  #   pdName: postgres-disk
  #   fsType: ext4
  # azureDisk:
  #   diskName: postgres-disk
  #   diskURI: https://...
  # nfs:
  #   server: nfs-server.example.com
  #   path: /exports/postgres

# persistent-volume-claim.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  volumeMode: Filesystem
  resources:
    requests:
      storage: 5Gi
  storageClassName: standard
  selector:
    matchLabels:
      type: local
      storage: ssd

kubectl apply -f persistent-volume.yaml
kubectl apply -f persistent-volume-claim.yaml
kubectl get pv
kubectl get pvc
kubectl describe pv postgres-pv
kubectl describe pvc postgres-pvc
Static PersistentVolumes and Claims

StorageClasses (Dynamic Provisioning)

# storage-class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast
provisioner: kubernetes.io/aws-ebs  # Cloud-specific
parameters:
  type: gp3
  fsType: ext4
  iopsPerGB: "10"
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true

# AWS EBS StorageClass
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gp3-retain
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
  encrypted: "true"
reclaimPolicy: Retain

# Azure Disk StorageClass
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: azure-disk
provisioner: disk.csi.azure.com
parameters:
  skuname: StandardSSD_LRS
  kind: Managed

# GCP Persistent Disk StorageClass
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: standard-rwo
provisioner: pd.csi.storage.gke.io
parameters:
  type: pd-standard
  replication-type: none

# Local SSD (for testing)
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-ssd
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

kubectl get storageclass
kubectl describe storageclass fast
kubectl get storageclass -o wide
StorageClasses for dynamic provisioning

Using PVC in Pods and Deployments

# pod-with-pvc.yaml
apiVersion: v1
kind: Pod
metadata:
  name: postgres-pod
spec:
  containers:
  - name: postgres
    image: postgres:15-alpine
    env:
    - name: POSTGRES_PASSWORD
      value: "password"
    volumeMounts:
    - name: postgres-storage
      mountPath: /var/lib/postgresql/data
      subPath: postgres
  volumes:
  - name: postgres-storage
    persistentVolumeClaim:
      claimName: postgres-pvc

# deployment-with-pvc.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - name: mysql
        image: mysql:8
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: "password"
        volumeMounts:
        - name: mysql-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-storage
        persistentVolumeClaim:
          claimName: mysql-pvc
Using PVCs in pods and deployments

StatefulSet with Persistent Storage

# statefulset-with-pvc.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres-statefulset
spec:
  serviceName: postgres-hl
  replicas: 3
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:15-alpine
        ports:
        - containerPort: 5432
        env:
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secret
              key: password
        volumeMounts:
        - name: data
          mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: ["ReadWriteOnce"]
      storageClassName: fast
      resources:
        requests:
          storage: 20Gi

# Headless service
apiVersion: v1
kind: Service
metadata:
  name: postgres-hl
spec:
  clusterIP: None
  selector:
    app: postgres
  ports:
  - port: 5432

# Each pod gets its own PVC:
# data-postgres-statefulset-0
# data-postgres-statefulset-1
# data-postgres-statefulset-2

kubectl get pvc
# NAME                         STATUS   VOLUME        CAPACITY
# data-postgres-statefulset-0  Bound    pv-abc123     20Gi
# data-postgres-statefulset-1  Bound    pv-def456     20Gi
# data-postgres-statefulset-2  Bound    pv-ghi789     20Gi
StatefulSet with volumeClaimTemplates

Volume Snapshot and Restore

# volume-snapshot-class.yaml
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
  name: csi-snapclass
driver: ebs.csi.aws.com
deletionPolicy: Delete
parameters:
  type: snap

# volume-snapshot.yaml
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: postgres-snapshot
spec:
  volumeSnapshotClassName: csi-snapclass
  source:
    persistentVolumeClaimName: postgres-pvc

# Restore from snapshot
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pvc-restored
spec:
  dataSource:
    name: postgres-snapshot
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi

kubectl get volumesnapshotclass
kubectl get volumesnapshot
kubectl describe volumesnapshot postgres-snapshot
Volume snapshots for backup and restore

4. Complete Project: Production PostgreSQL

# complete-postgres.yaml
---
# StorageClass
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: postgres-storage
provisioner: kubernetes.io/aws-ebs  # Adjust for cloud
parameters:
  type: gp3
  encrypted: "true"
reclaimPolicy: Retain
allowVolumeExpansion: true

---
# Secret for PostgreSQL
apiVersion: v1
kind: Secret
metadata:
  name: postgres-secret
type: Opaque
data:
  POSTGRES_PASSWORD: c3VwZXJzZWNyZXQK  # supersecret
  REPLICATION_PASSWORD: cmVwbGljYXRpb24K  # replication

---
# ConfigMap for PostgreSQL configuration
apiVersion: v1
kind: ConfigMap
metadata:
  name: postgres-config
data:
  postgresql.conf: |
    max_connections = 100
    shared_buffers = 128MB
    effective_cache_size = 384MB
    maintenance_work_mem = 64MB
    checkpoint_completion_target = 0.9
    wal_buffers = 16MB
    default_statistics_target = 100
    random_page_cost = 1.1
    effective_io_concurrency = 200
    work_mem = 4MB
    huge_pages = try
    shared_preload_libraries = 'pg_stat_statements'
    pg_stat_statements.track = all
  pg_hba.conf: |
    local all all trust
    host all all 0.0.0.0/0 md5
    host replication replicator 0.0.0.0/0 md5

---
# Headless Service
apiVersion: v1
kind: Service
metadata:
  name: postgres-hl
spec:
  clusterIP: None
  selector:
    app: postgres
  ports:
  - port: 5432
    name: postgres

---
# Read Service (for applications)
apiVersion: v1
kind: Service
metadata:
  name: postgres-service
spec:
  selector:
    app: postgres
  ports:
  - port: 5432
    targetPort: 5432

---
# StatefulSet
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
spec:
  serviceName: postgres-hl
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:15-alpine
        ports:
        - containerPort: 5432
          name: postgres
        env:
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secret
              key: POSTGRES_PASSWORD
        - name: POSTGRES_USER
          value: "postgres"
        - name: POSTGRES_DB
          value: "myapp"
        - name: PGDATA
          value: /var/lib/postgresql/data/pgdata
        volumeMounts:
        - name: data
          mountPath: /var/lib/postgresql/data
        - name: config
          mountPath: /etc/postgresql
        resources:
          requests:
            memory: "1Gi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "1000m"
        livenessProbe:
          exec:
            command:
            - pg_isready
            - -U
            - postgres
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          exec:
            command:
            - pg_isready
            - -U
            - postgres
          initialDelaySeconds: 5
          periodSeconds: 5
      volumes:
      - name: config
        configMap:
          name: postgres-config
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: ["ReadWriteOnce"]
      storageClassName: postgres-storage
      resources:
        requests:
          storage: 50Gi

---
# Backup CronJob
apiVersion: batch/v1
kind: CronJob
metadata:
  name: postgres-backup
spec:
  schedule: "0 2 * * *"  # Daily at 2 AM
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: postgres:15-alpine
            env:
            - name: PGPASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgres-secret
                  key: POSTGRES_PASSWORD
            command:
            - /bin/sh
            - -c
            - |
              pg_dump -h postgres-service -U postgres myapp > /backup/backup-$(date +%Y%m%d-%H%M%S).sql
              gzip /backup/*.sql
            volumeMounts:
            - name: backup
              mountPath: /backup
          volumes:
          - name: backup
            persistentVolumeClaim:
              claimName: postgres-backup-pvc
          restartPolicy: OnFailure

# Apply everything
kubectl apply -f complete-postgres.yaml

# Wait for PostgreSQL to be ready
kubectl wait --for=condition=ready pod/postgres-0 --timeout=300s

# Test connectivity
kubectl run test-postgres --image=postgres:15-alpine -it --rm --restart=Never -- \
  psql -h postgres-service -U postgres -d myapp -c "SELECT version();"

# Check PVC status
kubectl get pvc
kubectl get pv
Production PostgreSQL with persistent storage

5. Common Errors & Solutions

6. Summary Checklist

7. Next Steps

Next lesson: Kubernetes Lesson 6.6: Probes & Application Health

Tags: kubernetes, persistent-volumes, pvc, storage, statefulsets

Gataya Med

DevOps Engineer & Backend Developer. Sharing insights on cloud, automation, and scalable systems.

gatayaMed View Profile @https://github.com/gatayaMed

Comments (0)

Sarah Chen February 11, 2025

This is exactly what I needed! The initContainer approach solved our migration issues completely. Thanks for the detailed guide!

Reply

Leave a Comment