Moving a Wagtail CMS project from development to production requires a multi-layer stack: a reverse proxy, an application server, a pooled database connection, object storage for media, Redis for caching, and environment-based configuration. This lesson covers every layer from Dockerfile to deployment pipeline.

1. Learning Objectives

By the end of this lesson, you will be able to:

  • Configure a Wagtail project for production deployment with hardened Django settings
  • Deploy Wagtail to cloud platforms using Docker and a production-ready stack (Gunicorn + Nginx)
  • Set up S3-compatible media storage with django-storages and boto3 for image renditions and uploads
  • Serve static files through WhiteNoise or a CDN for optimal performance
  • Implement caching with Redis and configure Wagtail's cache backends
  • Tune database performance with connection pooling via PgBouncer
  • Automate deployment with CI/CD pipelines and environment variable management

2. Why This Matters (Real-World Scenario)

Your client runs a growing Wagtail news portal with 50,000 daily visitors. During peak hours, image renditions take 8 seconds to generate, database queries pile up behind the default connection limit, and the single-server setup crashes every time they push a new feature. The site is running with DEBUG=True in production because 'it breaks otherwise,' and SECRET_KEY is hardcoded in settings.py.

This is not an exaggeration — it is the default state of most first-time Wagtail deployments. Moving to a production-grade architecture means adopting a multi-layer stack: a reverse proxy (Nginx), an application server (Gunicorn), a connection-pooled database (PostgreSQL + PgBouncer), an object store for media (S3/MinIO), a CDN for static assets, Redis for caching and session storage, and environment-based configuration management. This lesson walks you through each layer from zero to production.

3. Core Concepts

3.1 The Production Stack

A production Wagtail deployment consists of several layers, each with a specific responsibility:

  • Nginx — Reverse proxy: terminates TLS, serves static/media files directly, forwards dynamic requests to Gunicorn
  • Gunicorn — Python WSGI server: runs Django/Wagtail application code. Configured with workers and threads
  • PostgreSQL — Primary database: stores pages, revisions, user data, and search index
  • PgBouncer — Connection pooler: sits between Gunicorn and PostgreSQL, reuses database connections to avoid connection storms
  • Redis — Cache backend + session store + task queue broker (Celery)
  • S3/MinIO — Object storage for uploaded images, documents, and generated image renditions
  • CDN — Content delivery network (CloudFront or Cloudflare) for caching static assets and media at the edge

3.2 Django Production Settings

The foundation of any Wagtail deployment is correct Django settings. These are the non-negotiable settings every production project must have:

import os
from pathlib import Path
import dj_database_url

# --- Security ---
SECRET_KEY = os.environ['DJANGO_SECRET_KEY']  # 50+ random chars, NEVER hardcoded
DEBUG = False
ALLOWED_HOSTS = os.environ.get('ALLOWED_HOSTS', 'yourdomain.com').split(',')
CSRF_TRUSTED_ORIGINS = os.environ.get(
    'CSRF_TRUSTED_ORIGINS', 'https://yourdomain.com'
).split(',')

# --- HTTPS ---
SECURE_SSL_REDIRECT = True
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True

# --- Database (connection pooling friendly) ---
DATABASES = {
    'default': dj_database_url.config(
        default=os.environ['DATABASE_URL'],
        conn_max_age=600,          # Keep connections alive for 10 min
        conn_health_checks=True,   # Verify connections before reuse
    )
}

# --- Static & Media ---
STATIC_URL = '/static/'
STATIC_ROOT = BASE_DIR / 'staticfiles'
STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'

# --- Media (see Section 4.3 for S3 config) ---
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
AWS_ACCESS_KEY_ID = os.environ['AWS_ACCESS_KEY_ID']
AWS_SECRET_ACCESS_KEY = os.environ['AWS_SECRET_ACCESS_KEY']
AWS_STORAGE_BUCKET_NAME = os.environ['AWS_STORAGE_BUCKET_NAME']
AWS_S3_REGION_NAME = os.environ.get('AWS_S3_REGION_NAME', 'us-east-1')
AWS_S3_CUSTOM_DOMAIN = os.environ.get('AWS_S3_CUSTOM_DOMAIN')
AWS_QUERYSTRING_AUTH = False  # Public URLs for media

# --- Cache ---
CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.redis.RedisCache',
        'LOCATION': os.environ.get('REDIS_URL', 'redis://127.0.0.1:6379/0'),
    },
    'renditions': {
        'BACKEND': 'django.core.cache.backends.redis.RedisCache',
        'LOCATION': os.environ.get('REDIS_URL', 'redis://127.0.0.1:6379/0'),
        'KEY_PREFIX': 'renditions',
        'TIMEOUT': 86400 * 30,  # 30 days — renditions rarely change
    },
}

# --- Wagtail-specific ---
WAGTAIL_CACHE = True
WAGTAIL_CACHE_BACKEND = 'default'
WAGTAILIMAGES_CACHE = True
WAGTAILIMAGES_CACHE_BACKEND = 'renditions'
WAGTAILSEARCH_BACKENDS = {
    'default': {
        'BACKEND': 'wagtail.search.backends.database',
    },
}

# --- Logging ---
LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'json': {
            '()': 'json_log_formatter.JSONFormatter',
        },
    },
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
            'formatter': 'json',
        },
    },
    'root': {
        'handlers': ['console'],
        'level': 'INFO',
    },
}
Production Django settings for Wagtail — security, database, caching, and logging

3.3 Deployment Strategies

Wagtail can be deployed on traditional VPS (Ubuntu + Nginx + Gunicorn), Platform-as-a-Service (Railway, Fly.io, Render), or containerized with Docker on Kubernetes. For most projects, the Docker + PaaS approach offers the best balance of simplicity and reliability:

# Option A: Platform-as-a-Service (Railway.app)
# Deploy with a Dockerfile and connect PostgreSQL + Redis add-ons
railway login
railway init
railway up

# Option B: Fly.io with flyctl
flyctl launch
flyctl postgres create
flyctl redis create
flyctl deploy

# Option C: Docker Compose (VPS with docker compose)
docker compose -f docker-compose.prod.yml up -d --build
Three deployment options: Railway (PaaS), Fly.io, and Docker Compose on a VPS

4. Hands-On Practice: Deploying Wagtail to Production

4.1 Dockerize Your Wagtail Project

Start with a production-grade Dockerfile that uses multi-stage builds to keep the image lean:

# Stage 1: Build static assets
FROM python:3.12-slim AS builder

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .
RUN python manage.py collectstatic --noinput

# Stage 2: Production image
FROM python:3.12-slim

RUN apt-get update && apt-get install -y --no-install-recommends \
    libpq-dev curl && \
    rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY --from=builder /usr/local/lib/python3.12/site-packages /usr/local/lib/python3.12/site-packages
COPY --from=builder /usr/local/bin /usr/local/bin
COPY --from=builder /app /app

ENV PYTHONUNBUFFERED=1

EXPOSE 8000
CMD ["gunicorn", "mysite.wsgi:application", "--bind", "0.0.0.0:8000",
     "--workers", "4", "--threads", "2", "--timeout", "120",
     "--access-logfile", "-", "--error-logfile", "-"]
Multi-stage Dockerfile — collectstatic in build stage, slim runtime image

Add a .dockerignore to exclude development files from the build context:

__pycache__/
*.pyc
.env
.env.*
.git/
.gitignore
*.md
Dockerfile
docker-compose*.yml
node_modules/
staticfiles/
media/
Docker ignore file to reduce build context size and speed up builds

4.2 Configure Nginx as a Reverse Proxy

Nginx sits in front of Gunicorn, terminating TLS, buffering requests, and serving static/media files directly without hitting the app server:

server {
    listen 80;
    server_name yourdomain.com www.yourdomain.com;
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl http2;
    server_name yourdomain.com www.yourdomain.com;

    # TLS certificates (use Let's Encrypt + Certbot)
    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;

    # Security headers
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header X-Content-Type-Options nosniff;
    add_header X-Frame-Options DENY;

    # Static files — served by Nginx directly
    location /static/ {
        alias /app/staticfiles/;
        expires 7d;
        add_header Cache-Control "public, immutable";
    }

    # Media files — served by Nginx directly
    location /media/ {
        alias /app/media/;
        expires 30d;
        add_header Cache-Control "public, immutable";
    }

    # Wagtail admin & all dynamic content → Gunicorn
    location / {
        proxy_pass http://app:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_redirect off;
        client_max_body_size 100M;
    }
}
Nginx reverse proxy configuration with TLS termination, static/media caching, and Gunicorn forwarding

4.3 Set Up S3-Compatible Media Storage

Storing media files on the server's filesystem does not scale — containers restart without warning, and multi-server setups require shared storage. Use S3-compatible object storage instead:

First install the required packages:

pip install django-storages[s3] boto3
Install S3 storage packages

Then configure in settings.py (shown in section 3.2 above). Create the S3 bucket and apply a public-read policy for media served directly, or use CloudFront signed URLs for private content:

# AWS CLI — create bucket and enable public access for media
export AWS_PROFILE=wagtail-production
aws s3 mb s3://wagtail-media-prod --region us-east-1
aws s3api put-bucket-policy --bucket wagtail-media-prod --policy '{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": "*",
    "Action": "s3:GetObject",
    "Resource": "arn:aws:s3:::wagtail-media-prod/*"
  }]
}'
Create S3 bucket with public-read policy for media uploads

For local development with S3 compatibility, use MinIO running in Docker:

# Run MinIO locally for development/testing
docker run -p 9000:9000 -p 9001:9001 \
  -e MINIO_ROOT_USER=minioadmin \
  -e MINIO_ROOT_PASSWORD=minioadmin \
  quay.io/minio/minio server /data --console-address ':9001'

# Then configure Django:
# AWS_ACCESS_KEY_ID=minioadmin
# AWS_SECRET_ACCESS_KEY=minioadmin
# AWS_S3_ENDPOINT_URL=http://localhost:9000
# AWS_S3_REGION_NAME=us-east-1
MinIO Docker container for local S3-compatible development

4.4 Database Connection Pooling with PgBouncer

Without connection pooling, each Gunicorn worker opens its own PostgreSQL connection. With 4 workers and 4 threads, that is 16 simultaneous connections per app instance — and most platforms limit PostgreSQL to 10-20 connections by default. PgBouncer sits between Gunicorn and PostgreSQL, multiplexing connections:

# docker-compose.prod.yml (PgBouncer section)
version: '3.8'

services:
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: wagtail
      POSTGRES_USER: wagtail
      POSTGRES_PASSWORD: CHANGEME_POSTGRES_PASSWORD
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ['CMD-SHELL', 'pg_isready -U wagtail']
      interval: 10s
      timeout: 5s
      retries: 5

  pgbouncer:
    image: edoburu/pgbouncer:1.23
    environment:
      DB_USER: wagtail
      DB_PASSWORD: CHANGEME_POSTGRES_PASSWORD
      DB_HOST: db
      DB_PORT: '5432'
      DB_NAME: wagtail
      PGBOUNCER_POOL_MODE: transaction
      PGBOUNCER_DEFAULT_POOL_SIZE: '25'
      PGBOUNCER_MAX_CLIENT_CONN: '100'
    depends_on:
      db:
        condition: service_healthy

  redis:
    image: redis:7-alpine
    healthcheck:
      test: ['CMD', 'redis-cli', 'ping']
      interval: 5s
      timeout: 3s
      retries: 5

  app:
    build: .
    env_file: .env.production
    depends_on:
      pgbouncer:
        condition: service_started
      redis:
        condition: service_healthy
    volumes:
      - media:/app/media

  nginx:
    image: nginx:alpine
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
      - media:/app/media:ro
    ports:
      - '80:80'
      - '443:443'
    depends_on:
      - app

volumes:
  pgdata:
  media:
Docker Compose production stack — PostgreSQL, PgBouncer, Redis, Gunicorn, Nginx

Update DATABASE_URL to point to PgBouncer: postgres://wagtail:CHANGEME_PASSWORD@pgbouncer:6432/wagtail. Set CONN_MAX_AGE=600 so Django reuses connections efficiently.

4.5 Enable Caching with Redis

Wagtail benefits from caching at multiple levels. Configure Redis for page output caching, image rendition caching, and template fragment caching. Use the django-redis backend for advanced features like compression and key prefixing (shown in section 3.2). Install:

pip install django-redis
Install django-redis for advanced Redis cache support

Wagtail-specific cache configuration:

# Wagtail image rendition caching — store URLs for 30 days
WAGTAILIMAGES_CACHE = True
WAGTAILIMAGES_CACHE_BACKEND = 'renditions'

# Page output caching middleware for anonymous users
MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'whitenoise.middleware.WhiteNoiseMiddleware',
    'django.middleware.cache.UpdateCacheMiddleware',  # Early: writes cache
    'django.middleware.common.CommonMiddleware',
    'django.middleware.cache.FetchFromCacheMiddleware',  # Late: reads cache
    # ... rest of middleware
]

# Cache entire site for anonymous users
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_SECONDS = 600  # Cache pages for 10 minutes
CACHE_MIDDLEWARE_KEY_PREFIX = 'wagtail'
Wagtail-specific cache configuration for images and page output

4.6 Configure Gunicorn for Production

Gunicorn is the recommended WSGI server for Django/Wagtail. Create a config file for fine-grained control:

# gunicorn.conf.py
import multiprocessing

bind = '0.0.0.0:8000'
workers = multiprocessing.cpu_count() * 2 + 1  # Recommended formula
threads = 2
worker_class = 'gthread'  # Threaded workers for async-friendly code
timeout = 120
keepalive = 5
max_requests = 1000  # Restart worker after 1000 requests to prevent memory leaks
max_requests_jitter = 100  # Add jitter to avoid thundering herd restarts
accesslog = '-'
errorlog = '-'
loglevel = 'info'
Gunicorn configuration file — auto-calculated workers, threading, and memory management

Now run with the config file instead of CLI arguments:

gunicorn mysite.wsgi:application --config gunicorn.conf.py
Run Gunicorn with config file

4.7 CI/CD Pipeline with GitHub Actions

Automate testing and deployment so every push to the main branch runs tests and triggers a deployment:

# .github/workflows/deploy.yml
name: Deploy Wagtail

on:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16-alpine
        env:
          POSTGRES_DB: wagtail_test
          POSTGRES_USER: wagtail
          POSTGRES_PASSWORD: test_password
        ports:
          - 5432:5432
      redis:
        image: redis:7-alpine
        ports:
          - 6379:6379

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - name: Install dependencies
        run: |
          pip install -r requirements.txt
      - name: Run tests
        run: python manage.py test
        env:
          DATABASE_URL: postgres://wagtail:test_password@localhost:5432/wagtail_test
          REDIS_URL: redis://localhost:6379/0
          DJANGO_SECRET_KEY: test-key-not-for-production
          DJANGO_DEBUG: 'False'
      - name: Check migrations
        run: python manage.py makemigrations --check --dry-run

  deploy:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Deploy to Railway
        run: npx railway up
        env:
          RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }}
GitHub Actions CI/CD pipeline: tests and migration check on push, deploy on success

After deployment, run database migrations and rebuild the search index:

# Post-deploy steps (run via Railway shell or as a release command)
python manage.py migrate
python manage.py update_index
python manage.py clear_cache
Post-deployment commands for database migrations and search index rebuild

4.8 Environment Variable Management

Use the platform's secret manager to inject sensitive values rather than committing .env files. Structure your settings to fail fast on missing variables:

# settings.py — Safe env loading pattern

def env_or_raise(key):
    value = os.environ.get(key)
    if not value:
        raise RuntimeError(f'Missing required env variable: {key}')
    return value

# Required — fail immediately if not set
SECRET_KEY = env_or_raise('DJANGO_SECRET_KEY')
DATABASE_URL = env_or_raise('DATABASE_URL')

# Optional with defaults
DEBUG = os.environ.get('DJANGO_DEBUG', 'False').lower() == 'true'
ALLOWED_HOSTS = os.environ.get('ALLOWED_HOSTS', '').split(',') if os.environ.get('ALLOWED_HOSTS') else []
Environment variable loading pattern that fails fast on missing secrets

5. Common Errors & Solutions

Error 1: 'Invalid HTTP_HOST header' after deployment
Cause: ALLOWED_HOSTS is missing the domain or is set to ['*'] which Django rejects in DEBUG=False mode.
Solution: Add all domains to ALLOWED_HOSTS. Also set CSRF_TRUSTED_ORIGINS for POST requests.

Error 2: 502 Bad Gateway from Nginx
Cause: Gunicorn workers crashed, or the proxy_pass URL is unreachable (e.g., Nginx using localhost instead of the Docker service name).
Solution: Use proxy_pass http://app:8000; (the Docker service name). Check Gunicorn logs with docker compose logs app.

Error 3: Image renditions not showing after S3 migration
Cause: S3 bucket policy blocks public access, or old rendition URLs are cached in the database.
Solution: Ensure AWS_QUERYSTRING_AUTH=False for public media. Run python manage.py wagtail_update_image_renditions to regenerate all renditions.

Error 4: 'Too many connections' from PostgreSQL
Cause: Each Gunicorn worker opens its own connection, exhausting PostgreSQL's default max_connections (100).
Solution: Deploy PgBouncer in transaction pooling mode. Set CONN_MAX_AGE=600. Limit Gunicorn workers to (CPU_COUNT * 2) + 1.

Error 5: Static files 404 (white Wagtail admin page)
Cause: collectstatic was not run during build, or STATIC_ROOT does not match Nginx alias. WhiteNoise must be the first middleware after SecurityMiddleware.
Solution: Run python manage.py collectstatic --noinput in the Docker build stage. Ensure STATICFILES_STORAGE points to WhiteNoise's compressed manifest storage.

6. Summary Checklist

  • SECRET_KEY injected via environment variable, never hardcoded
  • DEBUG=False set in production
  • ALLOWED_HOSTS and CSRF_TRUSTED_ORIGINS configured with all domains
  • HTTPS enforced via SECURE_SSL_REDIRECT + proxy header
  • Static files collected and served through WhiteNoise or Nginx
  • Media files stored in S3-compatible object storage
  • PostgreSQL configured with CONN_MAX_AGE=600 and health checks
  • PgBouncer installed in transaction pooling mode
  • Redis cache enabled with dedicated Wagtail cache backends
  • Gunicorn configured with calculated workers, threads, and timeout
  • Nginx terminates TLS and serves static/media with cache headers
  • Docker multi-stage build with .dockerignore
  • CI/CD pipeline runs tests, checks migrations, and deploys on push
  • .env files gitignored; secrets injected via platform secret manager
  • Structured logging configured for production observability

7. Practice Exercise

Challenge: Deploy a Wagtail blog to production with full observability.

Take the Wagtail blog you have built in previous lessons and deploy it using the stack covered here. You must:

  1. Create a Dockerfile and docker-compose.yml with the full stack (PostgreSQL + PgBouncer + Redis + Gunicorn + Nginx)
  2. Configure S3-compatible storage for media files and verify image renditions work
  3. Set up environment variable injection using your platform's secret manager
  4. Enable Redis caching and confirm page output is cached (check for Cache-Control headers)
  5. Add a health check endpoint at /health/ that returns the status of all backend services
  6. Create a CI/CD pipeline that runs tests and deploys on each push to main
  7. Add structured logging and verify logs appear in your platform's log stream

Bonus: Add a /status/ admin-only page that shows cache hit rate, database connection pool usage, and S3 bucket size using Django system checks.

8. Next Steps

Congratulations! You have configured a production-grade Wagtail deployment with a full multi-layer stack. Your Wagtail site now handles traffic at scale with proper caching, connection pooling, and object storage.

The next topic in the Wagtail CMS series is Wagtail REST API Deep Dive: Building Custom Endpoints, Pagination, Filtering, and API Authentication. You will learn how to extend Wagtail's built-in API framework with custom serializers, implement pagination strategies, add filtering and search to API responses, and secure your API endpoints with token-based and permission-based authentication.

Further resources:

  • Wagtail Performance Tuning Guide — official docs on caching, database optimization, and CDN setup
  • django-storages Documentation — S3, GCS, Azure storage backends
  • WhiteNoise Documentation — serving static files in production
  • Gunicorn Settings Reference — all configuration options
  • Nginx Beginner's Guide — reverse proxy and load balancing
  • Django Deployment Checklist — official checklist for production readiness
  • Railway.app Wagtail Template — pre-configured Wagtail deployment template

Common pitfalls to avoid:

  • Do not rely on the Django development server (runserver) in production — always use Gunicorn behind a reverse proxy
  • Never set DEBUG=True in production — it exposes stack traces and can execute arbitrary code
  • Avoid storing media files on ephemeral container storage — containers restart without warning and media is lost permanently
  • Do not skip load testing before launch — use k6 or locust to simulate traffic and identify bottlenecks
  • Never run update_index during peak traffic without a lock mechanism — schedule it during off-peak hours
Tags: wagtail, django, deployment, production, docker, nginx, gunicorn, postgresql, pgbouncer, s3,

Gataya Med

DevOps Engineer & Backend Developer. Sharing insights on cloud, automation, and scalable systems.

gatayaMed View Profile @https://github.com/gatayaMed

Comments (0)

Sarah Chen July 15, 2026

This is exactly what I needed! The initContainer approach solved our migration issues completely. Thanks for the detailed guide!

Reply

Leave a Comment