A single container is rarely enough. Modern applications need databases, caches, message queues, and multiple services. Docker Compose lets you define everything in a YAML file and start all services with one command. In this lesson, you'll learn to orchestrate complex applications with Compose.

1. Learning Objectives

By the end of this lesson, you will be able to:

  • Write docker-compose.yml for multi-container apps
  • Define services, networks, and volumes
  • Use environment variables and .env files
  • Set up development and production Compose files
  • Scale services horizontally
  • Debug multi-container applications

2. Why This Matters

Real-world scenario: Your app needs a web server, database, cache, and message queue. Without Compose, you'd run 4 docker run commands with complex networking. With Compose, you define all services in one file and run docker compose up -d.

3. Core Concepts

Basic Docker Compose

# docker-compose.yml
version: '3.8'

services:
  web:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./html:/usr/share/nginx/html
    restart: unless-stopped

  redis:
    image: redis:alpine
    ports:
      - "6379:6379"
    restart: unless-stopped

# Commands:
# docker compose up -d      # Start all services
# docker compose down       # Stop all services
# docker compose ps         # List services
# docker compose logs       # View logs
# docker compose logs web   # View logs for specific service
Basic docker-compose.yml

Complete Web Application Stack

version: '3.8'

services:
  # PostgreSQL Database
  db:
    image: postgres:15-alpine
    environment:
      POSTGRES_USER: ${DB_USER:-postgres}
      POSTGRES_PASSWORD: ${DB_PASSWORD:-password}
      POSTGRES_DB: ${DB_NAME:-myapp}
    volumes:
      - postgres_data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped
    networks:
      - app-network

  # Redis Cache
  redis:
    image: redis:7-alpine
    command: redis-server --appendonly yes --requirepass ${REDIS_PASSWORD:-}
    volumes:
      - redis_data:/data
    ports:
      - "6379:6379"
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped
    networks:
      - app-network

  # Backend API (Python Flask)
  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
      target: development
    environment:
      - DATABASE_URL=postgresql://${DB_USER}:${DB_PASSWORD}@db:5432/${DB_NAME}
      - REDIS_URL=redis://:${REDIS_PASSWORD:-}@redis:6379
      - FLASK_ENV=development
      - FLASK_DEBUG=1
    volumes:
      - ./backend:/app
      - /app/__pycache__
    ports:
      - "5000:5000"
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_healthy
    restart: unless-stopped
    networks:
      - app-network

  # Frontend (React with hot reload)
  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile
      target: development
    environment:
      - REACT_APP_API_URL=http://localhost:5000
    volumes:
      - ./frontend:/app
      - /app/node_modules
    ports:
      - "3000:3000"
    depends_on:
      - backend
    restart: unless-stopped
    networks:
      - app-network

  # Nginx Reverse Proxy
  nginx:
    image: nginx:alpine
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - ./nginx/conf.d:/etc/nginx/conf.d:ro
    ports:
      - "80:80"
      - "443:443"
    depends_on:
      - frontend
      - backend
    restart: unless-stopped
    networks:
      - app-network

  # Worker for background tasks
  worker:
    build:
      context: ./backend
      dockerfile: Dockerfile.worker
    environment:
      - DATABASE_URL=postgresql://${DB_USER}:${DB_PASSWORD}@db:5432/${DB_NAME}
      - REDIS_URL=redis://:${REDIS_PASSWORD:-}@redis:6379
    volumes:
      - ./backend:/app
    depends_on:
      - db
      - redis
    restart: unless-stopped
    networks:
      - app-network

volumes:
  postgres_data:
  redis_data:

networks:
  app-network:
    driver: bridge
Complete web application stack with Compose

Environment Variables

# .env file (automatically loaded by docker compose)
DB_USER=myuser
DB_PASSWORD=Sup3rSecure!
DB_NAME=production_db
REDIS_PASSWORD=RedisPass123
APP_ENV=production

# docker-compose.yml can reference them
environment:
  POSTGRES_USER: ${DB_USER}
  POSTGRES_PASSWORD: ${DB_PASSWORD}

# Override on command line
DB_PASSWORD=secret123 docker compose up

# Multiple env files
docker compose --env-file .env.production up
Environment variables in Compose

Development vs Production

# docker-compose.yml (base)
version: '3.8'
services:
  app:
    build: .
    environment:
      - APP_ENV=${APP_ENV}

# docker-compose.override.yml (auto-loaded in dev)
version: '3.8'
services:
  app:
    volumes:
      - .:/app
    command: npm run dev
    ports:
      - "3000:3000"

# docker-compose.prod.yml (production override)
version: '3.8'
services:
  app:
    restart: unless-stopped
    command: npm start
    ports:
      - "80:3000"

# Usage:
# docker compose up -f docker-compose.yml -f docker-compose.prod.yml up
# Or use -f to specify files
# Or put production settings in separate file
Environment-specific Compose files

4. Complete Project: Development Environment

# Full development environment with hot reload, debugging, and testing

version: '3.8'

services:
  # Database with admin interface
  db:
    image: postgres:15-alpine
    environment:
      POSTGRES_USER: devuser
      POSTGRES_PASSWORD: devpass
      POSTGRES_DB: devdb
    ports:
      - "5432:5432"
    volumes:
      - postgres_dev:/var/lib/postgresql/data

  # Adminer - Database management UI
  adminer:
    image: adminer:latest
    ports:
      - "8080:8080"
    depends_on:
      - db

  # Redis Commander - Redis UI
  redis-commander:
    image: rediscommander/redis-commander:latest
    environment:
      - REDIS_HOSTS=local:redis:6379
    ports:
      - "8081:8081"
    depends_on:
      - redis

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    volumes:
      - redis_dev:/data

  # MailHog - Email testing
  mailhog:
    image: mailhog/mailhog:latest
    ports:
      - "1025:1025"  # SMTP
      - "8025:8025"  # Web UI

  # MinIO - S3-compatible storage
  minio:
    image: minio/minio:latest
    command: server /data --console-address ":9001"
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minioadmin
    ports:
      - "9000:9000"
      - "9001:9001"
    volumes:
      - minio_data:/data

  # Elasticsearch for search
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ports:
      - "9200:9200"
    volumes:
      - elasticsearch_data:/usr/share/elasticsearch/data

  # Kibana for Elasticsearch UI
  kibana:
    image: docker.elastic.co/kibana/kibana:8.11.0
    ports:
      - "5601:5601"
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
    depends_on:
      - elasticsearch

  # Application with debugger
  app:
    build:
      context: .
      dockerfile: Dockerfile.dev
    environment:
      - DATABASE_URL=postgresql://devuser:devpass@db:5432/devdb
      - REDIS_URL=redis://redis:6379
      - MINIO_ENDPOINT=minio:9000
      - MAILHOST=mailhog
      - DEBUG=true
    volumes:
      - .:/app
      - /app/node_modules
      - /app/.venv
    ports:
      - "8000:8000"
      - "5678:5678"  # Python debugger
    depends_on:
      - db
      - redis
      - minio
      - mailhog
    command: python -m debugpy --listen 0.0.0.0:5678 -m uvicorn main:app --reload --host 0.0.0.0 --port 8000

  # Testing environment
  test:
    build:
      context: .
      dockerfile: Dockerfile.test
    environment:
      - DATABASE_URL=postgresql://testuser:testpass@test-db:5432/testdb
      - REDIS_URL=redis://test-redis:6379
    depends_on:
      - test-db
      - test-redis
    command: pytest --cov --cov-report=html

  test-db:
    image: postgres:15-alpine
    environment:
      POSTGRES_USER: testuser
      POSTGRES_PASSWORD: testpass
      POSTGRES_DB: testdb

  test-redis:
    image: redis:7-alpine

volumes:
  postgres_dev:
  redis_dev:
  minio_data:
  elasticsearch_data:

networks:
  default:
    name: dev-network
Complete development environment

5. Useful Compose Commands

# Basic commands
docker compose up -d              # Start in background
docker compose down               # Stop and remove containers
docker compose restart            # Restart all services
docker compose ps                 # List services

# Building
docker compose build              # Build images
docker compose up --build         # Rebuild and start
docker compose build --no-cache   # Build without cache

# Logs
docker compose logs               # All logs
docker compose logs -f            # Follow logs
docker compose logs web           # Specific service
docker compose logs --tail=50     # Last 50 lines

# Scaling
docker compose up -d --scale web=3   # Run 3 web instances

# Execute commands
docker compose exec web bash      # Open shell in container
docker compose exec web python manage.py migrate

# Copy files
docker compose cp web:/app/log.txt ./log.txt

# Environment
docker compose config             # View resolved config
docker compose --env-file .env.prod up

# Profiles (selective service starting)
# docker-compose.yml:
# services:
#   monitoring:
#     profiles: ["monitoring"]
#
docker compose --profile monitoring up

# Cleanup
docker compose down -v            # Remove volumes too
docker compose rm                 # Remove stopped containers
Essential Docker Compose commands

6. Common Errors & Solutions

7. Summary Checklist

8. Next Steps

Next lesson: Docker Lesson 5.4: Docker Networking & Volumes

Tags: docker, docker-compose, multi-container, devops, orchestration

Gataya Med

DevOps Engineer & Backend Developer. Sharing insights on cloud, automation, and scalable systems.

gatayaMed View Profile @https://github.com/gatayaMed

Comments (0)

Sarah Chen February 4, 2025

This is exactly what I needed! The initContainer approach solved our migration issues completely. Thanks for the detailed guide!

Reply

Leave a Comment