learning_ai_invt_trdg/docs/DOCKER_DEPLOYMENT_GUIDE.md

Docker Deployment Guide - Shared Base Image Strategy

Purpose: Comprehensive guide for Docker deployment across ByteLyst products using shared base images to minimize disk duplication and optimize build times.

Target Audience: DevOps engineers, product teams deploying ByteLyst products.

Last Updated: 2026-05-09

---

Overview

This guide documents the shared base image strategy for Docker deployment across all ByteLyst products. This approach minimizes disk space duplication by building a single base image containing all @bytelyst/* shared packages, which all product images then inherit from.

Problem Statement

Without shared base images:

• Each Docker image includes duplicate @bytelyst/* packages (~100-200MB per image)
• 3 products = ~300-600MB wasted disk space
• 20+ products = ~2-4GB wasted disk space
• Slower builds (reinstalling same packages in each image)
• Inconsistent package versions across products

With shared base images:

• Common packages stored once in base image (~200MB)
• Each product image only contains product-specific code (~50MB)
• Docker layer sharing means packages stored once on disk
• Faster builds (base image cached)
• Consistent package versions across all products

Architecture

bytelyst-common-base:latest (Built once, ~200MB)
├── node:22-slim
├── pnpm@10.6.5
├── @bytelyst/* packages (63 packages, shared layer)
└── Build tools (typescript, etc.)

Product Images (inherit from base):
├── learning_ai_notes-backend:latest (~50MB additional)
│   └── Inherits from bytelyst-common-base
│       └── + backend code
├── learning_ai_clock-backend:latest (~50MB additional)
│   └── Inherits from bytelyst-common-base
│       └── + backend code
└── learning_ai_invt_trdg-backend:latest (~50MB additional)
    └── Inherits from bytelyst-common-base
        └── + backend code

Disk usage comparison:

• Without base: 3 products × ~250MB = ~750MB
• With base: ~200MB (base) + 3 × ~50MB = ~350MB
• Savings: ~400MB (53% reduction)

---

Part 1: Base Image Strategy

1.1 Separate Base Images for Backend and Web

Rationale: Backend (Fastify) and web (Next.js) have different dependency sets. Separate base images optimize for each use case.

Base Images:

• bytelyst-common-base-backend:latest - Backend dependencies (Fastify, Zod, jose, etc.)
• bytelyst-common-base-web:latest - Web dependencies (Next.js, React, Tailwind, etc.)

1.2 Base Image Versioning Strategy

Semantic Versioning:

• Format: bytelyst-common-base-backend:1.0.0
• Increment major version for breaking changes
• Increment minor version for new packages
• Increment patch version for bug fixes

Date-Based Tags (for automation):

• Format: bytelyst-common-base-backend:20260509
• Useful for automated builds
• Easy to identify build date

Always maintain :latest tag:

• Points to most recent stable version
• Used by default in product Dockerfiles
• Update only after testing

1.3 Base Image Location

Build in common plat repo:

• Location: learning_ai_common_plat/Dockerfile.backend-base
• Build context: Common plat repo (has access to all packages)
• Push to registry: Docker Hub, Gitea container registry, or local registry

Registry Options:

1. Docker Hub - Public, but requires organization account
2. Gitea Container Registry - Private, integrated with existing Gitea
3. Local Registry - Fastest for single VM, requires registry setup
4. GitHub Container Registry - If using GitHub Actions

Recommendation: Use Gitea Container Registry for privacy and integration.

---

Part 2: Base Image Implementation

2.1 Backend Base Image

File: learning_ai_common_plat/Dockerfile.backend-base

# ── Stage 1: Builder ─────────────────────────────────────────────────────
FROM node:22-slim AS builder

# Install pnpm
RUN corepack enable && corepack prepare pnpm@10.6.5 --activate

WORKDIR /app

# Copy workspace files
COPY .npmrc .pnpmfile.cjs pnpm-workspace.yaml pnpm-lock.yaml* ./

# Copy all packages
COPY packages/ ./packages/

# Build all @bytelyst/* packages
RUN pnpm -r --filter './packages/*' build

# ── Stage 2: Production Base ───────────────────────────────────────────────
FROM node:22-slim

# Metadata
LABEL org.opencontainers.image.title="ByteLyst Common Base - Backend"
LABEL org.opencontainers.image.description="Base image with @bytelyst/* backend packages"
LABEL org.opencontainers.image.vendor="ByteLyst"
LABEL org.opencontainers.image.version="1.0.0"

# Install pnpm
RUN corepack enable && corepack prepare pnpm@10.6.5 --activate

WORKDIR /app

# Copy workspace files
COPY .npmrc .pnpmfile.cjs pnpm-workspace.yaml pnpm-lock.yaml* ./

# Copy packages from builder
COPY --from=builder /app/packages/ ./packages/
COPY --from=builder /app/packages/*/dist/ ./packages/*/dist/

# Install all @bytelyst/* packages (production only)
RUN pnpm install -r --filter './packages/*' --prod --ignore-scripts

# Verify installation
RUN pnpm list --depth=0 | grep @bytelyst

# Set default environment
ENV NODE_ENV=production
ENV BYTELYST_PACKAGE_SOURCE=common-plat

WORKDIR /app

Build command:

cd learning_ai_common_plat
docker build -f Dockerfile.backend-base \
  -t bytelyst-common-base-backend:1.0.0 \
  -t bytelyst-common-base-backend:latest \
  -t bytelyst-common-base-backend:$(date +%Y%m%d) \
  .

2.2 Web Base Image

File: learning_ai_common_plat/Dockerfile.web-base

# ── Stage 1: Builder ─────────────────────────────────────────────────────
FROM node:22-slim AS builder

# Install pnpm
RUN corepack enable && corepack prepare pnpm@10.6.5 --activate

WORKDIR /app

# Copy workspace files
COPY .npmrc .pnpmfile.cjs pnpm-workspace.yaml pnpm-lock.yaml* ./

# Copy all packages
COPY packages/ ./packages/

# Build all @bytelyst/* packages
RUN pnpm -r --filter './packages/*' build

# ── Stage 2: Production Base ───────────────────────────────────────────────
FROM node:22-slim

# Metadata
LABEL org.opencontainers.image.title="ByteLyst Common Base - Web"
LABEL org.opencontainers.image.description="Base image with @bytelyst/* web packages"
LABEL org.opencontainers.image.vendor="ByteLyst"
LABEL org.opencontainers.image.version="1.0.0"

# Install pnpm
RUN corepack enable && corepack prepare pnpm@10.6.5 --activate

WORKDIR /app

# Copy workspace files
COPY .npmrc .pnpmfile.cjs pnpm-workspace.yaml pnpm-lock.yaml* ./

# Copy packages from builder
COPY --from=builder /app/packages/ ./packages/
COPY --from=builder /app/packages/*/dist/ ./packages/*/dist/

# Install all @bytelyst/* packages (production only)
RUN pnpm install -r --filter './packages/*' --prod --ignore-scripts

# Verify installation
RUN pnpm list --depth=0 | grep @bytelyst

# Set default environment
ENV NODE_ENV=production
ENV BYTELYST_PACKAGE_SOURCE=common-plat
ENV NEXT_TELEMETRY_DISABLED=1

WORKDIR /app

Build command:

cd learning_ai_common_plat
docker build -f Dockerfile.web-base \
  -t bytelyst-common-base-web:1.0.0 \
  -t bytelyst-common-base-web:latest \
  -t bytelyst-common-base-web:$(date +%Y%m%d) \
  .

2.3 Development Base Image (Optional)

For local development with devDependencies:

# Dockerfile.dev-base
FROM node:22-slim

RUN corepack enable && corepack prepare pnpm@10.6.5 --activate

WORKDIR /app

COPY .npmrc .pnpmfile.cjs pnpm-workspace.yaml pnpm-lock.yaml* ./
COPY packages/ ./packages/

# Install with devDependencies
RUN pnpm install -r --filter './packages/*' --ignore-scripts

ENV NODE_ENV=development

---

Part 3: Product Dockerfiles

3.1 Backend Dockerfile (Using Base Image)

File: learning_ai_notes/backend/Dockerfile

# ── Stage 1: Build ───────────────────────────────────────────────────────
FROM bytelyst-common-base-backend:latest AS builder

WORKDIR /app/backend

# Copy backend package files
COPY backend/package.json ./package.json
COPY backend/tsconfig.json ./tsconfig.json

# Install backend-specific dependencies only
RUN pnpm install --prod --ignore-scripts

# Copy source code
COPY backend/src/ ./src/
COPY shared/ ../shared/

# Build backend
RUN pnpm run build

# ── Stage 2: Production ───────────────────────────────────────────────────
FROM bytelyst-common-base-backend:latest

WORKDIR /app/backend

# Copy backend package files
COPY backend/package.json ./package.json

# Install backend-specific dependencies
RUN pnpm install --prod --ignore-scripts

# Copy built artifacts from builder
COPY --from=builder /app/backend/dist ./dist
COPY --from=builder /app/backend/node_modules ./node_modules
COPY shared/ ../shared/

# Verify installation
RUN pnpm list --depth=0

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
  CMD node -e "require('http').get('http://localhost:4016/health', (r) => {process.exit(r.statusCode === 200 ? 0 : 1)})"

EXPOSE 4016
CMD ["node", "dist/server.js"]

3.2 Web Dockerfile (Using Base Image)

File: learning_ai_notes/web/Dockerfile

# ── Stage 1: Build ───────────────────────────────────────────────────────
FROM bytelyst-common-base-web:latest AS builder

WORKDIR /app/web

# Copy web package files
COPY web/package.json ./package.json
COPY web/next.config.ts ./next.config.ts
COPY web/tsconfig.json ./tsconfig.json
COPY web/next-env.d.ts ./next-env.d.ts

# Install web-specific dependencies only
RUN pnpm install --prod --ignore-scripts

# Copy source code
COPY web/src/ ./src/
COPY web/public/ ./public/  # if exists
COPY shared/ ../shared/

# Build web
RUN pnpm run build

# ── Stage 2: Production ───────────────────────────────────────────────────
FROM bytelyst-common-base-web:latest

WORKDIR /app/web

# Copy web package files
COPY web/package.json ./package.json

# Install web-specific dependencies (production only)
RUN pnpm install --prod --ignore-scripts

# Copy built artifacts from builder
COPY --from=builder /app/web/.next/standalone ./
COPY --from=builder /app/web/.next/static ./.next/static
COPY shared/ ../shared/

# Environment
ENV NODE_ENV=production
ENV PORT=3000
ENV HOSTNAME="0.0.0.0"

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
  CMD node -e "require('http').get('http://localhost:3000', (r) => {process.exit(r.statusCode === 200 ? 0 : 1)})"

EXPOSE 3000
CMD ["node", "server.js"]

---

Part 4: Build and Deployment Workflow

4.1 Base Image Build Workflow

When to rebuild base image:

1. When @bytelyst/* packages are updated
2. When Node.js version changes
3. When pnpm version changes
4. When security vulnerabilities are found in base layers

Manual build:

cd learning_ai_common_plat

# Build backend base
docker build -f Dockerfile.backend-base \
  -t bytelyst-common-base-backend:1.0.0 \
  -t bytelyst-common-base-backend:latest \
  .

# Build web base
docker build -f Dockerfile.web-base \
  -t bytelyst-common-base-web:1.0.0 \
  -t bytelyst-common-base-web:latest \
  .

Push to registry (Gitea example):

# Tag for Gitea registry
docker tag bytelyst-common-base-backend:latest \
  gitea.example.com/bytelyst/common-base-backend:1.0.0

# Push
docker push gitea.example.com/bytelyst/common-base-backend:1.0.0

4.2 Automated Base Image Builds

GitHub Actions workflow:

# .github/workflows/build-base-image.yml
name: Build Base Images

on:
  push:
    paths:
      - 'packages/**'
      - 'pnpm-lock.yaml'
  workflow_dispatch:

jobs:
  build-backend-base:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: corepack enable && corepack prepare pnpm@10.6.5 --activate
      - run: pnpm install
      - name: Build backend base image
        run: |
          docker build -f Dockerfile.backend-base \
            -t bytelyst-common-base-backend:$(date +%Y%m%d) \
            -t bytelyst-common-base-backend:latest \
            .          
      - name: Push to registry
        run: |
          echo ${{ secrets.GITEA_TOKEN }} | docker login gitea.example.com -u username --password-stdin
          docker push gitea.example.com/bytelyst/common-base-backend:latest          

  build-web-base:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: corepack enable && corepack prepare pnpm@10.6.5 --activate
      - run: pnpm install
      - name: Build web base image
        run: |
          docker build -f Dockerfile.web-base \
            -t bytelyst-common-base-web:$(date +%Y%m%d) \
            -t bytelyst-common-base-web:latest \
            .          
      - name: Push to registry
        run: |
          echo ${{ secrets.GITEA_TOKEN }} | docker login gitea.example.com -u username --password-stdin
          docker push gitea.example.com/bytelyst/common-base-web:latest          

4.3 Product Image Build Workflow

Build product image (using cached base):

cd learning_ai_notes

# Base image is already pulled and cached
docker build -f backend/Dockerfile -t learning_ai_notes-backend:latest .
docker build -f web/Dockerfile -t learning_ai_notes-web:latest .

Deployment script integration:

#!/bin/bash
# deploy-notes.sh

# Ensure latest base image is pulled
docker pull bytelyst-common-base-backend:latest
docker pull bytelyst-common-base-web:latest

# Build product images (uses cached base)
docker compose build

# Deploy
docker compose up -d

---

Part 5: Optimization Strategies

5.1 Layer Caching

Order layers by change frequency:

1. Base image layers (rarely change) - cached
2. package.json (changes when deps change) - rebuild
3. Source code (changes frequently) - rebuild
4. Build artifacts (changes with source) - rebuild

Example:

# Good - source code after dependencies
COPY package.json ./
RUN pnpm install
COPY src/ ./src/
RUN pnpm run build

# Bad - source code before dependencies (invalidates cache)
COPY src/ ./src/
COPY package.json ./
RUN pnpm install

5.2 Multi-Stage Builds

Minimize final image size:

• Builder stage includes all build tools
• Production stage only includes runtime dependencies
• Copy only built artifacts, not source code

Example:

# Builder stage - has TypeScript, devDependencies
FROM node:22-slim AS builder
RUN pnpm install
RUN pnpm run build

# Production stage - only runtime
FROM node:22-slim
COPY --from=builder /app/dist ./dist
# No TypeScript, no devDependencies

5.3 .dockerignore

Exclude unnecessary files:

# Dependencies
node_modules
npm-debug.log
.pnpm-debug.log

# Build outputs
dist
.next
build

# Git
.git
.gitignore

# IDE
.vscode
.idea
*.swp
*.swo

# OS
.DS_Store
Thumbs.db

# Tests
**/*.test.ts
**/*.spec.ts
**/__tests__/
coverage/

# Docs
README.md
docs/
*.md

# Misc
.env
.env.local
*.log

5.4 BuildKit Cache

Use BuildKit for faster builds:

# Enable BuildKit
export DOCKER_BUILDKIT=1

# Use cache mount for dependencies
docker build \
  --mount type=cache,target=/root/.npm \
  -f Dockerfile .

In Dockerfile:

# Cache mount for pnpm store
RUN --mount=type=cache,target=/root/.local/share/pnpm/store \
  pnpm install --prod

5.5 Parallel Builds

Build backend and web in parallel:

# Use docker compose
docker compose build

# Or manual parallel build
docker build -f backend/Dockerfile -t backend:latest . &
docker build -f web/Dockerfile -t web:latest . &
wait

---

Part 6: Local Development

6.1 Development Without Docker

Use .pnpmfile.cjs for local development:

// .pnpmfile.cjs
const PACKAGE_SOURCE = process.env.BYTELYST_PACKAGE_SOURCE || 'common-plat';

if (PACKAGE_SOURCE === 'common-plat') {
  // Resolve from local common plat
  // Works for local development without Docker
}

Benefits:

• Fast iteration (no Docker rebuild)
• Hot reload works
• Use local common plat changes immediately
• No need to rebuild base image

6.2 Development With Docker

Use development base image:

FROM bytelyst-common-base-backend:dev AS dev

# Mount source code for hot reload
COPY backend/src/ ./src/
CMD ["pnpm", "run", "dev"]

docker-compose.dev.yml:

services:
  backend:
    build:
      target: dev
    volumes:
      - ./backend/src:/app/backend/src
    environment:
      - NODE_ENV=development

---

Part 7: Troubleshooting

7.1 Base Image Not Found

Error:

ERROR: failed to solve: bytelyst-common-base-backend:latest: not found

Solution:

# Pull base image first
docker pull bytelyst-common-base-backend:latest

# Or build locally
cd learning_ai_common_plat
docker build -f Dockerfile.backend-base -t bytelyst-common-base-backend:latest .

7.2 Package Version Conflicts

Error:

ERR_PNPM_PEER_DEP_ISSUES  Unmet peer dependencies

Solution:

• Rebuild base image with updated packages
• Ensure product package.json versions align with base image
• Use pnpm overrides in product package.json if needed

7.3 Layer Cache Not Working

Symptoms:

• Builds are slow despite no changes
• All layers are being rebuilt

Solution:

# Clear Docker cache
docker builder prune -a

# Rebuild with BuildKit
export DOCKER_BUILDKIT=1
docker build -f Dockerfile .

7.4 Disk Space Issues

Check disk usage:

# Docker system disk usage
docker system df

# Image sizes
docker images

# Clean up unused images
docker image prune -a

# Clean up build cache
docker builder prune

---

Part 8: Security Considerations

8.1 Base Image Scanning

Scan for vulnerabilities:

# Use Trivy
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
  aquasec/trivy image bytelyst-common-base-backend:latest

# Use Docker Scout
docker scout quickview bytelyst-common-base-backend:latest
docker scout cves bytelyst-common-base-backend:latest

8.2 Non-Root User

Run as non-root in production:

# Create non-root user
RUN useradd -m -u 1000 appuser

# Change ownership
RUN chown -R appuser:appuser /app

# Switch to non-root
USER appuser

8.3 Minimal Base Image

Use slim instead of alpine:

• Alpine has glibc compatibility issues
• Slim is official Node.js variant
• Only slightly larger than alpine

Example:

# Good - official slim
FROM node:22-slim

# Avoid - alpine has compatibility issues
FROM node:22-alpine

---

Part 9: Monitoring and Maintenance

9.1 Base Image Version Tracking

Track base image versions in products:

# Pin to specific version for reproducibility
FROM bytelyst-common-base-backend:1.0.0

# Update :latest after testing
# FROM bytelyst-common-base-backend:latest

Audit product images for base version:

# Check base image version
docker history learning_ai_notes-backend:latest | grep bytelyst-common-base

9.2 Update Workflow

1. Update packages in common plat

cd learning_ai_common_plat
# Update packages
pnpm update @bytelyst/*

2. Rebuild base images

docker build -f Dockerfile.backend-base -t bytelyst-common-base-backend:1.1.0 .
docker build -f Dockerfile.web-base -t bytelyst-common-base-web:1.1.0 .

3. Test with one product

cd learning_ai_notes
# Update Dockerfile to use new base
# FROM bytelyst-common-base-backend:1.1.0
docker build -f backend/Dockerfile .
docker compose up -d
# Test functionality

4. Roll out to other products

• Update Dockerfiles in other products
• Rebuild and deploy
• Monitor for issues

5. Update :latest tag

docker tag bytelyst-common-base-backend:1.1.0 bytelyst-common-base-backend:latest
docker push bytelyst-common-base-backend:latest

9.3 Rollback Strategy

If base image update causes issues:

# Revert product Dockerfile to previous base version
FROM bytelyst-common-base-backend:1.0.0  # Revert from 1.1.0

# Rebuild product
docker build -f backend/Dockerfile .
docker compose up -d

---

Part 10: Migration Checklist

For New Products

• Use FROM bytelyst-common-base-backend:latest in backend Dockerfile
• Use FROM bytelyst-common-base-web:latest in web Dockerfile
• Remove direct installation of @bytelyst/* packages
• Only install product-specific dependencies
• Add health checks
• Add metadata labels
• Update deployment script to pull base image
• Test build and deployment
• Document base image version in README

For Existing Products

• Create new Dockerfile using base image (keep old as backup)
• Update deployment script to use new Dockerfile
• Test in staging environment
• Monitor for issues
• Deploy to production
• Remove old Dockerfile
• Document migration

---

Part 11: CI/CD Integration

11.1 GitHub Actions Example

# .github/workflows/deploy.yml
name: Deploy Product

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Pull base images
        run: |
          docker pull bytelyst-common-base-backend:latest
          docker pull bytelyst-common-base-web:latest          

      - name: Build product images
        run: |
          docker compose build          

      - name: Deploy
        run: |
          docker compose up -d          

      - name: Health check
        run: |
          sleep 30
          curl -f http://localhost:4016/health || exit 1          

11.2 GitLab CI Example

# .gitlab-ci.yml
deploy:
  stage: deploy
  image: docker:latest
  services:
    - docker:dind
  script:
    - docker pull bytelyst-common-base-backend:latest
    - docker compose build
    - docker compose up -d
  only:
    - main

---

Appendix A: Quick Reference

Build Commands

# Build backend base image
cd learning_ai_common_plat
docker build -f Dockerfile.backend-base -t bytelyst-common-base-backend:latest .

# Build web base image
docker build -f Dockerfile.web-base -t bytelyst-common-base-web:latest .

# Build product image
cd learning_ai_notes
docker build -f backend/Dockerfile -t learning_ai_notes-backend:latest .

# Build with docker compose
docker compose build

Deploy Commands

# Pull latest base images
docker pull bytelyst-common-base-backend:latest
docker pull bytelyst-common-base-web:latest

# Build and deploy
docker compose build
docker compose up -d

# Check status
docker compose ps
docker compose logs

Troubleshooting Commands

# Check image sizes
docker images

# Check disk usage
docker system df

# Clean up
docker system prune -a
docker builder prune -a

# Inspect image layers
docker history learning_ai_notes-backend:latest

# Check base image version
docker history learning_ai_notes-backend:latest | grep bytelyst-common-base

---

Appendix B: File Structure

learning_ai_common_plat/
├── Dockerfile.backend-base    # Backend base image definition
├── Dockerfile.web-base        # Web base image definition
├── Dockerfile.dev-base        # Development base image (optional)
└── packages/                  # @bytelyst/* packages

learning_ai_notes/
├── backend/
│   └── Dockerfile             # Uses bytelyst-common-base-backend:latest
├── web/
│   └── Dockerfile             # Uses bytelyst-common-base-web:latest
└── docker-compose.yml

learning_ai_clock/
├── backend/
│   └── Dockerfile             # Uses bytelyst-common-base-backend:latest
├── web/
│   └── Dockerfile             # Uses bytelyst-common-base-web:latest
└── docker-compose.yml

---

Appendix C: Environment Variables

Base Image Variables

# Package source (for .pnpmfile.cjs)
BYTELYST_PACKAGE_SOURCE=common-plat  # Use local packages (default)
BYTELYST_PACKAGE_SOURCE=vendor       # Use vendor directory
BYTELYST_PACKAGE_SOURCE=gitea        # Use Gitea registry

# Common plat root (for local development)
BYTELYST_COMMON_PLAT_ROOT=/path/to/learning_ai_common_plat

Build Arguments

# Dockerfile build arguments
ARG BYTELYST_PACKAGE_SOURCE=common-plat
ARG BYTELYST_COMMON_PLAT_ROOT

---

Summary

This shared base image strategy provides:

• 53% disk space reduction for 3 products (~400MB savings)
• Faster builds through layer caching
• Consistent package versions across all products
• Simplified updates - rebuild base, all products benefit
• Better maintainability - single source of truth for common packages

Key benefits:

• ✅ Minimal disk duplication
• ✅ Faster builds (base image cached)
• ✅ Easy to update packages
• ✅ Clean separation (common vs product code)
• ✅ Reusable across all current and future products

Next steps:

1. Create base image Dockerfiles in common plat repo
2. Build and test base images
3. Update product Dockerfiles to use base images
4. Update deployment scripts
5. Document and roll out to other products