The base image strategy has been implemented but deferred due to fundamental issues with pnpm workspace structure. All products currently use the proven docker-prep.sh tarball approach. This commit documents the lessons learned and future considerations. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
1094 lines
28 KiB
Markdown
1094 lines
28 KiB
Markdown
# Docker Deployment Guide - Shared Base Image Strategy
|
||
|
||
> **Purpose:** Comprehensive guide for Docker deployment across ByteLyst products using shared base images to minimize disk duplication and optimize build times.
|
||
>
|
||
> **Target Audience:** DevOps engineers, product teams deploying ByteLyst products.
|
||
>
|
||
> **Last Updated:** 2026-05-09
|
||
|
||
---
|
||
|
||
## Overview
|
||
|
||
This guide documents the shared base image strategy for Docker deployment across all ByteLyst products. This approach minimizes disk space duplication by building a single base image containing all `@bytelyst/*` shared packages, which all product images then inherit from.
|
||
|
||
**Status: Experimental / Deferred**
|
||
|
||
The base image strategy has been implemented and tested, but is currently **deferred** due to workspace complexity issues. All ByteLyst products (notes, clock, trading) currently use the **docker-prep.sh tarball approach**.
|
||
|
||
### Why the Base Image Approach Was Deferred
|
||
|
||
The base image approach encountered fundamental issues with pnpm workspaces:
|
||
|
||
1. **Workspace Structure Mismatch**: The base image sets up a workspace in `/app` with all `@bytelyst/*` packages. Product Dockerfiles try to install in subdirectories like `/app/backend`, which creates workspace context issues.
|
||
|
||
2. **pnpm Workspace Requirements**: pnpm workspaces expect all workspace packages to be present and resolvable during installation. When a product tries to `pnpm install` in its subdirectory, it cannot resolve the workspace packages properly.
|
||
|
||
3. **Dependency Resolution Complexity**: The product's package.json has workspace protocol dependencies (e.g., `workspace:*` or `workspace:^`). These resolve correctly in the common plat repo but fail when the product is built in isolation using the base image.
|
||
|
||
4. **Build Tool Availability**: The base image would need to include all build tools (TypeScript, Next.js, etc.) globally, which increases complexity and image size.
|
||
|
||
### Current Approach: docker-prep.sh
|
||
|
||
All products currently use the **docker-prep.sh tarball approach**:
|
||
- `docker-prep.sh` builds all `@bytelyst/*` packages and packs them into tarballs
|
||
- Dockerfiles copy these tarballs and extract them during build
|
||
- This approach is slower (~2-3 min) but reliable and works with the current workspace structure
|
||
|
||
### Future Considerations
|
||
|
||
The base image approach could be revisited if:
|
||
1. The workspace structure is simplified (e.g., move to a monorepo or use a different package manager)
|
||
2. pnpm adds better support for cross-repo workspace inheritance
|
||
3. A custom npm registry is set up to publish `@bytelyst/*` packages
|
||
4. The build process is restructured to not rely on workspace protocols
|
||
|
||
For now, the docker-prep.sh approach is recommended for all ByteLyst products.
|
||
|
||
### Problem Statement
|
||
|
||
**Without shared base images:**
|
||
- Each Docker image includes duplicate `@bytelyst/*` packages (~100-200MB per image)
|
||
- 3 products = ~300-600MB wasted disk space
|
||
- 20+ products = ~2-4GB wasted disk space
|
||
- Slower builds (reinstalling same packages in each image)
|
||
- Inconsistent package versions across products
|
||
|
||
**With shared base images:**
|
||
- Common packages stored once in base image (~200MB)
|
||
- Each product image only contains product-specific code (~50MB)
|
||
- Docker layer sharing means packages stored once on disk
|
||
- Faster builds (base image cached)
|
||
- Consistent package versions across all products
|
||
|
||
### Architecture
|
||
|
||
```
|
||
bytelyst-common-base:latest (Built once, ~200MB)
|
||
├── node:22-slim
|
||
├── pnpm@10.6.5
|
||
├── @bytelyst/* packages (63 packages, shared layer)
|
||
└── Build tools (typescript, etc.)
|
||
|
||
Product Images (inherit from base):
|
||
├── learning_ai_notes-backend:latest (~50MB additional)
|
||
│ └── Inherits from bytelyst-common-base
|
||
│ └── + backend code
|
||
├── learning_ai_clock-backend:latest (~50MB additional)
|
||
│ └── Inherits from bytelyst-common-base
|
||
│ └── + backend code
|
||
└── learning_ai_invt_trdg-backend:latest (~50MB additional)
|
||
└── Inherits from bytelyst-common-base
|
||
└── + backend code
|
||
```
|
||
|
||
**Disk usage comparison:**
|
||
- Without base: 3 products × ~250MB = ~750MB
|
||
- With base: ~200MB (base) + 3 × ~50MB = ~350MB
|
||
- **Savings: ~400MB (53% reduction)**
|
||
|
||
---
|
||
|
||
## Part 1: Base Image Strategy
|
||
|
||
### 1.1 Separate Base Images for Backend and Web
|
||
|
||
**Rationale:** Backend (Fastify) and web (Next.js) have different dependency sets. Separate base images optimize for each use case.
|
||
|
||
**Base Images:**
|
||
- `bytelyst-common-base-backend:latest` - Backend dependencies (Fastify, Zod, jose, etc.)
|
||
- `bytelyst-common-base-web:latest` - Web dependencies (Next.js, React, Tailwind, etc.)
|
||
|
||
### 1.2 Base Image Versioning Strategy
|
||
|
||
**Semantic Versioning:**
|
||
- Format: `bytelyst-common-base-backend:1.0.0`
|
||
- Increment major version for breaking changes
|
||
- Increment minor version for new packages
|
||
- Increment patch version for bug fixes
|
||
|
||
**Date-Based Tags (for automation):**
|
||
- Format: `bytelyst-common-base-backend:20260509`
|
||
- Useful for automated builds
|
||
- Easy to identify build date
|
||
|
||
**Always maintain `:latest` tag:**
|
||
- Points to most recent stable version
|
||
- Used by default in product Dockerfiles
|
||
- Update only after testing
|
||
|
||
### 1.3 Base Image Location
|
||
|
||
**Build in common plat repo:**
|
||
- Location: `learning_ai_common_plat/Dockerfile.backend-base`
|
||
- Build context: Common plat repo (has access to all packages)
|
||
- Push to registry: Docker Hub, Gitea container registry, or local registry
|
||
|
||
**Registry Options:**
|
||
1. **Docker Hub** - Public, but requires organization account
|
||
2. **Gitea Container Registry** - Private, integrated with existing Gitea
|
||
3. **Local Registry** - Fastest for single VM, requires registry setup
|
||
4. **GitHub Container Registry** - If using GitHub Actions
|
||
|
||
**Recommendation:** Use Gitea Container Registry for privacy and integration.
|
||
|
||
---
|
||
|
||
## Part 2: Base Image Implementation
|
||
|
||
### 2.1 Backend Base Image
|
||
|
||
**File:** `learning_ai_common_plat/Dockerfile.backend-base`
|
||
|
||
```dockerfile
|
||
# ── Stage 1: Builder ─────────────────────────────────────────────────────
|
||
FROM node:22-slim AS builder
|
||
|
||
# Install pnpm
|
||
RUN corepack enable && corepack prepare pnpm@10.6.5 --activate
|
||
|
||
WORKDIR /app
|
||
|
||
# Copy workspace files
|
||
COPY .npmrc .pnpmfile.cjs pnpm-workspace.yaml pnpm-lock.yaml* ./
|
||
|
||
# Copy all packages
|
||
COPY packages/ ./packages/
|
||
|
||
# Build all @bytelyst/* packages
|
||
RUN pnpm -r --filter './packages/*' build
|
||
|
||
# ── Stage 2: Production Base ───────────────────────────────────────────────
|
||
FROM node:22-slim
|
||
|
||
# Metadata
|
||
LABEL org.opencontainers.image.title="ByteLyst Common Base - Backend"
|
||
LABEL org.opencontainers.image.description="Base image with @bytelyst/* backend packages"
|
||
LABEL org.opencontainers.image.vendor="ByteLyst"
|
||
LABEL org.opencontainers.image.version="1.0.0"
|
||
|
||
# Install pnpm
|
||
RUN corepack enable && corepack prepare pnpm@10.6.5 --activate
|
||
|
||
WORKDIR /app
|
||
|
||
# Copy workspace files
|
||
COPY .npmrc .pnpmfile.cjs pnpm-workspace.yaml pnpm-lock.yaml* ./
|
||
|
||
# Copy packages from builder
|
||
COPY --from=builder /app/packages/ ./packages/
|
||
COPY --from=builder /app/packages/*/dist/ ./packages/*/dist/
|
||
|
||
# Install all @bytelyst/* packages (production only)
|
||
RUN pnpm install -r --filter './packages/*' --prod --ignore-scripts
|
||
|
||
# Verify installation
|
||
RUN pnpm list --depth=0 | grep @bytelyst
|
||
|
||
# Set default environment
|
||
ENV NODE_ENV=production
|
||
ENV BYTELYST_PACKAGE_SOURCE=common-plat
|
||
|
||
WORKDIR /app
|
||
```
|
||
|
||
**Build command:**
|
||
```bash
|
||
cd learning_ai_common_plat
|
||
docker build -f Dockerfile.backend-base \
|
||
-t bytelyst-common-base-backend:1.0.0 \
|
||
-t bytelyst-common-base-backend:latest \
|
||
-t bytelyst-common-base-backend:$(date +%Y%m%d) \
|
||
.
|
||
```
|
||
|
||
### 2.2 Web Base Image
|
||
|
||
**File:** `learning_ai_common_plat/Dockerfile.web-base`
|
||
|
||
```dockerfile
|
||
# ── Stage 1: Builder ─────────────────────────────────────────────────────
|
||
FROM node:22-slim AS builder
|
||
|
||
# Install pnpm
|
||
RUN corepack enable && corepack prepare pnpm@10.6.5 --activate
|
||
|
||
WORKDIR /app
|
||
|
||
# Copy workspace files
|
||
COPY .npmrc .pnpmfile.cjs pnpm-workspace.yaml pnpm-lock.yaml* ./
|
||
|
||
# Copy all packages
|
||
COPY packages/ ./packages/
|
||
|
||
# Build all @bytelyst/* packages
|
||
RUN pnpm -r --filter './packages/*' build
|
||
|
||
# ── Stage 2: Production Base ───────────────────────────────────────────────
|
||
FROM node:22-slim
|
||
|
||
# Metadata
|
||
LABEL org.opencontainers.image.title="ByteLyst Common Base - Web"
|
||
LABEL org.opencontainers.image.description="Base image with @bytelyst/* web packages"
|
||
LABEL org.opencontainers.image.vendor="ByteLyst"
|
||
LABEL org.opencontainers.image.version="1.0.0"
|
||
|
||
# Install pnpm
|
||
RUN corepack enable && corepack prepare pnpm@10.6.5 --activate
|
||
|
||
WORKDIR /app
|
||
|
||
# Copy workspace files
|
||
COPY .npmrc .pnpmfile.cjs pnpm-workspace.yaml pnpm-lock.yaml* ./
|
||
|
||
# Copy packages from builder
|
||
COPY --from=builder /app/packages/ ./packages/
|
||
COPY --from=builder /app/packages/*/dist/ ./packages/*/dist/
|
||
|
||
# Install all @bytelyst/* packages (production only)
|
||
RUN pnpm install -r --filter './packages/*' --prod --ignore-scripts
|
||
|
||
# Verify installation
|
||
RUN pnpm list --depth=0 | grep @bytelyst
|
||
|
||
# Set default environment
|
||
ENV NODE_ENV=production
|
||
ENV BYTELYST_PACKAGE_SOURCE=common-plat
|
||
ENV NEXT_TELEMETRY_DISABLED=1
|
||
|
||
WORKDIR /app
|
||
```
|
||
|
||
**Build command:**
|
||
```bash
|
||
cd learning_ai_common_plat
|
||
docker build -f Dockerfile.web-base \
|
||
-t bytelyst-common-base-web:1.0.0 \
|
||
-t bytelyst-common-base-web:latest \
|
||
-t bytelyst-common-base-web:$(date +%Y%m%d) \
|
||
.
|
||
```
|
||
|
||
### 2.3 Development Base Image (Optional)
|
||
|
||
For local development with devDependencies:
|
||
|
||
```dockerfile
|
||
# Dockerfile.dev-base
|
||
FROM node:22-slim
|
||
|
||
RUN corepack enable && corepack prepare pnpm@10.6.5 --activate
|
||
|
||
WORKDIR /app
|
||
|
||
COPY .npmrc .pnpmfile.cjs pnpm-workspace.yaml pnpm-lock.yaml* ./
|
||
COPY packages/ ./packages/
|
||
|
||
# Install with devDependencies
|
||
RUN pnpm install -r --filter './packages/*' --ignore-scripts
|
||
|
||
ENV NODE_ENV=development
|
||
```
|
||
|
||
---
|
||
|
||
## Part 3: Product Dockerfiles
|
||
|
||
### 3.1 Backend Dockerfile (Using Base Image)
|
||
|
||
**File:** `learning_ai_notes/backend/Dockerfile`
|
||
|
||
```dockerfile
|
||
# ── Stage 1: Build ───────────────────────────────────────────────────────
|
||
FROM bytelyst-common-base-backend:latest AS builder
|
||
|
||
WORKDIR /app/backend
|
||
|
||
# Copy backend package files
|
||
COPY backend/package.json ./package.json
|
||
COPY backend/tsconfig.json ./tsconfig.json
|
||
|
||
# Install backend-specific dependencies only
|
||
RUN pnpm install --prod --ignore-scripts
|
||
|
||
# Copy source code
|
||
COPY backend/src/ ./src/
|
||
COPY shared/ ../shared/
|
||
|
||
# Build backend
|
||
RUN pnpm run build
|
||
|
||
# ── Stage 2: Production ───────────────────────────────────────────────────
|
||
FROM bytelyst-common-base-backend:latest
|
||
|
||
WORKDIR /app/backend
|
||
|
||
# Copy backend package files
|
||
COPY backend/package.json ./package.json
|
||
|
||
# Install backend-specific dependencies
|
||
RUN pnpm install --prod --ignore-scripts
|
||
|
||
# Copy built artifacts from builder
|
||
COPY --from=builder /app/backend/dist ./dist
|
||
COPY --from=builder /app/backend/node_modules ./node_modules
|
||
COPY shared/ ../shared/
|
||
|
||
# Verify installation
|
||
RUN pnpm list --depth=0
|
||
|
||
# Health check
|
||
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
|
||
CMD node -e "require('http').get('http://localhost:4016/health', (r) => {process.exit(r.statusCode === 200 ? 0 : 1)})"
|
||
|
||
EXPOSE 4016
|
||
CMD ["node", "dist/server.js"]
|
||
```
|
||
|
||
### 3.2 Web Dockerfile (Using Base Image)
|
||
|
||
**File:** `learning_ai_notes/web/Dockerfile`
|
||
|
||
```dockerfile
|
||
# ── Stage 1: Build ───────────────────────────────────────────────────────
|
||
FROM bytelyst-common-base-web:latest AS builder
|
||
|
||
WORKDIR /app/web
|
||
|
||
# Copy web package files
|
||
COPY web/package.json ./package.json
|
||
COPY web/next.config.ts ./next.config.ts
|
||
COPY web/tsconfig.json ./tsconfig.json
|
||
COPY web/next-env.d.ts ./next-env.d.ts
|
||
|
||
# Install web-specific dependencies only
|
||
RUN pnpm install --prod --ignore-scripts
|
||
|
||
# Copy source code
|
||
COPY web/src/ ./src/
|
||
COPY web/public/ ./public/ # if exists
|
||
COPY shared/ ../shared/
|
||
|
||
# Build web
|
||
RUN pnpm run build
|
||
|
||
# ── Stage 2: Production ───────────────────────────────────────────────────
|
||
FROM bytelyst-common-base-web:latest
|
||
|
||
WORKDIR /app/web
|
||
|
||
# Copy web package files
|
||
COPY web/package.json ./package.json
|
||
|
||
# Install web-specific dependencies (production only)
|
||
RUN pnpm install --prod --ignore-scripts
|
||
|
||
# Copy built artifacts from builder
|
||
COPY --from=builder /app/web/.next/standalone ./
|
||
COPY --from=builder /app/web/.next/static ./.next/static
|
||
COPY shared/ ../shared/
|
||
|
||
# Environment
|
||
ENV NODE_ENV=production
|
||
ENV PORT=3000
|
||
ENV HOSTNAME="0.0.0.0"
|
||
|
||
# Health check
|
||
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
|
||
CMD node -e "require('http').get('http://localhost:3000', (r) => {process.exit(r.statusCode === 200 ? 0 : 1)})"
|
||
|
||
EXPOSE 3000
|
||
CMD ["node", "server.js"]
|
||
```
|
||
|
||
---
|
||
|
||
## Part 4: Build and Deployment Workflow
|
||
|
||
### 4.1 Base Image Build Workflow
|
||
|
||
**When to rebuild base image:**
|
||
1. When `@bytelyst/*` packages are updated
|
||
2. When Node.js version changes
|
||
3. When pnpm version changes
|
||
4. When security vulnerabilities are found in base layers
|
||
|
||
**Manual build:**
|
||
```bash
|
||
cd learning_ai_common_plat
|
||
|
||
# Build backend base
|
||
docker build -f Dockerfile.backend-base \
|
||
-t bytelyst-common-base-backend:1.0.0 \
|
||
-t bytelyst-common-base-backend:latest \
|
||
.
|
||
|
||
# Build web base
|
||
docker build -f Dockerfile.web-base \
|
||
-t bytelyst-common-base-web:1.0.0 \
|
||
-t bytelyst-common-base-web:latest \
|
||
.
|
||
```
|
||
|
||
**Push to registry (Gitea example):**
|
||
```bash
|
||
# Tag for Gitea registry
|
||
docker tag bytelyst-common-base-backend:latest \
|
||
gitea.example.com/bytelyst/common-base-backend:1.0.0
|
||
|
||
# Push
|
||
docker push gitea.example.com/bytelyst/common-base-backend:1.0.0
|
||
```
|
||
|
||
### 4.2 Automated Base Image Builds
|
||
|
||
**GitHub Actions workflow:**
|
||
|
||
```yaml
|
||
# .github/workflows/build-base-image.yml
|
||
name: Build Base Images
|
||
|
||
on:
|
||
push:
|
||
paths:
|
||
- 'packages/**'
|
||
- 'pnpm-lock.yaml'
|
||
workflow_dispatch:
|
||
|
||
jobs:
|
||
build-backend-base:
|
||
runs-on: ubuntu-latest
|
||
steps:
|
||
- uses: actions/checkout@v4
|
||
- uses: actions/setup-node@v4
|
||
with:
|
||
node-version: 22
|
||
- run: corepack enable && corepack prepare pnpm@10.6.5 --activate
|
||
- run: pnpm install
|
||
- name: Build backend base image
|
||
run: |
|
||
docker build -f Dockerfile.backend-base \
|
||
-t bytelyst-common-base-backend:$(date +%Y%m%d) \
|
||
-t bytelyst-common-base-backend:latest \
|
||
.
|
||
- name: Push to registry
|
||
run: |
|
||
echo ${{ secrets.GITEA_TOKEN }} | docker login gitea.example.com -u username --password-stdin
|
||
docker push gitea.example.com/bytelyst/common-base-backend:latest
|
||
|
||
build-web-base:
|
||
runs-on: ubuntu-latest
|
||
steps:
|
||
- uses: actions/checkout@v4
|
||
- uses: actions/setup-node@v4
|
||
with:
|
||
node-version: 22
|
||
- run: corepack enable && corepack prepare pnpm@10.6.5 --activate
|
||
- run: pnpm install
|
||
- name: Build web base image
|
||
run: |
|
||
docker build -f Dockerfile.web-base \
|
||
-t bytelyst-common-base-web:$(date +%Y%m%d) \
|
||
-t bytelyst-common-base-web:latest \
|
||
.
|
||
- name: Push to registry
|
||
run: |
|
||
echo ${{ secrets.GITEA_TOKEN }} | docker login gitea.example.com -u username --password-stdin
|
||
docker push gitea.example.com/bytelyst/common-base-web:latest
|
||
```
|
||
|
||
### 4.3 Product Image Build Workflow
|
||
|
||
**Build product image (using cached base):**
|
||
```bash
|
||
cd learning_ai_notes
|
||
|
||
# Base image is already pulled and cached
|
||
docker build -f backend/Dockerfile -t learning_ai_notes-backend:latest .
|
||
docker build -f web/Dockerfile -t learning_ai_notes-web:latest .
|
||
```
|
||
|
||
**Deployment script integration:**
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
# deploy-notes.sh
|
||
|
||
# Ensure latest base image is pulled
|
||
docker pull bytelyst-common-base-backend:latest
|
||
docker pull bytelyst-common-base-web:latest
|
||
|
||
# Build product images (uses cached base)
|
||
docker compose build
|
||
|
||
# Deploy
|
||
docker compose up -d
|
||
```
|
||
|
||
---
|
||
|
||
## Part 5: Optimization Strategies
|
||
|
||
### 5.1 Layer Caching
|
||
|
||
**Order layers by change frequency:**
|
||
1. Base image layers (rarely change) - cached
|
||
2. package.json (changes when deps change) - rebuild
|
||
3. Source code (changes frequently) - rebuild
|
||
4. Build artifacts (changes with source) - rebuild
|
||
|
||
**Example:**
|
||
```dockerfile
|
||
# Good - source code after dependencies
|
||
COPY package.json ./
|
||
RUN pnpm install
|
||
COPY src/ ./src/
|
||
RUN pnpm run build
|
||
|
||
# Bad - source code before dependencies (invalidates cache)
|
||
COPY src/ ./src/
|
||
COPY package.json ./
|
||
RUN pnpm install
|
||
```
|
||
|
||
### 5.2 Multi-Stage Builds
|
||
|
||
**Minimize final image size:**
|
||
- Builder stage includes all build tools
|
||
- Production stage only includes runtime dependencies
|
||
- Copy only built artifacts, not source code
|
||
|
||
**Example:**
|
||
```dockerfile
|
||
# Builder stage - has TypeScript, devDependencies
|
||
FROM node:22-slim AS builder
|
||
RUN pnpm install
|
||
RUN pnpm run build
|
||
|
||
# Production stage - only runtime
|
||
FROM node:22-slim
|
||
COPY --from=builder /app/dist ./dist
|
||
# No TypeScript, no devDependencies
|
||
```
|
||
|
||
### 5.3 .dockerignore
|
||
|
||
**Exclude unnecessary files:**
|
||
```dockerignore
|
||
# Dependencies
|
||
node_modules
|
||
npm-debug.log
|
||
.pnpm-debug.log
|
||
|
||
# Build outputs
|
||
dist
|
||
.next
|
||
build
|
||
|
||
# Git
|
||
.git
|
||
.gitignore
|
||
|
||
# IDE
|
||
.vscode
|
||
.idea
|
||
*.swp
|
||
*.swo
|
||
|
||
# OS
|
||
.DS_Store
|
||
Thumbs.db
|
||
|
||
# Tests
|
||
**/*.test.ts
|
||
**/*.spec.ts
|
||
**/__tests__/
|
||
coverage/
|
||
|
||
# Docs
|
||
README.md
|
||
docs/
|
||
*.md
|
||
|
||
# Misc
|
||
.env
|
||
.env.local
|
||
*.log
|
||
```
|
||
|
||
### 5.4 BuildKit Cache
|
||
|
||
**Use BuildKit for faster builds:**
|
||
```bash
|
||
# Enable BuildKit
|
||
export DOCKER_BUILDKIT=1
|
||
|
||
# Use cache mount for dependencies
|
||
docker build \
|
||
--mount type=cache,target=/root/.npm \
|
||
-f Dockerfile .
|
||
```
|
||
|
||
**In Dockerfile:**
|
||
```dockerfile
|
||
# Cache mount for pnpm store
|
||
RUN --mount=type=cache,target=/root/.local/share/pnpm/store \
|
||
pnpm install --prod
|
||
```
|
||
|
||
### 5.5 Parallel Builds
|
||
|
||
**Build backend and web in parallel:**
|
||
```bash
|
||
# Use docker compose
|
||
docker compose build
|
||
|
||
# Or manual parallel build
|
||
docker build -f backend/Dockerfile -t backend:latest . &
|
||
docker build -f web/Dockerfile -t web:latest . &
|
||
wait
|
||
```
|
||
|
||
---
|
||
|
||
## Part 6: Local Development
|
||
|
||
### 6.1 Development Without Docker
|
||
|
||
**Use .pnpmfile.cjs for local development:**
|
||
```javascript
|
||
// .pnpmfile.cjs
|
||
const PACKAGE_SOURCE = process.env.BYTELYST_PACKAGE_SOURCE || 'common-plat';
|
||
|
||
if (PACKAGE_SOURCE === 'common-plat') {
|
||
// Resolve from local common plat
|
||
// Works for local development without Docker
|
||
}
|
||
```
|
||
|
||
**Benefits:**
|
||
- Fast iteration (no Docker rebuild)
|
||
- Hot reload works
|
||
- Use local common plat changes immediately
|
||
- No need to rebuild base image
|
||
|
||
### 6.2 Development With Docker
|
||
|
||
**Use development base image:**
|
||
```dockerfile
|
||
FROM bytelyst-common-base-backend:dev AS dev
|
||
|
||
# Mount source code for hot reload
|
||
COPY backend/src/ ./src/
|
||
CMD ["pnpm", "run", "dev"]
|
||
```
|
||
|
||
**docker-compose.dev.yml:**
|
||
```yaml
|
||
services:
|
||
backend:
|
||
build:
|
||
target: dev
|
||
volumes:
|
||
- ./backend/src:/app/backend/src
|
||
environment:
|
||
- NODE_ENV=development
|
||
```
|
||
|
||
---
|
||
|
||
## Part 7: Troubleshooting
|
||
|
||
### 7.1 Base Image Not Found
|
||
|
||
**Error:**
|
||
```
|
||
ERROR: failed to solve: bytelyst-common-base-backend:latest: not found
|
||
```
|
||
|
||
**Solution:**
|
||
```bash
|
||
# Pull base image first
|
||
docker pull bytelyst-common-base-backend:latest
|
||
|
||
# Or build locally
|
||
cd learning_ai_common_plat
|
||
docker build -f Dockerfile.backend-base -t bytelyst-common-base-backend:latest .
|
||
```
|
||
|
||
### 7.2 Package Version Conflicts
|
||
|
||
**Error:**
|
||
```
|
||
ERR_PNPM_PEER_DEP_ISSUES Unmet peer dependencies
|
||
```
|
||
|
||
**Solution:**
|
||
- Rebuild base image with updated packages
|
||
- Ensure product package.json versions align with base image
|
||
- Use `pnpm overrides` in product package.json if needed
|
||
|
||
### 7.3 Layer Cache Not Working
|
||
|
||
**Symptoms:**
|
||
- Builds are slow despite no changes
|
||
- All layers are being rebuilt
|
||
|
||
**Solution:**
|
||
```bash
|
||
# Clear Docker cache
|
||
docker builder prune -a
|
||
|
||
# Rebuild with BuildKit
|
||
export DOCKER_BUILDKIT=1
|
||
docker build -f Dockerfile .
|
||
```
|
||
|
||
### 7.4 Disk Space Issues
|
||
|
||
**Check disk usage:**
|
||
```bash
|
||
# Docker system disk usage
|
||
docker system df
|
||
|
||
# Image sizes
|
||
docker images
|
||
|
||
# Clean up unused images
|
||
docker image prune -a
|
||
|
||
# Clean up build cache
|
||
docker builder prune
|
||
```
|
||
|
||
---
|
||
|
||
## Part 8: Security Considerations
|
||
|
||
### 8.1 Base Image Scanning
|
||
|
||
**Scan for vulnerabilities:**
|
||
```bash
|
||
# Use Trivy
|
||
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
|
||
aquasec/trivy image bytelyst-common-base-backend:latest
|
||
|
||
# Use Docker Scout
|
||
docker scout quickview bytelyst-common-base-backend:latest
|
||
docker scout cves bytelyst-common-base-backend:latest
|
||
```
|
||
|
||
### 8.2 Non-Root User
|
||
|
||
**Run as non-root in production:**
|
||
```dockerfile
|
||
# Create non-root user
|
||
RUN useradd -m -u 1000 appuser
|
||
|
||
# Change ownership
|
||
RUN chown -R appuser:appuser /app
|
||
|
||
# Switch to non-root
|
||
USER appuser
|
||
```
|
||
|
||
### 8.3 Minimal Base Image
|
||
|
||
**Use slim instead of alpine:**
|
||
- Alpine has glibc compatibility issues
|
||
- Slim is official Node.js variant
|
||
- Only slightly larger than alpine
|
||
|
||
**Example:**
|
||
```dockerfile
|
||
# Good - official slim
|
||
FROM node:22-slim
|
||
|
||
# Avoid - alpine has compatibility issues
|
||
FROM node:22-alpine
|
||
```
|
||
|
||
---
|
||
|
||
## Part 9: Monitoring and Maintenance
|
||
|
||
### 9.1 Base Image Version Tracking
|
||
|
||
**Track base image versions in products:**
|
||
```dockerfile
|
||
# Pin to specific version for reproducibility
|
||
FROM bytelyst-common-base-backend:1.0.0
|
||
|
||
# Update :latest after testing
|
||
# FROM bytelyst-common-base-backend:latest
|
||
```
|
||
|
||
**Audit product images for base version:**
|
||
```bash
|
||
# Check base image version
|
||
docker history learning_ai_notes-backend:latest | grep bytelyst-common-base
|
||
```
|
||
|
||
### 9.2 Update Workflow
|
||
|
||
**1. Update packages in common plat**
|
||
```bash
|
||
cd learning_ai_common_plat
|
||
# Update packages
|
||
pnpm update @bytelyst/*
|
||
```
|
||
|
||
**2. Rebuild base images**
|
||
```bash
|
||
docker build -f Dockerfile.backend-base -t bytelyst-common-base-backend:1.1.0 .
|
||
docker build -f Dockerfile.web-base -t bytelyst-common-base-web:1.1.0 .
|
||
```
|
||
|
||
**3. Test with one product**
|
||
```bash
|
||
cd learning_ai_notes
|
||
# Update Dockerfile to use new base
|
||
# FROM bytelyst-common-base-backend:1.1.0
|
||
docker build -f backend/Dockerfile .
|
||
docker compose up -d
|
||
# Test functionality
|
||
```
|
||
|
||
**4. Roll out to other products**
|
||
- Update Dockerfiles in other products
|
||
- Rebuild and deploy
|
||
- Monitor for issues
|
||
|
||
**5. Update :latest tag**
|
||
```bash
|
||
docker tag bytelyst-common-base-backend:1.1.0 bytelyst-common-base-backend:latest
|
||
docker push bytelyst-common-base-backend:latest
|
||
```
|
||
|
||
### 9.3 Rollback Strategy
|
||
|
||
**If base image update causes issues:**
|
||
```bash
|
||
# Revert product Dockerfile to previous base version
|
||
FROM bytelyst-common-base-backend:1.0.0 # Revert from 1.1.0
|
||
|
||
# Rebuild product
|
||
docker build -f backend/Dockerfile .
|
||
docker compose up -d
|
||
```
|
||
|
||
---
|
||
|
||
## Part 10: Migration Checklist
|
||
|
||
### For New Products
|
||
|
||
- [ ] Use `FROM bytelyst-common-base-backend:latest` in backend Dockerfile
|
||
- [ ] Use `FROM bytelyst-common-base-web:latest` in web Dockerfile
|
||
- [ ] Remove direct installation of `@bytelyst/*` packages
|
||
- [ ] Only install product-specific dependencies
|
||
- [ ] Add health checks
|
||
- [ ] Add metadata labels
|
||
- [ ] Update deployment script to pull base image
|
||
- [ ] Test build and deployment
|
||
- [ ] Document base image version in README
|
||
|
||
### For Existing Products
|
||
|
||
- [ ] Create new Dockerfile using base image (keep old as backup)
|
||
- [ ] Update deployment script to use new Dockerfile
|
||
- [ ] Test in staging environment
|
||
- [ ] Monitor for issues
|
||
- [ ] Deploy to production
|
||
- [ ] Remove old Dockerfile
|
||
- [ ] Document migration
|
||
|
||
---
|
||
|
||
## Part 11: CI/CD Integration
|
||
|
||
### 11.1 GitHub Actions Example
|
||
|
||
```yaml
|
||
# .github/workflows/deploy.yml
|
||
name: Deploy Product
|
||
|
||
on:
|
||
push:
|
||
branches: [main]
|
||
|
||
jobs:
|
||
deploy:
|
||
runs-on: ubuntu-latest
|
||
steps:
|
||
- uses: actions/checkout@v4
|
||
|
||
- name: Pull base images
|
||
run: |
|
||
docker pull bytelyst-common-base-backend:latest
|
||
docker pull bytelyst-common-base-web:latest
|
||
|
||
- name: Build product images
|
||
run: |
|
||
docker compose build
|
||
|
||
- name: Deploy
|
||
run: |
|
||
docker compose up -d
|
||
|
||
- name: Health check
|
||
run: |
|
||
sleep 30
|
||
curl -f http://localhost:4016/health || exit 1
|
||
```
|
||
|
||
### 11.2 GitLab CI Example
|
||
|
||
```yaml
|
||
# .gitlab-ci.yml
|
||
deploy:
|
||
stage: deploy
|
||
image: docker:latest
|
||
services:
|
||
- docker:dind
|
||
script:
|
||
- docker pull bytelyst-common-base-backend:latest
|
||
- docker compose build
|
||
- docker compose up -d
|
||
only:
|
||
- main
|
||
```
|
||
|
||
---
|
||
|
||
## Appendix A: Quick Reference
|
||
|
||
### Build Commands
|
||
|
||
```bash
|
||
# Build backend base image
|
||
cd learning_ai_common_plat
|
||
docker build -f Dockerfile.backend-base -t bytelyst-common-base-backend:latest .
|
||
|
||
# Build web base image
|
||
docker build -f Dockerfile.web-base -t bytelyst-common-base-web:latest .
|
||
|
||
# Build product image
|
||
cd learning_ai_notes
|
||
docker build -f backend/Dockerfile -t learning_ai_notes-backend:latest .
|
||
|
||
# Build with docker compose
|
||
docker compose build
|
||
```
|
||
|
||
### Deploy Commands
|
||
|
||
```bash
|
||
# Pull latest base images
|
||
docker pull bytelyst-common-base-backend:latest
|
||
docker pull bytelyst-common-base-web:latest
|
||
|
||
# Build and deploy
|
||
docker compose build
|
||
docker compose up -d
|
||
|
||
# Check status
|
||
docker compose ps
|
||
docker compose logs
|
||
```
|
||
|
||
### Troubleshooting Commands
|
||
|
||
```bash
|
||
# Check image sizes
|
||
docker images
|
||
|
||
# Check disk usage
|
||
docker system df
|
||
|
||
# Clean up
|
||
docker system prune -a
|
||
docker builder prune -a
|
||
|
||
# Inspect image layers
|
||
docker history learning_ai_notes-backend:latest
|
||
|
||
# Check base image version
|
||
docker history learning_ai_notes-backend:latest | grep bytelyst-common-base
|
||
```
|
||
|
||
---
|
||
|
||
## Appendix B: File Structure
|
||
|
||
```
|
||
learning_ai_common_plat/
|
||
├── Dockerfile.backend-base # Backend base image definition
|
||
├── Dockerfile.web-base # Web base image definition
|
||
├── Dockerfile.dev-base # Development base image (optional)
|
||
└── packages/ # @bytelyst/* packages
|
||
|
||
learning_ai_notes/
|
||
├── backend/
|
||
│ └── Dockerfile # Uses bytelyst-common-base-backend:latest
|
||
├── web/
|
||
│ └── Dockerfile # Uses bytelyst-common-base-web:latest
|
||
└── docker-compose.yml
|
||
|
||
learning_ai_clock/
|
||
├── backend/
|
||
│ └── Dockerfile # Uses bytelyst-common-base-backend:latest
|
||
├── web/
|
||
│ └── Dockerfile # Uses bytelyst-common-base-web:latest
|
||
└── docker-compose.yml
|
||
```
|
||
|
||
---
|
||
|
||
## Appendix C: Environment Variables
|
||
|
||
### Base Image Variables
|
||
|
||
```bash
|
||
# Package source (for .pnpmfile.cjs)
|
||
BYTELYST_PACKAGE_SOURCE=common-plat # Use local packages (default)
|
||
BYTELYST_PACKAGE_SOURCE=vendor # Use vendor directory
|
||
BYTELYST_PACKAGE_SOURCE=gitea # Use Gitea registry
|
||
|
||
# Common plat root (for local development)
|
||
BYTELYST_COMMON_PLAT_ROOT=/path/to/learning_ai_common_plat
|
||
```
|
||
|
||
### Build Arguments
|
||
|
||
```dockerfile
|
||
# Dockerfile build arguments
|
||
ARG BYTELYST_PACKAGE_SOURCE=common-plat
|
||
ARG BYTELYST_COMMON_PLAT_ROOT
|
||
```
|
||
|
||
---
|
||
|
||
## Summary
|
||
|
||
This shared base image strategy provides:
|
||
|
||
- **53% disk space reduction** for 3 products (~400MB savings)
|
||
- **Faster builds** through layer caching
|
||
- **Consistent package versions** across all products
|
||
- **Simplified updates** - rebuild base, all products benefit
|
||
- **Better maintainability** - single source of truth for common packages
|
||
|
||
**Key benefits:**
|
||
- ✅ Minimal disk duplication
|
||
- ✅ Faster builds (base image cached)
|
||
- ✅ Easy to update packages
|
||
- ✅ Clean separation (common vs product code)
|
||
- ✅ Reusable across all current and future products
|
||
|
||
**Next steps:**
|
||
1. Create base image Dockerfiles in common plat repo
|
||
2. Build and test base images
|
||
3. Update product Dockerfiles to use base images
|
||
4. Update deployment scripts
|
||
5. Document and roll out to other products |