From ca085ce63e9d02d88ea23b8a1b06f6d71c8951c2 Mon Sep 17 00:00:00 2001 From: root Date: Sat, 9 May 2026 21:42:58 +0000 Subject: [PATCH] Add production deployment scripts for ByteLyst services - deploy-invttrdg.sh: Single-repo deployment for trading platform * Dirty checks (uncommitted changes, unpushed commits) * Pre-deployment smoke tests (backend contracts, web DOM tests) * Docker build and deployment (backend + web) * Post-deployment validation (health checks, endpoint verification) * Comprehensive smoke testing for production-grade deployment - deploy-all.sh: Multi-repo deployment orchestration * Deploy all 4 production repos or specific ones * Same safety checks and deployment process for each repo * Health checks for all services - DEPLOYMENT_GUIDE.md: Complete deployment documentation * Usage instructions for both scripts * Service endpoint mappings * Troubleshooting guide * Docker management commands These scripts enable safe, tested production deployments with comprehensive validation at every stage. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- DEPLOYMENT_GUIDE.md | 260 ++++++++++++++++++++++++++++++++++++++++++++ deploy-all.sh | 193 ++++++++++++++++++++++++++++++++ deploy-invttrdg.sh | 223 +++++++++++++++++++++++++++++++++++++ 3 files changed, 676 insertions(+) create mode 100644 DEPLOYMENT_GUIDE.md create mode 100755 deploy-all.sh create mode 100755 deploy-invttrdg.sh diff --git a/DEPLOYMENT_GUIDE.md b/DEPLOYMENT_GUIDE.md new file mode 100644 index 0000000..48d9b5e --- /dev/null +++ b/DEPLOYMENT_GUIDE.md @@ -0,0 +1,260 @@ +# ByteLyst Production Deployment Guide + +## Overview + +This directory contains production deployment scripts for ByteLyst services. + +## Production Repos + +- `learning_ai_invt_trdg` - Trading Platform (https://api.bytelyst.com/invttrdg, https://invttrdg.bytelyst.com) +- `learning_ai_common_plat` - Platform Services (auth, flags, telemetry, etc.) +- `learning_ai_clock` - ChronoMind (AI-powered time management) +- `learning_ai_notes` - NoteLett (Agentic note-taking) + +## Deployment Scripts + +### 1. Single Repo Deployment (`deploy-invttrdg.sh`) + +Deploy only the investment trading platform with comprehensive testing. + +```bash +# Standard deployment (with dirty checks + smoke tests) +./deploy-invttrdg.sh + +# Force deployment (skip dirty checks + smoke tests) +./deploy-invttrdg.sh --force + +# Skip health checks and smoke tests +./deploy-invttrdg.sh --skip-health-check +``` + +**What it does:** +1. Dirty check (uncommitted changes, unpushed commits) +2. Pull and rebase origin/main +3. **Pre-deployment smoke tests:** + - Backend contract checks (API, audit repository, WebSocket, session rules) + - Web typecheck + production build + - Web DOM smoke tests (auth, kill-switch, components) + - Mobile typecheck compilation +4. Build and deploy Docker containers (backend + web) +5. **Post-deployment validation:** + - Health checks on localhost endpoints + - Production endpoint verification (api.bytelyst.com, invttrdg.bytelyst.com) + - API smoke tests (health endpoint, content validation) + - Web frontend validation (HTML content check) + +### 2. Multi-Repo Deployment (`deploy-all.sh`) + +Deploy all production repos or specific ones. + +```bash +# Deploy all production repos +./deploy-all.sh + +# Deploy specific repos +./deploy-all.sh learning_ai_invt_trdg learning_ai_clock + +# Force deployment (skip dirty checks) +./deploy-all.sh --force + +# Skip health checks +./deploy-all.sh --skip-health-check + +# Deploy specific repos with force +./deploy-all.sh --force learning_ai_invt_trdg +``` + +**What it does:** +1. Runs dirty checks for each repo +2. Pulls and rebases origin/main for each repo +3. Builds and deploys Docker containers +4. Runs health checks on all services + +## Service Endpoints + +### Investment Trading (learning_ai_invt_trdg) +**Docker Services Deployed:** +- **Backend:** `invttrdg-backend` (port 4018 → mapped to 4025) + - Trading engine + REST API + Socket.IO + - Health endpoint: `/health/live` + - API endpoints for trading operations +- **Web:** `invttrdg-web` (port 3085) + - Vite SPA served via nginx + - React trading interface + +**Exposed Endpoints:** +- **Local Backend API:** http://localhost:4025 +- **Local Web App:** http://localhost:3085 +- **Production API:** https://api.bytelyst.com/invttrdg (via Caddy reverse proxy) +- **Production Web:** https://invttrdg.bytelyst.com (via Caddy reverse proxy) + +### Platform Services (learning_ai_common_plat) +- **Platform Service:** http://localhost:4003 +- **Extraction Service:** http://localhost:4005 +- **MCP Server:** http://localhost:4007 +- **Admin Web:** http://localhost:3001 +- **Tracker Web:** http://localhost:3003 + +### ChronoMind (learning_ai_clock) +- **Backend:** http://localhost:4011 +- **Web:** http://localhost:3030 + +### NoteLett (learning_ai_notes) +- **Backend:** http://localhost:4016 +- **Web:** http://localhost:3000 + +## Docker Management + +### Check running containers +```bash +docker ps +``` + +### View logs +```bash +# Specific repo +cd learning_ai_invttrdg +docker compose logs -f + +# All containers +docker compose -f learning_ai_common_plat/docker-compose.ecosystem.yml logs -f +``` + +### Restart services +```bash +# Specific repo +cd learning_ai_invttrdg +docker compose restart + +# All services +docker compose -f learning_ai_common_plat/docker-compose.ecosystem.yml restart +``` + +### Stop services +```bash +# Specific repo +cd learning_ai_invttrdg +docker compose down + +# All services +docker compose -f learning_ai_common_plat/docker-compose.ecosystem.yml down +``` + +## Troubleshooting + +### Dirty Check Failures + +If deployment fails due to uncommitted changes: +```bash +# Option 1: Commit changes +cd learning_ai_invttrdg +git add . +git commit -m "Your commit message" + +# Option 2: Stash changes +git stash + +# Option 3: Force deployment +./deploy-invttrdg.sh --force +``` + +### Rebase Conflicts + +If rebase fails due to conflicts: +```bash +cd learning_ai_invttrdg +# Resolve conflicts +git add . +git rebase --continue +``` + +### Health Check Failures + +If health checks fail but services are starting: +```bash +# Skip health checks and smoke tests +./deploy-invttrdg.sh --skip-health-check + +# Check manually +curl http://localhost:4025/health/live +curl http://localhost:3085 +``` + +### Smoke Test Failures + +If pre-deployment smoke tests fail: +```bash +# Run smoke tests manually to debug +cd learning_ai_invt_trdg +./scripts/smoke-release.sh + +# Run individual backend checks +cd backend +npm run check:api-contract +npm run check:websocket-contract +npm run check:session-rule-normalization + +# Run web tests individually +cd ../web +pnpm vitest run src/components/Login.dom.test.tsx +``` + +### Docker Build Failures + +If Docker build fails: +```bash +# Clean build +cd learning_ai_invttrdg +docker compose build --no-cache + +# Check disk space +df -h +``` + +## DNS Configuration + +DNS is managed via GoDaddy. To update DNS records: + +```bash +cd learning_ai_common_plat +./scripts/godaddy-sync-bytelyst-dns.sh --ip --validate +``` + +## Monitoring + +### Health Check Script +```bash +./check-health.sh +``` + +### View Logs +- **Grafana:** http://localhost:3000 (admin/bytelyst) +- **Loki:** http://localhost:3100 +- **Traefik Dashboard:** http://localhost:8080 + +## Production Checklist + +Before deploying to production: + +- [ ] All tests pass locally +- [ ] No uncommitted changes +- [ ] No unpushed commits +- [ ] Environment variables are set correctly +- [ ] DNS records point to correct IP +- [ ] Database migrations are applied +- [ ] Backup current deployment +- [ ] Monitor logs after deployment + +## Rollback + +If deployment causes issues: + +```bash +cd learning_ai_invttrdg +# Revert to previous commit +git log --oneline -5 +git revert + +# Redeploy +docker compose up -d --build +``` diff --git a/deploy-all.sh b/deploy-all.sh new file mode 100755 index 0000000..7d3eaf1 --- /dev/null +++ b/deploy-all.sh @@ -0,0 +1,193 @@ +#!/usr/bin/env bash +set -euo pipefail + +# ═══════════════════════════════════════════════════════════════════════ +# ByteLyst Production Deployment - All Repos +# ═══════════════════════════════════════════════════════════════════════ +# Usage: ./deploy-all.sh [--force] [--skip-health-check] [repo1 repo2 ...] +# +# Deploys all production repos or specific repos: +# - learning_ai_invt_trdg (Trading) +# - learning_ai_common_plat (Platform Services) +# - learning_ai_clock (ChronoMind) +# - learning_ai_notes (NoteLett) +# +# Options: +# --force Skip dirty checks and force deployment +# --skip-health-check Skip endpoint health verification +# ═══════════════════════════════════════════════════════════════════════ + +RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'; CYAN='\033[0;36m'; NC='\033[0m' + +log() { echo -e "${CYAN}[$(date +%H:%M:%S)]${NC} $*"; } +ok() { echo -e "${GREEN}[$(date +%H:%M:%S)] ✓${NC} $*"; } +warn() { echo -e "${YELLOW}[$(date +%H:%M:%S)] ⚠${NC} $*"; } +fail() { echo -e "${RED}[$(date +%H:%M:%S)] ✗${NC} $*"; exit 1; } + +# ── Configuration ──────────────────────────────────────────────────── +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +cd "$SCRIPT_DIR" + +PRODUCTION_REPOS=( + "learning_ai_invt_trdg" + "learning_ai_common_plat" + "learning_ai_clock" + "learning_ai_notes" +) + +FORCE=false +SKIP_HEALTH_CHECK=false +SPECIFIC_REPOS=() + +while [[ $# -gt 0 ]]; do + case "$1" in + --force) FORCE=true; shift ;; + --skip-health-check) SKIP_HEALTH_CHECK=true; shift ;; + -*) fail "Unknown option: $1" ;; + *) SPECIFIC_REPOS+=("$1"); shift ;; + esac +done + +# If specific repos provided, validate them +if [ ${#SPECIFIC_REPOS[@]} -gt 0 ]; then + for repo in "${SPECIFIC_REPOS[@]}"; do + valid=false + for valid_repo in "${PRODUCTION_REPOS[@]}"; do + if [ "$repo" = "$valid_repo" ]; then + valid=true + break + fi + done + if [ "$valid" = false ]; then + fail "Unknown repo: $repo. Valid repos: ${PRODUCTION_REPOS[*]}" + fi + done + REPOS_TO_DEPLOY=("${SPECIFIC_REPOS[@]}") +else + REPOS_TO_DEPLOY=("${PRODUCTION_REPOS[@]}") +fi + +# ── Deploy Function ─────────────────────────────────────────────────── +deploy_repo() { + local repo="$1" + local repo_dir="${SCRIPT_DIR}/${repo}" + + log "══════════════════════════════════════════════════════════════════════" + log "Deploying: $repo" + log "══════════════════════════════════════════════════════════════════════" + + if [ ! -d "$repo_dir" ]; then + fail "Repo directory not found: $repo_dir" + fi + + cd "$repo_dir" + + # ── Dirty Check ─────────────────────────────────────────────────── + if [ "$FORCE" = false ]; then + log "Running dirty checks for $repo..." + + if ! git diff-index --quiet HEAD --; then + fail "Uncommitted changes in $repo. Commit or stash first, or use --force" + fi + + if [ -n "$(git ls-files --others --exclude-standard)" ]; then + fail "Untracked files in $repo. Commit or remove them, or use --force" + fi + + LOCAL_COMMIT=$(git rev-parse @) + REMOTE_COMMIT=$(git rev-parse '@{u}' 2>/dev/null || echo "") + + if [ -n "$REMOTE_COMMIT" ] && [ "$LOCAL_COMMIT" != "$REMOTE_COMMIT" ]; then + fail "Unpushed commits in $repo. Push first or use --force" + fi + + ok "Dirty checks passed for $repo" + else + warn "Skipping dirty checks for $repo (--force enabled)" + fi + + # ── Pull and Rebase ─────────────────────────────────────────────── + log "Pulling latest changes for $repo..." + git fetch origin + + LOCAL_MAIN=$(git rev-parse main) + REMOTE_MAIN=$(git rev-parse origin/main) + + if [ "$LOCAL_MAIN" != "$REMOTE_MAIN" ]; then + log "Local main is behind origin/main, rebasing $repo..." + git rebase origin/main || { + fail "Rebase failed for $repo. Resolve conflicts and run: git rebase --continue" + } + ok "Rebase completed for $repo" + else + ok "$repo is already up to date" + fi + + # ── Build and Deploy ─────────────────────────────────────────────── + if [ -f "docker-compose.yml" ]; then + log "Building and deploying $repo..." + docker compose build || fail "Docker build failed for $repo" + docker compose up -d || fail "Docker compose up failed for $repo" + ok "Deployment completed for $repo" + else + warn "No docker-compose.yml found in $repo, skipping Docker deployment" + fi + + cd "$SCRIPT_DIR" +} + +# ── Deploy All Repos ─────────────────────────────────────────────────── +FAILED_REPOS=() +SUCCESSFUL_REPOS=() + +for repo in "${REPOS_TO_DEPLOY[@]}"; do + if deploy_repo "$repo"; then + SUCCESSFUL_REPOS+=("$repo") + else + FAILED_REPOS+=("$repo") + fi +done + +# ── Summary ───────────────────────────────────────────────────────────── +log "══════════════════════════════════════════════════════════════════════" +log "Deployment Summary" +log "══════════════════════════════════════════════════════════════════════" + +if [ ${#SUCCESSFUL_REPOS[@]} -gt 0 ]; then + ok "Successfully deployed: ${SUCCESSFUL_REPOS[*]}" +fi + +if [ ${#FAILED_REPOS[@]} -gt 0 ]; then + fail "Failed to deploy: ${FAILED_REPOS[*]}" +fi + +# ── Health Checks (if not skipped) ─────────────────────────────────────── +if [ "$SKIP_HEALTH_CHECK" = false ]; then + log "Running health checks..." + + # Define health endpoints for each service + declare -A HEALTH_ENDPOINTS=( + ["learning_ai_invt_trdg"]="http://localhost:4025/health/live" + ["learning_ai_common_plat"]="http://localhost:4003/health" + ["learning_ai_clock"]="http://localhost:4011/health" + ["learning_ai_notes"]="http://localhost:4016/health" + ) + + for repo in "${SUCCESSFUL_REPOS[@]}"; do + endpoint="${HEALTH_ENDPOINTS[$repo]:-}" + if [ -n "$endpoint" ]; then + log "Checking $repo at $endpoint" + if curl -sf "$endpoint" > /dev/null 2>&1; then + ok "$repo is healthy" + else + warn "$repo health check failed (may still be starting)" + fi + fi + done +else + warn "Skipping health checks (--skip-health-check enabled)" +fi + +log "══════════════════════════════════════════════════════════════════════" +ok "All deployments completed successfully!" +log "══════════════════════════════════════════════════════════════════════" diff --git a/deploy-invttrdg.sh b/deploy-invttrdg.sh new file mode 100755 index 0000000..5f7ff91 --- /dev/null +++ b/deploy-invttrdg.sh @@ -0,0 +1,223 @@ +#!/usr/bin/env bash +set -euo pipefail + +# ═══════════════════════════════════════════════════════════════════════ +# ByteLyst Investment Trading - Production Deployment Script +# ═══════════════════════════════════════════════════════════════════════ +# Usage: ./deploy-invttrdg.sh [--force] [--skip-health-check] +# +# What it does: +# 1. Dirty check: uncommitted changes, unpushed commits +# 2. Pull and rebase origin/main +# 3. Build and deploy Docker containers +# 4. Verify endpoints: https://api.bytelyst.com/invttrdg, https://invttrdg.bytelyst.com +# +# Options: +# --force Skip dirty checks and force deployment +# --skip-health-check Skip endpoint health verification +# ═══════════════════════════════════════════════════════════════════════ + +RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'; CYAN='\033[0;36m'; NC='\033[0m' + +log() { echo -e "${CYAN}[$(date +%H:%M:%S)]${NC} $*"; } +ok() { echo -e "${GREEN}[$(date +%H:%M:%S)] ✓${NC} $*"; } +warn() { echo -e "${YELLOW}[$(date +%H:%M:%S)] ⚠${NC} $*"; } +fail() { echo -e "${RED}[$(date +%H:%M:%S)] ✗${NC} $*"; exit 1; } + +# ── Configuration ──────────────────────────────────────────────────── +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +REPO_DIR="${SCRIPT_DIR}/learning_ai_invt_trdg" +cd "$SCRIPT_DIR" + +FORCE=false +SKIP_HEALTH_CHECK=false + +while [[ $# -gt 0 ]]; do + case "$1" in + --force) FORCE=true; shift ;; + --skip-health-check) SKIP_HEALTH_CHECK=true; shift ;; + *) fail "Unknown option: $1" ;; + esac +done + +# ── Prerequisites ──────────────────────────────────────────────────── +if [ ! -d "$REPO_DIR" ]; then + fail "Repo directory not found: $REPO_DIR" +fi + +cd "$REPO_DIR" + +# ── Dirty Check ─────────────────────────────────────────────────────── +if [ "$FORCE" = false ]; then + log "Running dirty checks..." + + # Check for uncommitted changes + if ! git diff-index --quiet HEAD --; then + fail "Uncommitted changes detected. Commit or stash first, or use --force" + fi + + # Check for untracked files + if [ -n "$(git ls-files --others --exclude-standard)" ]; then + fail "Untracked files detected. Commit or remove them, or use --force" + fi + + # Check for unpushed commits + LOCAL_COMMIT=$(git rev-parse @) + REMOTE_COMMIT=$(git rev-parse '@{u}' 2>/dev/null || echo "") + + if [ -n "$REMOTE_COMMIT" ] && [ "$LOCAL_COMMIT" != "$REMOTE_COMMIT" ]; then + fail "Unpushed commits detected. Push first or use --force" + fi + + ok "Dirty checks passed" +else + warn "Skipping dirty checks (--force enabled)" +fi + +# ── Pull and Rebase ─────────────────────────────────────────────────── +log "Pulling latest changes from origin/main..." +git fetch origin + +LOCAL_MAIN=$(git rev-parse main) +REMOTE_MAIN=$(git rev-parse origin/main) + +if [ "$LOCAL_MAIN" != "$REMOTE_MAIN" ]; then + log "Local main is behind origin/main, rebasing..." + git rebase origin/main || { + fail "Rebase failed. Resolve conflicts and run: git rebase --continue" + } + ok "Rebase completed successfully" +else + ok "Already up to date with origin/main" +fi + +# ── Run Smoke Tests ───────────────────────────────────────────────────── +if [ "$SKIP_HEALTH_CHECK" = false ]; then + log "Running smoke tests before deployment..." + if [ -f "scripts/smoke-release.sh" ]; then + chmod +x scripts/smoke-release.sh + ./scripts/smoke-release.sh || { + fail "Smoke tests failed. Fix issues before deploying or use --skip-health-check" + } + ok "Smoke tests passed" + else + warn "Smoke test script not found, skipping pre-deployment tests" + fi +fi + +# ── Build and Deploy ────────────────────────────────────────────────── +log "Building and deploying Docker containers..." + +# Check if docker-compose files exist +if [ ! -f "docker-compose.yml" ]; then + fail "docker-compose.yml not found in $REPO_DIR" +fi + +# Build and start services +log "Building Docker images..." +docker compose build || fail "Docker build failed" + +log "Starting services..." +docker compose up -d || fail "Docker compose up failed" + +ok "Deployment completed" + +# ── Health Check ────────────────────────────────────────────────────── +if [ "$SKIP_HEALTH_CHECK" = true ]; then + warn "Skipping health checks (--skip-health-check enabled)" + exit 0 +fi + +log "Waiting for services to be healthy..." +sleep 10 + +# Check backend health +BACKEND_HEALTH=false +for _ in {1..30}; do + if curl -sf http://localhost:4025/health/live > /dev/null 2>&1; then + BACKEND_HEALTH=true + break + fi + echo -n "." + sleep 2 +done +echo "" + +if [ "$BACKEND_HEALTH" = true ]; then + ok "Backend health check passed (http://localhost:4025)" +else + fail "Backend health check failed" +fi + +# Check web health +WEB_HEALTH=false +for _ in {1..30}; do + if curl -sf http://localhost:3085 > /dev/null 2>&1; then + WEB_HEALTH=true + break + fi + echo -n "." + sleep 2 +done +echo "" + +if [ "$WEB_HEALTH" = true ]; then + ok "Web health check passed (http://localhost:3085)" +else + warn "Web health check failed (may be starting up)" +fi + +# ── Endpoint Verification ───────────────────────────────────────────── +log "Verifying production endpoints..." + +API_ENDPOINT="https://api.bytelyst.com/invttrdg" +WEB_ENDPOINT="https://invttrdg.bytelyst.com" + +# Check API endpoint +if curl -sf "$API_ENDPOINT/health/live" > /dev/null 2>&1; then + ok "API endpoint accessible: $API_ENDPOINT" +else + warn "API endpoint not accessible: $API_ENDPOINT (may need DNS propagation)" +fi + +# Check web endpoint +if curl -sf "$WEB_ENDPOINT" > /dev/null 2>&1; then + ok "Web endpoint accessible: $WEB_ENDPOINT" +else + warn "Web endpoint not accessible: $WEB_ENDPOINT (may need DNS propagation)" +fi + +# ── API Smoke Tests (Post-Deployment) ─────────────────────────────────── +log "Running post-deployment API smoke tests..." + +# Test backend health endpoint +BACKEND_URL="http://localhost:4025" +if curl -sf "$BACKEND_URL/health/live" > /dev/null 2>&1; then + ok "Backend health endpoint responding" + # Try to get health details + HEALTH_RESPONSE=$(curl -s "$BACKEND_URL/health/live" 2>/dev/null || echo "{}") + log "Health response: $HEALTH_RESPONSE" +else + fail "Backend health endpoint not responding" +fi + +# Test web is serving content +WEB_URL="http://localhost:3085" +if curl -sf "$WEB_URL" > /dev/null 2>&1; then + ok "Web frontend is serving content" + # Check if it's actually HTML (nginx serving the SPA) + CONTENT_TYPE=$(curl -sI "$WEB_URL" | grep -i content-type || echo "") + if echo "$CONTENT_TYPE" | grep -qi "text/html"; then + ok "Web frontend is serving HTML content" + else + warn "Web frontend content type unexpected: $CONTENT_TYPE" + fi +else + fail "Web frontend not responding" +fi + +log "══════════════════════════════════════════════════════════════════════" +ok "Deployment completed successfully!" +log "Backend: http://localhost:4025 → $API_ENDPOINT" +log "Web: http://localhost:3085 → $WEB_ENDPOINT" +log "══════════════════════════════════════════════════════════════════════"