Saravana Achu Mac cdbdd48a35 docs(devops): add canonical endpoint inventory

2026-04-03 17:29:46 -07:00

11 KiB

Raw Blame History

Ecosystem Deployment Guide

File: docker-compose.ecosystem.yml Purpose: Run the entire ByteLyst ecosystem in a single Docker Compose stack.

ENDPOINT_INVENTORY.md — canonical endpoint inventory across local, Azure VM, and domain-based ingress

Prerequisites

Docker Desktop with Compose v2+
All product repos cloned as siblings of learning_ai_common_plat:

~/code/mygh/
├── learning_ai_common_plat/          # This repo
├── learning_voice_ai_agent/          # LysnrAI
├── learning_multimodal_memory_agents/ # MindLyst
├── learning_ai_clock/                # ChronoMind
├── learning_ai_jarvis_jr/            # JarvisJr
├── learning_ai_fastgap/              # NomGap
├── learning_ai_peakpulse/            # PeakPulse
├── learning_ai_flowmonk/             # FlowMonk
├── learning_ai_notes/                # NoteLett
├── learning_ai_trails/               # ActionTrail
└── learning_ai_local_memory_gpt/     # LocalMemGPT

@bytelyst/* packages built (for platform-service / extraction-service):

cd learning_ai_common_plat
pnpm -r --filter './packages/*' build

Quick Start

cd learning_ai_common_plat

# 1. Create env file from example
cp .env.ecosystem.example .env.ecosystem
# Edit .env.ecosystem — fill in Cosmos emulator key, Azurite key, Stripe keys, Gemini key, etc.

# 2. Export Gitea npm token (required for product Docker builds)
export GITEA_NPM_TOKEN=<your-gitea-package-token>

# 3. Validate compose config
docker compose -f docker-compose.ecosystem.yml --env-file .env.ecosystem config --quiet

# 4. Build and start everything
docker compose -f docker-compose.ecosystem.yml --env-file .env.ecosystem up --build -d

# 5. Check health
docker compose -f docker-compose.ecosystem.yml ps

Deployment Split

This compose stack is the VM-hosted side of the ecosystem.

Azure VM / Docker: backend services, shared infrastructure, internal dashboards, and internal product web surfaces
Vercel: only clearly public-facing portal web surfaces

Examples:

keep admin-web and tracker-web on the VM
keep product backends on the VM
only deploy user-facing portal experiences to Vercel when they are intended to be public-facing browser surfaces

Do not assume every web app belongs on Vercel.

Port Map

Infrastructure

Service	Port	URL
Cosmos DB Emulator	8081	http://localhost:8081
Cosmos Data Explorer	1234	http://localhost:1234
Azurite (Blob)	10000	http://localhost:10000
Mailpit SMTP	1025	—
Mailpit UI	8025	http://localhost:8025
Loki	3100	http://localhost:3100
Grafana	3000	http://localhost:3000
Traefik Dashboard	8080	http://localhost:8080

Platform Services

Service	Port	URL
platform-service	4003	http://localhost:4003/health
extraction-service	4005	http://localhost:4005/health
mcp-server	4007	http://localhost:4007/health

Platform Dashboards

Dashboard	Port	URL
Admin Console	3001	http://localhost:3001
Issue Tracker	3003	http://localhost:3003

Product Backends

Product	Port	URL
PeakPulse	4010	http://localhost:4010/health
ChronoMind	4011	http://localhost:4011/health
JarvisJr	4012	http://localhost:4012/health
NomGap	4013	http://localhost:4013/health
MindLyst	4014	http://localhost:4014/health
LysnrAI	4015	http://localhost:4015/health
NoteLett	4016	http://localhost:4016/health
FlowMonk	4017	http://localhost:4017/health
ActionTrail	4018	http://localhost:4018/health
LocalMemGPT	4019	http://localhost:4019/health

Product Web Apps

Product	Port	URL
LysnrAI Dashboard	3002	http://localhost:3002
ChronoMind	3030	http://localhost:3030
JarvisJr	3035	http://localhost:3035
FlowMonk	3040	http://localhost:3040
NoteLett	3045	http://localhost:3045
MindLyst	3050	http://localhost:3050
NomGap	3055	http://localhost:3055
ActionTrail	3060	http://localhost:3060
LocalMemGPT	3070	http://localhost:3070

Selective Startup

You don't need to run everything. Use Docker Compose profiles or just name services:

# Infrastructure + platform only
docker compose -f docker-compose.ecosystem.yml --env-file .env.ecosystem up -d \
  cosmos-emulator azurite mailpit loki grafana gateway \
  platform-service extraction-service mcp-server \
  admin-web tracker-web

# Infrastructure + platform + one product
docker compose -f docker-compose.ecosystem.yml --env-file .env.ecosystem up -d \
  cosmos-emulator azurite mailpit platform-service \
  flowmonk-backend flowmonk-web

# Just backends (no web apps)
docker compose -f docker-compose.ecosystem.yml --env-file .env.ecosystem up -d \
  cosmos-emulator azurite mailpit platform-service extraction-service \
  flowmonk-backend notelett-backend actiontrail-backend

Teardown

# Stop all
docker compose -f docker-compose.ecosystem.yml --env-file .env.ecosystem down

# Stop and remove volumes (clean slate)
docker compose -f docker-compose.ecosystem.yml --env-file .env.ecosystem down -v

Troubleshooting

Cosmos emulator slow to start

The emulator needs 20-30s on first boot. Other services wait via depends_on: condition: service_healthy.

Port conflicts

If a port is already in use (e.g., Grafana on 3000), either stop the conflicting process or override the host port:

# Override in .env.ecosystem or via CLI
docker compose -f docker-compose.ecosystem.yml --env-file .env.ecosystem up -d \
  -e GRAFANA_PORT=3099

LocalMemGPT needs Ollama

LocalMemGPT connects to Ollama on the host via host.docker.internal:11434. Make sure Ollama is running locally.

Build failures for product repos

Product Dockerfiles use the Gitea npm registry for @bytelyst/* packages. If building on a fresh machine, ensure:

GITEA_NPM_TOKEN is set in your environment
Gitea is running at localhost:3300
All 49 @bytelyst/* packages are published

See docs/devops/GITEA_NPM_REGISTRY_MIGRATION.md for details.

Single-VM Bootstrap (Azure / bare Ubuntu)

For deploying the entire ecosystem from scratch on a raw Ubuntu VM, use the automated bootstrap script:

# Location: docs/devops/single_azure_vm/setup.sh

This script installs everything — Docker, Node.js, pnpm, Gitea (npm registry), Ollama (LLM), clones all repos, builds packages, publishes to Gitea, and deploys the full 30-service ecosystem.

Quick Start

# Fresh install (runs all 8 phases)
sudo ./setup.sh

# Check what's done
sudo ./setup.sh --status

Resume & Retry

Phase completion is tracked in /opt/bytelyst/.setup-state/. If anything fails, you don't have to start over:

# Phase 7 failed? Retry just that phase
sudo ./setup.sh --phase=7

# SSH disconnected mid-run? Auto-resume from where it left off
sudo ./setup.sh --resume

# Jump straight to deploy after a manual fix
sudo ./setup.sh --resume-from=7

# Start completely over
sudo ./setup.sh --reset

Phases

Phase	Description
1	System dependencies (Docker, Node.js 22, pnpm, Ollama)
2	Gitea npm registry (container on :3300)
3	Clone 11 repositories from GitHub
4	Build all @bytelyst/* packages
5	Publish packages to Gitea npm registry
6	Generate `.env.ecosystem` config
7	Build + deploy 30 Docker services (per-service, with fallback)
8	Health check (30 endpoints)

Per-Service Build & Fallback

Phase 7 builds each of the 30 services individually:

Pre-built images (infra: cosmos-emulator, azurite, etc.) are detected and skipped
Failed builds are skipped — remaining services still start
Build logs are saved per-service to /opt/bytelyst/.setup-state/builds/<service>.log
After fixing a build failure, re-run with sudo ./setup.sh --phase=7

Health Check

After deployment, a standalone health-check script is created at /opt/bytelyst/check-health.sh:

/opt/bytelyst/check-health.sh

This checks all 30 endpoints (infrastructure, platform services, dashboards, product backends, product web apps) plus Gitea and Ollama.

Architecture

┌─────────────────────────────────────────────────────────────┐
│                      Traefik Gateway (:80)                  │
└──────┬──────────────┬──────────────┬────────────────────────┘
       │              │              │
  ┌────▼────┐   ┌─────▼─────┐  ┌────▼────┐
  │Platform │   │Extraction │  │  MCP    │
  │ :4003   │   │  :4005    │  │ :4007   │
  └────┬────┘   └───────────┘  └─────────┘
       │
  ┌────▼────────────────────────────────────────────────┐
  │              10 Product Backends                     │
  │  :4010-4019 (PeakPulse…LocalMemGPT)                │
  └────┬────────────────────────────────────────────────┘
       │
  ┌────▼────────────────────────────────────────────────┐
  │              9 Product Web Apps                      │
  │  :3002, :3030-3070                                  │
  └─────────────────────────────────────────────────────┘
       │
  ┌────▼────────────────────────────────────────────────┐
  │              Infrastructure                          │
  │  Cosmos :8081  Azurite :10000  Mailpit :1025/8025   │
  │  Loki :3100  Grafana :3000                          │
  └─────────────────────────────────────────────────────┘

11 KiB Raw Blame History

Ecosystem Deployment Guide

Prerequisites

Quick Start

Deployment Split

Port Map

Infrastructure

Platform Services

Platform Dashboards

Product Backends

Product Web Apps

Selective Startup

Teardown

Troubleshooting

Cosmos emulator slow to start

Port conflicts

LocalMemGPT needs Ollama

Build failures for product repos

Single-VM Bootstrap (Azure / bare Ubuntu)

Quick Start

Resume & Retry

Phases

Per-Service Build & Fallback

Health Check

Architecture

11 KiB

Raw Blame History