bytelyst/learning_ai_common_plat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
d64ea4fba7
learning_ai_common_plat/docs/devops/ECOSYSTEM_DEPLOYMENT.md
saravanakumardb1 acbab75aaa docs(infra): add complete CLI reference, examples, and phase docs to setup.sh + ECOSYSTEM_DEPLOYMENT.md 
setup.sh header now includes:
- All 6 CLI flags (--resume, --resume-from, --phase, --reset, --status, --help)
- Phase descriptions (1-8)
- 6 usage examples (fresh install, retry, resume, jump, status, reset)
- Resume/retry explanation with state dir and build log paths

ECOSYSTEM_DEPLOYMENT.md now includes:
- Single-VM Bootstrap section with quick start
- Resume & Retry examples
- Phase table
- Per-service build & fallback explanation
- Health check script reference
2026-03-24 12:24:16 -07:00

284 lines
9.9 KiB
Markdown
Raw Blame History

# Ecosystem Deployment Guide
> **File:** `docker-compose.ecosystem.yml`
> **Purpose:** Run the entire ByteLyst ecosystem in a single Docker Compose stack.
---
## Prerequisites
1. **Docker Desktop** with Compose v2+
2. All product repos cloned as siblings of `learning_ai_common_plat`:
```
~/code/mygh/
├── learning_ai_common_plat/ # This repo
├── learning_voice_ai_agent/ # LysnrAI
├── learning_multimodal_memory_agents/ # MindLyst
├── learning_ai_clock/ # ChronoMind
├── learning_ai_jarvis_jr/ # JarvisJr
├── learning_ai_fastgap/ # NomGap
├── learning_ai_peakpulse/ # PeakPulse
├── learning_ai_flowmonk/ # FlowMonk
├── learning_ai_notes/ # NoteLett
├── learning_ai_trails/ # ActionTrail
└── learning_ai_local_memory_gpt/ # LocalMemGPT
```
3. `@bytelyst/*` packages built (for platform-service / extraction-service):
```bash
cd learning_ai_common_plat
pnpm -r --filter './packages/*' build
```
---
## Quick Start
```bash
cd learning_ai_common_plat
# 1. Create env file from example
cp .env.ecosystem.example .env.ecosystem
# Edit .env.ecosystem — fill in Cosmos emulator key, Azurite key, Stripe keys, Gemini key, etc.
# 2. Export Gitea npm token (required for product Docker builds)
export GITEA_NPM_TOKEN=<your-gitea-package-token>
# 3. Validate compose config
docker compose -f docker-compose.ecosystem.yml --env-file .env.ecosystem config --quiet
# 4. Build and start everything
docker compose -f docker-compose.ecosystem.yml --env-file .env.ecosystem up --build -d
# 5. Check health
docker compose -f docker-compose.ecosystem.yml ps
```
---
## Port Map
### Infrastructure
| Service | Port | URL |
|---------|------|-----|
| Cosmos DB Emulator | 8081 | http://localhost:8081 |
| Cosmos Data Explorer | 1234 | http://localhost:1234 |
| Azurite (Blob) | 10000 | http://localhost:10000 |
| Mailpit SMTP | 1025 | — |
| Mailpit UI | 8025 | http://localhost:8025 |
| Loki | 3100 | http://localhost:3100 |
| Grafana | 3000 | http://localhost:3000 |
| Traefik Dashboard | 8080 | http://localhost:8080 |
### Platform Services
| Service | Port | URL |
|---------|------|-----|
| platform-service | 4003 | http://localhost:4003/health |
| extraction-service | 4005 | http://localhost:4005/health |
| mcp-server | 4007 | http://localhost:4007/health |
### Platform Dashboards
| Dashboard | Port | URL |
|-----------|------|-----|
| Admin Console | 3001 | http://localhost:3001 |
| Issue Tracker | 3003 | http://localhost:3003 |
### Product Backends
| Product | Port | URL |
|---------|------|-----|
| PeakPulse | 4010 | http://localhost:4010/health |
| ChronoMind | 4011 | http://localhost:4011/health |
| JarvisJr | 4012 | http://localhost:4012/health |
| NomGap | 4013 | http://localhost:4013/health |
| MindLyst | 4014 | http://localhost:4014/health |
| LysnrAI | 4015 | http://localhost:4015/health |
| NoteLett | 4016 | http://localhost:4016/health |
| FlowMonk | 4017 | http://localhost:4017/health |
| ActionTrail | 4018 | http://localhost:4018/health |
| LocalMemGPT | 4019 | http://localhost:4019/health |
### Product Web Apps
| Product | Port | URL |
|---------|------|-----|
| LysnrAI Dashboard | 3002 | http://localhost:3002 |
| ChronoMind | 3030 | http://localhost:3030 |
| JarvisJr | 3035 | http://localhost:3035 |
| FlowMonk | 3040 | http://localhost:3040 |
| NoteLett | 3045 | http://localhost:3045 |
| MindLyst | 3050 | http://localhost:3050 |
| NomGap | 3055 | http://localhost:3055 |
| ActionTrail | 3060 | http://localhost:3060 |
| LocalMemGPT | 3070 | http://localhost:3070 |
---
## Selective Startup
You don't need to run everything. Use Docker Compose profiles or just name services:
```bash
# Infrastructure + platform only
docker compose -f docker-compose.ecosystem.yml --env-file .env.ecosystem up -d \
cosmos-emulator azurite mailpit loki grafana gateway \
platform-service extraction-service mcp-server \
admin-web tracker-web
# Infrastructure + platform + one product
docker compose -f docker-compose.ecosystem.yml --env-file .env.ecosystem up -d \
cosmos-emulator azurite mailpit platform-service \
flowmonk-backend flowmonk-web
# Just backends (no web apps)
docker compose -f docker-compose.ecosystem.yml --env-file .env.ecosystem up -d \
cosmos-emulator azurite mailpit platform-service extraction-service \
flowmonk-backend notelett-backend actiontrail-backend
```
---
## Teardown
```bash
# Stop all
docker compose -f docker-compose.ecosystem.yml --env-file .env.ecosystem down
# Stop and remove volumes (clean slate)
docker compose -f docker-compose.ecosystem.yml --env-file .env.ecosystem down -v
```
---
## Troubleshooting
### Cosmos emulator slow to start
The emulator needs 20-30s on first boot. Other services wait via `depends_on: condition: service_healthy`.
### Port conflicts
If a port is already in use (e.g., Grafana on 3000), either stop the conflicting process or override the host port:
```bash
# Override in .env.ecosystem or via CLI
docker compose -f docker-compose.ecosystem.yml --env-file .env.ecosystem up -d \
-e GRAFANA_PORT=3099
```
### LocalMemGPT needs Ollama
LocalMemGPT connects to Ollama on the host via `host.docker.internal:11434`. Make sure Ollama is running locally.
### Build failures for product repos
Product Dockerfiles use the Gitea npm registry for `@bytelyst/*` packages. If building on a fresh machine, ensure:
1. `GITEA_NPM_TOKEN` is set in your environment
2. Gitea is running at `localhost:3300`
3. All 49 `@bytelyst/*` packages are published
See `docs/devops/GITEA_NPM_REGISTRY_MIGRATION.md` for details.
---
## Single-VM Bootstrap (Azure / bare Ubuntu)
For deploying the entire ecosystem from scratch on a raw Ubuntu VM, use the automated bootstrap script:
```bash
# Location: docs/devops/single_azure_vm/setup.sh
```
This script installs **everything** — Docker, Node.js, pnpm, Gitea (npm registry), Ollama (LLM), clones all repos, builds packages, publishes to Gitea, and deploys the full 30-service ecosystem.
### Quick Start
```bash
# Fresh install (runs all 8 phases)
sudo ./setup.sh
# Check what's done
sudo ./setup.sh --status
```
### Resume & Retry
Phase completion is tracked in `/opt/bytelyst/.setup-state/`. If anything fails, you don't have to start over:
```bash
# Phase 7 failed? Retry just that phase
sudo ./setup.sh --phase=7
# SSH disconnected mid-run? Auto-resume from where it left off
sudo ./setup.sh --resume
# Jump straight to deploy after a manual fix
sudo ./setup.sh --resume-from=7
# Start completely over
sudo ./setup.sh --reset
```
### Phases
| Phase | Description |
|-------|-------------|
| 1 | System dependencies (Docker, Node.js 22, pnpm, Ollama) |
| 2 | Gitea npm registry (container on :3300) |
| 3 | Clone 11 repositories from GitHub |
| 4 | Build all @bytelyst/* packages |
| 5 | Publish packages to Gitea npm registry |
| 6 | Generate `.env.ecosystem` config |
| 7 | Build + deploy 30 Docker services (per-service, with fallback) |
| 8 | Health check (30 endpoints) |
### Per-Service Build & Fallback
Phase 7 builds each of the 30 services individually:
- **Pre-built images** (infra: cosmos-emulator, azurite, etc.) are detected and skipped
- **Failed builds** are skipped — remaining services still start
- **Build logs** are saved per-service to `/opt/bytelyst/.setup-state/builds/<service>.log`
- After fixing a build failure, re-run with `sudo ./setup.sh --phase=7`
### Health Check
After deployment, a standalone health-check script is created at `/opt/bytelyst/check-health.sh`:
```bash
/opt/bytelyst/check-health.sh
```
This checks all 30 endpoints (infrastructure, platform services, dashboards, product backends, product web apps) plus Gitea and Ollama.
---
## Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Traefik Gateway (:80) │
└──────┬──────────────┬──────────────┬────────────────────────┘
│ │ │
┌────▼────┐ ┌─────▼─────┐ ┌────▼────┐
│Platform │ │Extraction │ │ MCP │
│ :4003 │ │ :4005 │ │ :4007 │
└────┬────┘ └───────────┘ └─────────┘
│
┌────▼────────────────────────────────────────────────┐
│ 10 Product Backends │
│ :4010-4019 (PeakPulse…LocalMemGPT) │
└────┬────────────────────────────────────────────────┘
│
┌────▼────────────────────────────────────────────────┐
│ 9 Product Web Apps │
│ :3002, :3030-3070 │
└─────────────────────────────────────────────────────┘
│
┌────▼────────────────────────────────────────────────┐
│ Infrastructure │
│ Cosmos :8081 Azurite :10000 Mailpit :1025/8025 │
│ Loki :3100 Grafana :3000 │
└─────────────────────────────────────────────────────┘
```
Reference in New Issue View Git Blame Copy Permalink