bytelyst/learning_ai_common_plat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update documentation

Browse Source
This commit is contained in:
saravanakumardb1 2026-03-22 14:06:44 -07:00
parent d934bb3286
commit 828d31b63d
2 changed files with 192 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
__LOCAL_LLMs/AI_IDE_CHAT_HISTORY/WINDSURF/.last-refresh.log
View File

@ -1,9 +1,9 @@
Last refresh: 2026-03-21T21:44:43Z (2026-03-21 14:44:44 PDT)
Cascade conversations: 50 (387M)
Memories: 95
Last refresh: 2026-03-22T18:43:11Z (2026-03-22 11:43:11 PDT)
Cascade conversations: 50 (417M)
Memories: 98
Implicit context: 20
Code tracker dirs: 89
File edit history: 3457 entries
Workspace storage: 36 workspaces
Code tracker dirs: 139
File edit history: 3547 entries
Workspace storage: 34 workspaces
Repo docs: 7 files across 2 repos
Repo workflows: 43 files across 10 repos

205
docs/devops/SINGLE_VM_DEPLOYMENT.md
View File

@ -4,6 +4,25 @@
---
## Package-Manager Strategy (current transition plan)
- `learning_ai_common_plat` is already the canonical **`pnpm` workspace** monorepo for shared packages, services, and dashboards.
- Node/TypeScript product repos are moving toward **`pnpm` as the long-term standard**, but that migration is still **repo-by-repo** and **incremental**.
- During the transition, each repo's Docker/build flow must follow the repo's own:
- `packageManager` field
- lockfile
- Dockerfile
- `docker-prep.sh` behavior
- This plan does **not** merge all repos into one mega-monorepo. Product repos remain independent repositories.
- Once a repo migrates to `pnpm`, it must be fully aligned in the same change set:
- no `pnpm-lock.yaml` with `npm ci`
- no stale `package-lock.json`
- no mixed package-manager assumptions in CI, Docker, or docs
> **Migration-impact note:** The deployment architecture in this guide stays the same during the `pnpm` migration (Compose, K3s, ingress, namespaces, VM sizing). The main maintenance surface is Docker/build instructions and dependency-prep flow. The biggest operational risk is stale templates or stale docs after an individual repo migrates.
---
## 1. Service Inventory
### Shared Infrastructure (common-plat)
@ -112,7 +131,7 @@
### Phase 1: Docker Compose (after prerequisite work)
> **⚠️ Prerequisite:** ALL product repos must run `docker-prep.sh` before building Docker images (see §12 Audit Findings). All Dockerfiles and `output: 'standalone'` configs are now in place (completed 2026-03-22).
> **⚠️ Prerequisite:** ALL product repos must run `docker-prep.sh` before building Docker images (see §12 Audit Findings). All Dockerfiles and `output: 'standalone'` configs are now in place (completed 2026-03-22). During the package-manager transition, each repo's Docker build must follow that repo's declared package manager and lockfile semantics rather than assuming `npm` or `pnpm` globally.
Create a **unified** `docker-compose.ecosystem.yml` that brings everything up.
@ -158,6 +177,8 @@ Create `docker-compose.ecosystem.yml` at workspace root (`~/code/mygh/`) that co
# Pack @bytelyst/* file: dependencies into tarballs for each product repo.
# Every product repo has file: refs to ../learning_ai_common_plat/packages/*
# which don't resolve inside Docker build context. docker-prep.sh packs them.
# The prep flow must preserve each repo's package-manager semantics while rewriting
# file: refs for Docker contexts.
for repo in learning_voice_ai_agent learning_multimodal_memory_agents learning_ai_clock \
learning_ai_jarvis_jr learning_ai_peakpulse learning_ai_flowmonk \
learning_ai_fastgap learning_ai_notes learning_ai_trails learning_ai_local_memory_gpt; do
@ -796,30 +817,32 @@ kubectl get pods -A
## 10. Dockerization Status (all complete)
| Repo | Backend Dockerfile | Web Dockerfile | `docker-prep.sh` | `output:'standalone'` | Status |
| --------------- | ------------------ | ------------------- | ---------------- | --------------------- | ------------------------------------ |
| **LysnrAI** | ✅ | ✅ user-dashboard | ✅ | ✅ (conditional) | ✅ Ready |
| **MindLyst** | ✅ | ✅ | ✅ | ✅ (conditional) | ✅ Ready |
| **ChronoMind** | ✅ | ✅ | ✅ | ✅ (conditional) | ✅ Ready |
| **JarvisJr** | ✅ | ✅ | ✅ | ✅ (conditional) | ✅ Ready |
| **PeakPulse** | ✅ | — (no web) | ✅ | — | ✅ Ready |
| **FlowMonk** | ✅ | ✅ | ✅ | ✅ (conditional) | ✅ Ready |
| **NomGap** | ✅ | ✅ | ✅ | ✅ | ✅ Fixed (added `.tarballs/` COPY) |
| **NoteLett** | ✅ | ✅ | ✅ | ✅ | ✅ Fixed (explicit COPY, not `.`) |
| **ActionTrail** | ✅ | ✅ | ✅ | ✅ | ✅ Ready (uses `.tarballs/` pattern) |
| **LocalMemGPT** | ✅ | ✅ | ✅ | ✅ | ✅ Ready (repo-root build context) |
| **admin-web** | — | ✅ (in common-plat) | N/A (pnpm) | ✅ (conditional) | ✅ Ready |
| **tracker-web** | — | ✅ (in common-plat) | N/A (pnpm) | ✅ (conditional) | ✅ Ready |
| Repo | Backend Dockerfile | Web Dockerfile | `docker-prep.sh` | `output:'standalone'` | Package manager state | Lockfile state | Docker template type | Status |
| --------------- | ------------------ | ------------------- | ---------------- | --------------------- | ---------------------------------------- | ---------------------------------- | ------------------------------- | ------------------------------------ |
| **LysnrAI** | ✅ | ✅ user-dashboard | ✅ | ✅ (conditional) | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready |
| **MindLyst** | ✅ | ✅ | ✅ | ✅ (conditional) | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready |
| **ChronoMind** | ✅ | ✅ | ✅ | ✅ (conditional) | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready |
| **JarvisJr** | ✅ | ✅ | ✅ | ✅ (conditional) | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready |
| **PeakPulse** | ✅ | — (no web) | ✅ | — | No Node web surface in this repo | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready |
| **FlowMonk** | ✅ | ✅ | ✅ | ✅ (conditional) | **Pilot candidate** for `pnpm` migration | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready |
| **NomGap** | ✅ | ✅ | ✅ | ✅ | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Fixed (added `.tarballs/` COPY) |
| **NoteLett** | ✅ | ✅ | ✅ | ✅ | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Fixed (explicit COPY, not `.`) |
| **ActionTrail** | ✅ | ✅ | ✅ | ✅ | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready (uses `.tarballs/` pattern) |
| **LocalMemGPT** | ✅ | ✅ | ✅ | ✅ | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready (repo-root build context) |
| **admin-web** | — | ✅ (in common-plat) | N/A (`pnpm`) | ✅ (conditional) | `pnpm` workspace today | `pnpm-lock.yaml` via common-plat | `pnpm` workspace template | ✅ Ready |
| **tracker-web** | — | ✅ (in common-plat) | N/A (`pnpm`) | ✅ (conditional) | `pnpm` workspace today | `pnpm-lock.yaml` via common-plat | `pnpm` workspace template | ✅ Ready |
**All 10 product repos now have Dockerfiles, `docker-prep.sh`, and `output:'standalone'`.** Created 2026-03-22.
> **Note:** The table above tracks Docker readiness, not completed package-manager migration. For product repos, use each repo's actual `packageManager` field and lockfile until that repo is explicitly migrated to `pnpm`.
---
## 11. Dockerfile Template (reference)
> **Critical:** These templates assume you run `docker-prep.sh` first to pack `@bytelyst/*` file: deps into `.tarballs/`. Without this, `npm ci` will fail because `file:../../learning_ai_common_plat/packages/*` doesn't exist inside the Docker build context.
> **Critical:** Run `docker-prep.sh` first for product repos that use `@bytelyst/*` `file:` dependencies. The prep step packs those dependencies into `.tarballs/` so Docker builds can resolve them inside the repo's own build context. During the migration window, Dockerfiles must match the repo's package manager and lockfile instead of assuming a single global install command.
### Backend (Fastify 5 + TypeScript)
### Backend / service template — `npm` repo variant
```dockerfile
# Pre-requisite: run ./scripts/docker-prep.sh to pack @bytelyst/* tarballs
@ -851,7 +874,41 @@ EXPOSE ${PORT:-4010}
CMD ["node", "dist/server.js"]
```
### Web (Next.js 16)
### Backend / service template — `pnpm` repo variant
```dockerfile
# Pre-requisite: run ./scripts/docker-prep.sh if this repo rewrites @bytelyst/* file: deps
FROM node:22-alpine AS builder
WORKDIR /app
RUN corepack enable && corepack prepare pnpm@10 --activate
COPY package.json pnpm-lock.yaml ./
COPY .tarballs/ ./.tarballs/
RUN pnpm install --frozen-lockfile --ignore-scripts
COPY tsconfig.json ./
COPY src/ ./src/
RUN pnpm run build
FROM node:22-alpine
WORKDIR /app
ENV NODE_ENV=production
RUN corepack enable && corepack prepare pnpm@10 --activate
COPY package.json pnpm-lock.yaml ./
COPY .tarballs/ ./.tarballs/
RUN pnpm install --frozen-lockfile --prod --ignore-scripts
COPY --from=builder /app/dist ./dist
COPY shared/ ./shared/ 2>/dev/null || true
EXPOSE ${PORT:-4010}
CMD ["node", "dist/server.js"]
```
### Web (Next.js 16) — `npm` repo variant
> **Prerequisite:** `next.config.ts` MUST have `output: 'standalone'` for the standalone Dockerfile pattern to work. Without it, `.next/standalone/` won't be generated and the COPY will fail.
@ -884,15 +941,121 @@ EXPOSE 3000
CMD ["node", "server.js"]
```
### Web (Next.js 16) — `pnpm` repo variant
> **Prerequisite:** `next.config.ts` MUST have `output: 'standalone'` for the standalone Dockerfile pattern to work. Keep the repo's `build` script authoritative, including `--webpack` where required.
```dockerfile
# Pre-requisite: run ./scripts/docker-prep.sh to pack @bytelyst/* tarballs when applicable
FROM node:22-alpine AS builder
WORKDIR /app
RUN corepack enable && corepack prepare pnpm@10 --activate
COPY package.json pnpm-lock.yaml ./
COPY .tarballs/ ./.tarballs/
RUN pnpm install --frozen-lockfile
COPY . .
# Dummy env vars for Next.js build-time static page collection
ENV NEXT_PUBLIC_BACKEND_URL=http://localhost:4010
ENV NEXT_PUBLIC_PLATFORM_SERVICE_URL=http://localhost:4003
RUN pnpm run build
FROM node:22-alpine
WORKDIR /app
ENV NODE_ENV=production
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static
COPY --from=builder /app/public ./public 2>/dev/null || true
EXPOSE 3000
CMD ["node", "server.js"]
```
> **Template selection rule:**
>
> - Use the `npm` variant only for repos that are still on `npm` with `package-lock.json` and matching Docker/CI scripts.
> - Use the `pnpm` variant for repos that have migrated to `pnpm` and carry `pnpm-lock.yaml` plus aligned CI/Docker/docs.
> - Do **not** leave a repo in mixed state after migration.
### docker-prep.sh (for repos that don't have one yet)
Copy from `learning_ai_trails/scripts/docker-prep.sh` — it handles both `backend/` and `web/` targets, packs all `file:` refs into `.tarballs/`, and rewrites `package.json` to point at them.
The important rule is **behavior**, not shell-script ancestry:
- `docker-prep.sh` must support both legacy `npm` repos and migrated `pnpm` repos.
- It must **not** hardcode `npm` assumptions into tarball rewrite flow.
- It must preserve the repo's package-manager semantics after prep:
- keep the correct lockfile
- keep the correct install command in Docker/CI
- keep `.tarballs/` handling compatible with the repo's active package manager
```bash
cp learning_ai_trails/scripts/docker-prep.sh <target-repo>/scripts/docker-prep.sh
chmod +x <target-repo>/scripts/docker-prep.sh
```
## 11.1 Long-Term Package-Manager Migration Roadmap
### End-state
- `learning_ai_common_plat` remains the canonical **`pnpm` workspace** monorepo.
- Node-based product repos migrate to **`pnpm` over time**.
- Product repos remain **independent repositories**, not one combined workspace.
- Current `.tarballs/` handling for `@bytelyst/*` remains supported unless it is explicitly simplified later.
### Migration principles
- No big-bang migration.
- One repo at a time.
- Fully green before moving to the next repo.
- Do not combine package-manager migration with unrelated dependency upgrades.
- Migrate CI, Docker, and docs together in the same repo migration.
- No mixed lockfile/package-manager state after migration.
### Phase 0 — policy and checklist
- Define package-manager policy.
- Define migration checklist.
- Define validation gates.
### Pilot
- `learning_ai_flowmonk`
### Wave 1
- `learning_ai_trails`
- `learning_ai_local_memory_gpt`
### Wave 2
- `learning_ai_notes`
- `learning_ai_fastgap`
- `learning_ai_clock`
### Wave 3
- `learning_ai_jarvis_jr`
- `learning_voice_ai_agent`
### Validation gates per migrated repo
A repo is only considered migrated when all of the following are aligned and passing:
- install
- test
- typecheck
- build
- Docker build
- local shared package resolution
- docs/CI updated
---
## 12. Audit Findings (Review 2026-03-22)
@ -921,9 +1084,12 @@ Systematic code review of all claims in this document against the actual codebas
1. Runs `pnpm build` in common-plat
2. Packs each `@bytelyst/*` package into a `.tarballs/*.tgz`
3. Rewrites package.json `file:` refs → `file:.tarballs/bytelyst-*.tgz`
4. Preserves the product repo's active package-manager semantics during the rewrite
**All 10 repos now have `docker-prep.sh`** (created 2026-03-22). Previously only ActionTrail, LocalMemGPT, NoteLett, NomGap had them.
> **Long-term note:** As product repos migrate to `pnpm`, this pattern remains valid. What changes is the repo-local install/runtime contract (`pnpm install --frozen-lockfile` instead of `npm ci`), not the deployment architecture or the need to package `@bytelyst/*` dependencies for isolated Docker contexts.
### F3. NomGap Backend Dockerfile Ignores `file:` Deps (BUG)
`@/learning_ai_fastgap/backend/Dockerfile` does `COPY package.json → npm ci` but doesn't copy `.tarballs/`. The `file:` refs will fail. Needs the `.tarballs/` COPY step added.
@ -967,7 +1133,7 @@ Existing Dockerfiles use mixed Node versions:
### F8. Missing `--webpack` Flag for Next.js Builds
Several web apps require `--webpack` flag for builds (Serwist PWA incompatible with Turbopack, or `@bytelyst/*` file: ref transpilation). The Dockerfile template uses `npm run build` which should map to `next build --webpack` in package.json — verify each repo's `build` script.
Several web apps require `--webpack` flag for builds (Serwist PWA incompatible with Turbopack, or `@bytelyst/*` file: ref transpilation). The Dockerfile template should call the repo's package-manager-appropriate build command (`npm run build` or `pnpm run build`) and that script should map to `next build --webpack` where required.
### F9. Missing `.env.ecosystem` Template
@ -1319,6 +1485,7 @@ esac
| ------------------------------ | -------------------------------------------------------------------------------------------------------------- |
| **Can deploy on single VM?** | **Yes.** All ~25 services fit in 32 GB RAM. |
| **All Dockerized?** | **Yes.** All 10 product repos now have Dockerfiles + docker-prep.sh. |
| **Package-manager direction?** | **`pnpm` is the long-term standard** for Node/TS repos, but migration is phased repo-by-repo, not big-bang. |
| **K8s practice on single VM?** | **Docker Desktop K8s** (Mac/Windows) or **K3s** (Linux). Same manifests scale to AKS/EKS/GKE. |
| **Recommended VM?** | 8 vCPU / 32 GB (min) or 16 vCPU / 64 GB (with Ollama). Hetzner ~$45/mo for dev. |
| **Time to production K8s?** | Phase 1 (compose) → Phase 2 (Docker Desktop / K3s) → Phase 3 (multi-node) → Phase 4 (managed). Same manifests. |