bytelyst/learning_ai_common_plat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(devops): record local gitea registry rehearsal

Browse Source
This commit is contained in:
saravanakumardb1 2026-03-23 15:48:18 -07:00
parent 1d2003a30f
commit 7116749bbd
3 changed files with 843 additions and 335 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

490
docs/devops/GITEA_NPM_REGISTRY_MIGRATION.md Normal file
View File

@ -0,0 +1,490 @@
# Gitea npm Package Registry Migration
> **Goal:** Replace the `docker-prep.sh` + tarball workflow with Gitea's built-in npm package registry for `@bytelyst/*` shared packages.
>
> **Phase policy:** use **this Mac as the future single VM** and validate the entire flow locally first. No Azure rollout and no publishing to any external or corporate registry in this phase.
---
## 1. Decision
We should adopt the local Gitea npm registry as the long-term package-distribution path for `@bytelyst/*` packages.
We should **not** switch directly in Azure first.
Instead, we should validate the exact end-to-end flow locally on this Mac:
- local Gitea git hosting
- local Gitea Actions
- local Gitea npm package registry
- local Docker / Docker Compose builds
- local product repos consuming `@bytelyst/*` from local Gitea
Once that is stable, we replicate the same validated shape on the Azure single VM.
---
## 2. Why We Want This
### Current pain
Today many repos still depend on one of these Docker-time workarounds:
- `file:` references to sibling `learning_ai_common_plat/packages/*`
- `docker-prep.sh`
- `.docker-deps` or older `.tarballs` copy steps
- temporary `package.json` rewriting before builds
This works, but it creates repeated operational drag:
- every build may need prep
- Dockerfiles become repo-specific and fragile
- CI, local development, and Docker are not using one clean dependency source
- path and build-context mismatches keep surfacing during ecosystem validation
### Target outcome
We want one dependency source for shared packages:
- local development
- local CI via Gitea Actions
- local Docker builds
- later Azure single-VM builds
That dependency source should be the **local Gitea npm registry**.
---
## 3. Local-First Target Architecture
### 3.1 This Mac as the rehearsal VM
```text
This Mac
├── local Gitea (git + actions + npm registry)
├── local Docker / Docker Compose
├── learning_ai_common_plat
└── product repos
```
Flow:
```text
learning_ai_common_plat/packages/*
↓ build locally
↓ publish to local Gitea npm registry
local Gitea npm registry (localhost:3300)
↓ install via semver refs
product repos / Docker builds / local Gitea CI
```
### 3.2 Future Azure single VM
After the local rehearsal is proven, Azure should mirror the same shape:
```text
Azure VM
├── Gitea
├── Gitea Actions runner
├── Gitea npm registry
├── Docker / Compose
└── same repo + build flow
```
The Azure plan should reuse the validated local approach, not invent a second design.
---
## 4. What Is In Scope Right Now
### Included
- validate local Gitea package-registry access
- validate local Gitea Actions availability
- define local-only package publish flow to local Gitea
- define local-only consumer install flow from local Gitea
- define local-only Docker build flow from local Gitea
- define the later Azure single-VM replication steps
### Excluded
- Azure changes right now
- external npm registry publishing
- corporate registry publishing
- non-local deployment actions
- K3s / Helm work
---
## 5. Current Local Readiness Snapshot
Validated locally on this Mac:
- local Gitea health endpoint responds
- local Gitea version endpoint responds
- local `bytelyst` user auth works
- local Gitea Actions workflow exists for `learning_ai_common_plat`
- local Gitea package registry endpoint is reachable
- local pilot packages have been published to local Gitea successfully
- a clean scratch consumer install from local Gitea works on the host
That is enough to treat this Mac as the single-VM rehearsal environment.
---
## 5.1 Validation Results So Far
### Proven locally on this Mac
- shared packages build cleanly in `learning_ai_common_plat`
- a local-only Gitea package token was created and used successfully
- pilot packages were published to local Gitea npm registry successfully
- a clean scratch `pnpm install` from the local Gitea registry worked on the host
- `learning_ai_flowmonk` was migrated on the host from tarball refs to semver `@bytelyst/*` refs
- `learning_ai_flowmonk` host-side install, typecheck, and tests passed against the local Gitea registry
### Important implementation finding
Publishing `@bytelyst/*` workspace packages with raw `npm publish` is not sufficient for this migration.
Why:
- packages with internal `workspace:*` dependencies can be published with unresolved workspace specifiers
- those published artifacts break downstream consumers
What worked:
- `pnpm pack` produces normalized package metadata for workspace dependencies
- publishing the normalized tarball to Gitea works
This is now codified in the local-only helper script:
`scripts/publish-local-gitea-packages.sh`
That script is currently the authoritative local rehearsal path for publishing `@bytelyst/*` packages to local Gitea.
### Current blocker
The remaining blocker is not host-side package publishing or host-side package consumption.
The blocker is specifically:
- Docker / BuildKit installs against the local Homebrew Gitea instance on this Mac
- registry metadata and tarball fetch behavior across the Mac host ↔ Docker build boundary
At the time of writing:
- host-side registry usage is validated
- FlowMonk host-side pilot migration is validated
- Docker-side local-registry validation is still blocked
That blocker must be resolved before we can claim full local E2E completion.
---
## 6. Migration Strategy
### Stage A — Local registry rehearsal
Validate the package-registry path locally before changing every repo.
Required outcomes:
1. local package token exists
2. a small pilot set of `@bytelyst/*` packages can be published to local Gitea
3. a local consumer can install them via semver refs
4. a Docker build can install them without `docker-prep.sh`
5. local Gitea Actions can run the same package build/publish path
### Stage B — Pilot repo migration
Migrate one repo first.
Recommended pilot:
- `learning_ai_flowmonk`
Reason:
- recent Docker/build work already exposed its edge cases
- both backend and web surfaces exist
- it is a strong canary for dependency-distribution changes
### Stage C — Sequential ecosystem rollout
After the pilot is stable, migrate remaining repos sequentially.
---
## 7. Local Implementation Plan
### 7.1 Verify package metadata in `learning_ai_common_plat`
Every `packages/*/package.json` should have:
- a valid `name`
- a valid `version`
- a valid `files` / `exports` setup
- a successful local `build`
Local validation command:
```bash
cd /Users/sd9235/code/mygh/learning_ai_common_plat
pnpm -r --filter './packages/*' build
```
### 7.2 Create a local-only Gitea package token
Use a token intended only for this Mac rehearsal.
Required scopes:
- package read
- package write
Do not use Azure or external registry credentials here.
### 7.3 Publish only to local Gitea
In this phase, all `npm publish` activity must point only to:
`http://localhost:3300/api/packages/bytelyst/npm/`
No Azure Artifacts.
No npmjs.org.
No external corporate npm registry.
Use the local-only helper script for this rehearsal:
```bash
cd /Users/sd9235/code/mygh/learning_ai_common_plat
GITEA_NPM_TOKEN='<local-token>' bash ./scripts/publish-local-gitea-packages.sh
```
For a single package:
```bash
cd /Users/sd9235/code/mygh/learning_ai_common_plat
GITEA_NPM_TOKEN='<local-token>' bash ./scripts/publish-local-gitea-packages.sh '@bytelyst/errors'
```
### 7.4 Start with a minimal package pilot set
Publish a small set first:
- `@bytelyst/errors`
- `@bytelyst/config`
- `@bytelyst/api-client`
Then expand once the local install path is proven.
### 7.5 Consumer dependency model after migration
Consumers move from local file refs:
```json
"@bytelyst/auth": "file:../../learning_ai_common_plat/packages/auth"
```
to semver refs:
```json
"@bytelyst/auth": "^0.1.0"
```
And resolve via scoped `.npmrc` or `.pnpmrc` config pointing `@bytelyst` to local Gitea.
### 7.6 Docker model after migration
After a repo is migrated to the registry model, its Dockerfiles should:
- stop copying `.docker-deps` or `.tarballs`
- stop depending on `docker-prep.sh`
- install from local Gitea through scoped registry config
That is the main operational simplification we want.
---
## 8. Local Validation Sequence
### Step 1 — Validate local Gitea surfaces
Confirm locally:
- Gitea UI is reachable
- package registry API is reachable with auth
- Actions workflow is visible and active
### Step 2 — Validate package build output
Run local builds for the shared packages and confirm pack/publish inputs are clean.
### Step 3 — Publish a minimal package pilot set to local Gitea only
Success means:
- packages appear in local Gitea package list
- metadata looks correct
- no external registry interaction occurred
### Step 4 — Validate local install from Gitea
Use a scratch consumer or a pilot repo surface and verify:
- `pnpm install` resolves `@bytelyst/*` from local Gitea
- lockfile updates are clean
- no fallback to tarballs is required for the tested packages
### Step 5 — Validate Docker build from local Gitea
For the pilot repo:
- remove the tested surface's dependency on `docker-prep.sh`
- add scoped registry config for Docker
- build backend and web without tarball prep
Success means:
- image build works without `.docker-deps` or `.tarballs`
- build does not require local package rewrite logic
Current status on this Mac:
- **not yet fully green**
- the remaining issue is the local Homebrew Gitea instance serving package metadata/tarball URLs across the Docker build boundary
- host-side validation is complete, Docker-side validation is still open
### Step 6 — Validate local Gitea Actions path
Add or adapt a local-only workflow in `learning_ai_common_plat` that:
- builds packages
- optionally publishes to local Gitea npm registry
- never targets an external registry
### Step 7 — Validate full local E2E
At the end of the local rehearsal we should be able to say:
- shared packages build locally
- shared packages publish to local Gitea registry
- a pilot consumer installs from local Gitea
- a pilot Docker build succeeds from local Gitea
- local Gitea Actions can drive the same path
Only then should we expand repo-by-repo.
---
## 9. Recommended Rollout Order
### Pilot
1. `learning_ai_flowmonk`
### Then
2. `learning_ai_local_memory_gpt`
3. `learning_ai_notes`
4. `learning_ai_trails`
5. `learning_ai_fastgap`
6. `learning_ai_clock`
7. `learning_ai_jarvis_jr`
8. `learning_ai_peakpulse`
9. `learning_multimodal_memory_agents`
10. `learning_voice_ai_agent`
The ordering prioritizes recently exercised Docker paths before higher blast-radius repos.
---
## 10. CI Model For The Local Rehearsal
For the local rehearsal phase, Gitea Actions should do only local work:
- build packages
- test packages
- typecheck packages
- optionally publish to **local Gitea only**
This is intentionally different from a final enterprise rollout.
The objective is to prove the local VM pattern first.
---
## 11. Risks And Guardrails
### Risks
- version drift between local source and published package versions
- peer dependency mismatches hidden by current tarball workflow
- Docker auth/config differences from shell installs
- accidental publishing to the wrong registry
- switching too many repos before the pilot is stable
### Guardrails
- use local-only credentials for this phase
- keep `@bytelyst` scoped registry config explicit
- do not remove tarball fallback globally until the pilot is green
- migrate one consumer repo at a time
- keep rollback steps documented per repo
---
## 12. Rollback Plan
If the registry-based flow fails during pilot migration:
1. revert the tested consumer back to its current working dependency mode
2. restore its `docker-prep.sh` path if needed
3. keep the Gitea registry work isolated to the local rehearsal branch/state
4. fix the issue locally before retrying
No repo should lose its known-good build path until the registry model is proven.
---
## 13. Azure Single-VM Follow-Through
After the local rehearsal is green, Azure should follow the same validated recipe:
1. provision one VM
2. install Docker and Gitea
3. enable Gitea Actions runner
4. enable Gitea packages
5. clone the same repos onto the VM
6. apply the same local registry/token/package flow
7. re-run the pilot repo first
8. then expand sequentially across the ecosystem
Azure should be a replication step, not a redesign step.
---
## 14. Definition Of Done
This migration plan is locally validated only when all are true:
- [x] local Gitea package-registry auth verified
- [x] local package publish path verified for a pilot package set
- [x] local consumer install path verified
- [ ] local Docker build path verified without `docker-prep.sh`
- [ ] local Gitea Actions path verified
- [ ] pilot repo migrated successfully end-to-end including Docker
- [ ] rollback path documented and tested conceptually
- [ ] Azure single-VM reproduction steps documented from the validated local process
---
## 15. Immediate Next Actions
1. create or verify a local-only Gitea package token
2. publish a minimal pilot `@bytelyst/*` package set to local Gitea only
3. validate install from local Gitea in one pilot consumer
4. validate Docker build from local Gitea in that pilot repo
5. validate the same path via local Gitea Actions
6. only then expand to broader ecosystem migration

611
docs/devops/SINGLE_VM_DEPLOYMENT.md
View File

@ -27,17 +27,19 @@
### Shared Infrastructure (common-plat) ### Shared Infrastructure (common-plat)
| Service | Port | Image | RAM Est. | | Service | Port | Image | RAM Est. |
| ------------------------ | ---------- | ---------------------------------------------------------------------- | -------- | | ----------------------------------- | ---------- | ---------------------------------------------------------------------- | -------- |
| **platform-service** | 4003 | Fastify 5 + TS | ~200 MB | | **platform-service** | 4003 | Fastify 5 + TS | ~200 MB |
| **extraction-service** | 4005 | Fastify 5 + Python sidecar | ~350 MB | | **extraction-service** | 4005 | Fastify 5 + Python sidecar | ~350 MB |
| **mcp-server** | 4007 | Fastify 5 + TS | ~150 MB | | **mcp-server** | 4007 | Fastify 5 + TS | ~150 MB |
| **Cosmos DB Emulator** | 8081, 1234 | `mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-preview` | ~2 GB | | **Cosmos DB Emulator** | 8081, 1234 | `mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-preview` | ~2 GB |
| **Azurite** (blob) | 10000 | `mcr.microsoft.com/azure-storage/azurite` | ~100 MB | | **Azurite** (blob) | 10000 | `mcr.microsoft.com/azure-storage/azurite` | ~100 MB |
| **Mailpit** (SMTP) | 1025, 8025 | `axllent/mailpit` | ~50 MB | | **Mailpit** (SMTP) | 1025, 8025 | `axllent/mailpit` | ~50 MB |
| **Traefik** (gateway) | 80, 8080 | `traefik:v3.3` | ~100 MB | | **Traefik** (gateway) | 80, 8080 | `traefik:v3.3` | ~100 MB |
| **Loki** (logs) | 3100 | `grafana/loki` | ~200 MB | | **Loki** (logs) | 3100 | `grafana/loki` | ~200 MB |
| **Grafana** (dashboards) | 3000 | `grafana/grafana` | ~200 MB | | **Grafana** (dashboards) | 3000 | `grafana/grafana` | ~200 MB |
| **Gitea** (git + CI + pkg registry) | 3300, 2222 | `gitea/gitea` | ~150 MB |
| **act_runner** (CI runner) | — | `gitea/act_runner` (or host-mode binary) | ~100 MB |
### Product Backends (Fastify 5 + TypeScript) ### Product Backends (Fastify 5 + TypeScript)
@ -98,8 +100,9 @@
- 3 shared services × 250 MB = ~0.75 GB - 3 shared services × 250 MB = ~0.75 GB
- 11 Next.js webs × 200 MB = ~2.2 GB - 11 Next.js webs × 200 MB = ~2.2 GB
- Infra (Traefik, Loki, Grafana, Azurite, Mailpit) = ~0.65 GB - Infra (Traefik, Loki, Grafana, Azurite, Mailpit) = ~0.65 GB
- Gitea + act_runner = ~0.25 GB
- K3s overhead = ~0.5 GB - K3s overhead = ~0.5 GB
- **Subtotal: ~7.4 GB** → headroom for spikes + build cache = **32 GB** - **Subtotal: ~7.65 GB** → headroom for spikes + build cache = **32 GB**
### Recommended (with Ollama, small models) ### Recommended (with Ollama, small models)
@ -241,6 +244,32 @@ services:
volumes: [grafana-data:/var/lib/grafana] volumes: [grafana-data:/var/lib/grafana]
restart: unless-stopped restart: unless-stopped
# ══════════════════════════════════════════════════════
# GIT + CI + PACKAGE REGISTRY
# ══════════════════════════════════════════════════════
gitea:
image: gitea/gitea:1.23
ports: ['3300:3300', '2222:22']
environment:
GITEA__server__HTTP_PORT: 3300
GITEA__server__ROOT_URL: http://gitea:3300
GITEA__packages__ENABLED: 'true' # Built-in npm/container registry
GITEA__service__DISABLE_REGISTRATION: 'true'
volumes:
- gitea-data:/data
restart: unless-stopped
act-runner:
image: gitea/act_runner:latest
environment:
GITEA_INSTANCE_URL: http://gitea:3300
GITEA_RUNNER_REGISTRATION_TOKEN: '<generate-from-gitea-admin>'
GITEA_RUNNER_LABELS: 'ubuntu-latest:host'
volumes:
- /var/run/docker.sock:/var/run/docker.sock
depends_on: [gitea]
restart: unless-stopped
# ══════════════════════════════════════════════════════ # ══════════════════════════════════════════════════════
# SHARED SERVICES (common-plat — no file: deps, pnpm workspace handles it) # SHARED SERVICES (common-plat — no file: deps, pnpm workspace handles it)
# ══════════════════════════════════════════════════════ # ══════════════════════════════════════════════════════
@ -479,6 +508,7 @@ volumes:
azurite-data: azurite-data:
loki-data: loki-data:
grafana-data: grafana-data:
gitea-data:
localmemgpt-data: localmemgpt-data:
``` ```
@ -663,8 +693,8 @@ kubectl scale deploy/platform-service --replicas=3 -n bytelyst-platform
```bash ```bash
kubectl create secret generic bytelyst-secrets \ kubectl create secret generic bytelyst-secrets \
--from-literal=JWT_SECRET=new-secret \ --from-literal=JWT_SECRET='<replace-with-real-jwt-secret>' \
--from-literal=COSMOS_KEY=new-key \ --from-literal=COSMOS_KEY='<replace-with-real-cosmos-key>' \
-n bytelyst-platform --dry-run=client -o yaml | kubectl apply -f - -n bytelyst-platform --dry-run=client -o yaml | kubectl apply -f -
kubectl rollout restart deploy -n bytelyst-platform kubectl rollout restart deploy -n bytelyst-platform
``` ```
@ -817,20 +847,21 @@ kubectl get pods -A
## 10. Dockerization Status (all complete) ## 10. Dockerization Status (all complete)
| Repo | Backend Dockerfile | Web Dockerfile | `docker-prep.sh` | `output:'standalone'` | Package manager state | Lockfile state | Docker template type | Status | | Repo | Backend Dockerfile | Web Dockerfile | `docker-prep.sh` | `output:'standalone'` | Package manager state | Lockfile state | Docker template type | Status |
| --------------- | ------------------ | ------------------- | ---------------- | --------------------- | ---------------------------------------- | ---------------------------------- | ------------------------------- | ------------------------------------ | | ------------------ | ------------------ | ------------------- | ---------------- | --------------------- | ---------------------------------------- | ---------------------------------- | ------------------------------- | ------------------------------------ |
| **LysnrAI** | ✅ | ✅ user-dashboard | ✅ | ✅ (conditional) | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready | | **LysnrAI** | ✅ | ✅ user-dashboard | ✅ | ✅ (conditional) | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready |
| **MindLyst** | ✅ | ✅ | ✅ | ✅ (conditional) | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready | | **MindLyst** | ✅ | ✅ | ✅ | ✅ (conditional) | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready |
| **ChronoMind** | ✅ | ✅ | ✅ | ✅ (conditional) | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready | | **ChronoMind** | ✅ | ✅ | ✅ | ✅ (conditional) | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready |
| **JarvisJr** | ✅ | ✅ | ✅ | ✅ (conditional) | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready | | **JarvisJr** | ✅ | ✅ | ✅ | ✅ (conditional) | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready |
| **PeakPulse** | ✅ | — (no web) | ✅ | — | No Node web surface in this repo | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready | | **PeakPulse** | ✅ | — (no web) | ✅ | — | No Node web surface in this repo | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready |
| **FlowMonk** | ✅ | ✅ | ✅ | ✅ (conditional) | **Pilot candidate** for `pnpm` migration | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready | | **FlowMonk** | ✅ | ✅ | ✅ | ✅ (conditional) | **Pilot candidate** for `pnpm` migration | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready |
| **NomGap** | ✅ | ✅ | ✅ | ✅ | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Fixed (added `.tarballs/` COPY) | | **NomGap** | ✅ | ✅ | ✅ | ✅ | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Fixed (added `.tarballs/` COPY) |
| **NoteLett** | ✅ | ✅ | ✅ | ✅ | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Fixed (explicit COPY, not `.`) | | **NoteLett** | ✅ | ✅ | ✅ | ✅ | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Fixed (explicit COPY, not `.`) |
| **ActionTrail** | ✅ | ✅ | ✅ | ✅ | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready (uses `.tarballs/` pattern) | | **ActionTrail** | ✅ | ✅ | ✅ | ✅ | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready (uses `.tarballs/` pattern) |
| **LocalMemGPT** | ✅ | ✅ | ✅ | ✅ | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready (repo-root build context) | | **LocalMemGPT** | ✅ | ✅ | ✅ | ✅ | Transitioning toward `pnpm` target | Follow repo-local current lockfile | Repo-specific during transition | ✅ Ready (repo-root build context) |
| **admin-web** | — | ✅ (in common-plat) | N/A (`pnpm`) | ✅ (conditional) | `pnpm` workspace today | `pnpm-lock.yaml` via common-plat | `pnpm` workspace template | ✅ Ready | | **admin-web** | — | ✅ (in common-plat) | N/A (`pnpm`) | ✅ (conditional) | `pnpm` workspace today | `pnpm-lock.yaml` via common-plat | `pnpm` workspace template | ✅ Ready |
| **tracker-web** | — | ✅ (in common-plat) | N/A (`pnpm`) | ✅ (conditional) | `pnpm` workspace today | `pnpm-lock.yaml` via common-plat | `pnpm` workspace template | ✅ Ready | | **tracker-web** | — | ✅ (in common-plat) | N/A (`pnpm`) | ✅ (conditional) | `pnpm` workspace today | `pnpm-lock.yaml` via common-plat | `pnpm` workspace template | ✅ Ready |
| **user-dashboard** | — | ✅ (in common-plat) | N/A (`pnpm`) | ✅ (conditional) | `pnpm` workspace today | `pnpm-lock.yaml` via common-plat | `pnpm` workspace template | ✅ Ready |
**All 10 product repos now have Dockerfiles, `docker-prep.sh`, and `output:'standalone'`.** Created 2026-03-22. **All 10 product repos now have Dockerfiles, `docker-prep.sh`, and `output:'standalone'`.** Created 2026-03-22.
@ -1058,6 +1089,181 @@ A repo is only considered migrated when all of the following are aligned and pas
--- ---
## 11.2 Gitea — Self-Hosted Git + CI + Package Registry
### Why Gitea on the VM?
Gitea is a lightweight, self-hosted Git forge (~150 MB RAM) that provides three capabilities in one container:
1. **Git hosting** — mirror or primary for all 13+ repos
2. **CI/CD** — GitHub Actions–compatible workflows via `act_runner`
3. **npm package registry** — built-in, eliminates the `docker-prep.sh` tarball workflow
### Current pain: `docker-prep.sh` + `.tgz` tarballs
Every product repo has `file:../../learning_ai_common_plat/packages/*` dependencies. These don't resolve inside Docker build contexts, so each repo needs a `docker-prep.sh` that:
1. Builds all `@bytelyst/*` packages in common-plat
2. Packs each into a `.tarballs/*.tgz` file
3. Rewrites package.json `file:` refs → `file:.tarballs/bytelyst-*.tgz`
4. Preserves the repo's active package-manager semantics during the rewrite
This is fragile, slow, and must be repeated for every image rebuild.
### Solution: Gitea npm Package Registry
Gitea has a built-in npm registry at `http://gitea:3300/api/packages/<org>/npm/`. Publishing `@bytelyst/*` packages there means **all product repos install from the registry like any normal npm dependency** — no tarballs, no rewriting, no `docker-prep.sh`.
> **Current local rehearsal status on this Mac:** host-side package publishing and host-side consumer installs are validated. Full Docker-side validation against the local Homebrew Gitea instance is still blocked by host/Docker registry URL reachability and tarball URL generation behavior. Treat the Docker portion of this section as the target architecture, not yet a fully cleared local result.
#### Setup (one-time)
```bash
# 1. Create a Gitea API token (Settings → Applications → Generate Token)
# Scopes: package:write, package:read
# 2. Configure npm/pnpm to use Gitea registry for @bytelyst scope
# In ~/.npmrc or project .npmrc:
@bytelyst:registry=http://localhost:3300/api/packages/bytelyst/npm/
//localhost:3300/api/packages/bytelyst/npm/:_authToken=<gitea-token>
# For Docker builds, use the compose service name:
@bytelyst:registry=http://gitea:3300/api/packages/bytelyst/npm/
```
#### Publish workflow (CI or manual)
```bash
# In learning_ai_common_plat — publish all packages after build
cd /path/to/learning_ai_common_plat
# Build all packages
pnpm -r --filter './packages/**' build
# Publish via the local helper that uses the proven pnpm-pack-based flow
GITEA_NPM_TOKEN='<local-token>' bash ./scripts/publish-local-gitea-packages.sh
```
This helper is important because raw `npm publish` is not sufficient for packages that contain internal `workspace:*` references. The validated local path uses `pnpm pack` first so workspace dependencies are normalized before the tarball is published to Gitea.
Or add a CI step in `learning_ai_common_plat/.gitea/workflows/ci.yml`:
```yaml
publish-packages:
name: Publish @bytelyst/* to Gitea npm registry
runs-on: ubuntu-latest
needs: [build-test-typecheck]
steps:
- name: Pull latest
working-directory: /path/to/learning_ai_common_plat
run: git pull --ff-only origin main || true
- name: Build packages
working-directory: /path/to/learning_ai_common_plat
run: pnpm -r --filter './packages/**' build
- name: Publish to Gitea registry
working-directory: /path/to/learning_ai_common_plat
env:
NPM_TOKEN: ${{ secrets.GITEA_NPM_TOKEN }}
run: |
echo "@bytelyst:registry=http://gitea:3300/api/packages/bytelyst/npm/" > .npmrc
echo "//gitea:3300/api/packages/bytelyst/npm/:_authToken=${NPM_TOKEN}" >> .npmrc
for pkg in packages/*/; do
(cd "$pkg" && npm publish --registry http://gitea:3300/api/packages/bytelyst/npm/ 2>/dev/null || true)
done
```
#### Product repo migration
For each product repo, replace `file:` refs with versioned (or `latest`) registry refs:
```jsonc
// Before (package.json)
{
"@bytelyst/auth": "file:../../learning_ai_common_plat/packages/auth",
"@bytelyst/config": "file:../../learning_ai_common_plat/packages/config",
"@bytelyst/cosmos": "file:../../learning_ai_common_plat/packages/cosmos"
}
// After
{
"@bytelyst/auth": "^0.1.0",
"@bytelyst/config": "^0.1.0",
"@bytelyst/cosmos": "^0.1.0"
}
```
With `.npmrc` pointing `@bytelyst` scope to Gitea, host-side `pnpm install` / `npm install` resolves from the local registry successfully in the current local rehearsal.
For Docker builds, the final single-VM target is still the same, but local validation on this Mac showed that a Homebrew-hosted Gitea instance may require extra reachability and `ROOT_URL` alignment work before BuildKit can consume package metadata and tarballs reliably.
#### Dockerfile simplification (after registry migration)
```dockerfile
# NO MORE docker-prep.sh or .tarballs/ needed!
FROM node:22-alpine AS builder
WORKDIR /app
# .npmrc baked in (or passed as build arg)
COPY .npmrc ./
COPY package.json pnpm-lock.yaml ./
RUN corepack enable && pnpm install --frozen-lockfile
COPY tsconfig.json ./
COPY src/ ./src/
RUN pnpm run build
FROM node:22-alpine
WORKDIR /app
ENV NODE_ENV=production
COPY .npmrc ./
COPY package.json pnpm-lock.yaml ./
RUN corepack enable && pnpm install --frozen-lockfile --prod
COPY --from=builder /app/dist ./dist
EXPOSE ${PORT:-4010}
CMD ["node", "dist/server.js"]
```
#### Comparison: before vs after
| Aspect | Current (`docker-prep.sh`) | With Gitea npm registry |
| ------------------- | ----------------------------------- | ---------------------------------------- |
| **Pre-build step** | Run `docker-prep.sh` per repo | None (registry resolves deps) |
| **Package format** | `.tgz` tarballs in `.tarballs/` | Standard npm registry install |
| **`package.json`** | Rewritten with `file:.tarballs/...` | Normal `^0.1.0` semver refs |
| **Docker build** | Must COPY `.tarballs/` into context | Standard `pnpm install` |
| **Version pinning** | Whatever was packed at build time | Explicit semver in lockfile |
| **Local dev** | `file:` symlinks (fast, direct) | Can keep `file:` for dev OR use registry |
| **CI publish** | N/A | Auto-publish on common-plat push |
| **Disk overhead** | ~50 MB tarballs per repo | Centralized in Gitea storage |
#### Migration path (incremental)
1. **Phase 0:** Gitea running locally with packages enabled
2. **Phase 1:** Publish `@bytelyst/*` packages to local Gitea using the `pnpm pack`-based helper flow
3. **Phase 2:** Migrate one pilot repo on the host: replace tarball/file refs with semver refs and validate install/typecheck/test
4. **Phase 3:** Validate Docker builds against the local registry
5. **Phase 4:** Only after local Docker validation is green, remove `docker-prep.sh` patterns repo-by-repo
6. **Keep `file:` refs available** for local dev via overrides or link-based workflows if needed
### Gitea Container Registry (bonus)
Gitea also provides a built-in **Docker container registry** at `http://gitea:3300/v2/`. Instead of building images and importing them into K3s, you can:
```bash
# Tag and push to Gitea container registry
docker tag bytelyst/platform-service:latest gitea:3300/bytelyst/platform-service:latest
docker push gitea:3300/bytelyst/platform-service:latest
# K3s / K8s pulls from Gitea registry
# In deployment YAML: image: gitea:3300/bytelyst/platform-service:latest
```
This gives you a **complete self-hosted DevOps stack**: Git → CI → npm registry → container registry → deploy.
---
## 12. Audit Findings (Review 2026-03-22) ## 12. Audit Findings (Review 2026-03-22)
Systematic code review of all claims in this document against the actual codebase. Systematic code review of all claims in this document against the actual codebase.
@ -1084,7 +1290,7 @@ Systematic code review of all claims in this document against the actual codebas
1. Runs `pnpm build` in common-plat 1. Runs `pnpm build` in common-plat
2. Packs each `@bytelyst/*` package into a `.tarballs/*.tgz` 2. Packs each `@bytelyst/*` package into a `.tarballs/*.tgz`
3. Rewrites package.json `file:` refs → `file:.tarballs/bytelyst-*.tgz` 3. Rewrites package.json `file:` refs → `file:.tarballs/bytelyst-*.tgz`
4. Preserves the product repo's active package-manager semantics during the rewrite 4. Preserves the repo's active package-manager semantics during the rewrite
**All 10 repos now have `docker-prep.sh`** (created 2026-03-22). Previously only ActionTrail, LocalMemGPT, NoteLett, NomGap had them. **All 10 repos now have `docker-prep.sh`** (created 2026-03-22). Previously only ActionTrail, LocalMemGPT, NoteLett, NomGap had them.
@ -1092,11 +1298,15 @@ Systematic code review of all claims in this document against the actual codebas
### F3. NomGap Backend Dockerfile Ignores `file:` Deps (BUG) ### F3. NomGap Backend Dockerfile Ignores `file:` Deps (BUG)
`@/learning_ai_fastgap/backend/Dockerfile` does `COPY package.json → npm ci` but doesn't copy `.tarballs/`. The `file:` refs will fail. Needs the `.tarballs/` COPY step added. `@/learning_ai_fastgap/backend/Dockerfile` previously did `COPY package.json → npm ci` without copying `.tarballs/`, which broke `file:` dependency resolution inside Docker.
**Status:** ✅ Fixed. The backend Dockerfile was updated to include the `.tarballs/` pattern.
### F4. NoteLett Backend Dockerfile Copies Everything (BUG) ### F4. NoteLett Backend Dockerfile Copies Everything (BUG)
`@/learning_ai_notes/backend/Dockerfile` does `COPY . .` in the build stage, which includes broken `node_modules` symlinks from `file:` deps. Should use explicit `COPY` of `src/`, `tsconfig.json`, and `.tarballs/` instead. `@/learning_ai_notes/backend/Dockerfile` previously used `COPY . .` in the build stage, which pulled in broken `node_modules` symlinks from `file:` dependencies.
**Status:** ✅ Fixed. The backend Dockerfile now uses explicit `COPY` steps instead of broad context copy.
### F5. Missing `output: 'standalone'` in next.config.ts (CRITICAL) ### F5. Missing `output: 'standalone'` in next.config.ts (CRITICAL)
@ -1137,29 +1347,36 @@ Several web apps require `--webpack` flag for builds (Serwist PWA incompatible w
### F9. Missing `.env.ecosystem` Template ### F9. Missing `.env.ecosystem` Template
The compose references `.env.ecosystem` but the doc doesn't define its contents. Key vars needed: The compose references `.env.ecosystem`. A working template now exists at the workspace root. Current key vars:
```env ```env
# .env.ecosystem — shared env for all services # .env.ecosystem — shared env for all services
COSMOS_ENDPOINT=https://cosmos-emulator:8081 COSMOS_ENDPOINT=http://cosmos-emulator:8081
COSMOS_KEY=<emulator-key> COSMOS_KEY=<cosmos-emulator-key-or-local-placeholder>
COSMOS_DATABASE=bytelyst COSMOS_DATABASE=bytelyst
JWT_SECRET=dev-ecosystem-secret-change-me DB_PROVIDER=cosmos
JWT_SECRET=dev-ecosystem-secret-do-not-use-in-production
AZURE_BLOB_CONNECTION_STRING=DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=...;BlobEndpoint=http://azurite:10000/devstoreaccount1; AZURE_BLOB_CONNECTION_STRING=DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=...;BlobEndpoint=http://azurite:10000/devstoreaccount1;
PLATFORM_SERVICE_URL=http://platform-service:4003 PLATFORM_SERVICE_URL=http://platform-service:4003
EXTRACTION_SERVICE_URL=http://extraction-service:4005 EXTRACTION_SERVICE_URL=http://extraction-service:4005
DB_PROVIDER=memory
NODE_ENV=production
CORS_ORIGIN=*
SMTP_HOST=mailpit SMTP_HOST=mailpit
SMTP_PORT=1025 SMTP_PORT=1025
SMTP_FROM=noreply@bytelyst.local
FIELD_ENCRYPT_KEY_PROVIDER=env
FIELD_ENCRYPT_KEY=<64-char-hex-dev-key>
FIELD_ENCRYPT_MEK_NAME=dev-ecosystem-mek
NODE_ENV=production
CORS_ORIGIN=*
LOG_LEVEL=info
``` ```
**Status:** ✅ Implemented as `.env.ecosystem` in the workspace root.
### F10. `host.docker.internal` Only Works on Docker Desktop (Mac/Windows) ### F10. `host.docker.internal` Only Works on Docker Desktop (Mac/Windows)
LocalMemGPT uses `OLLAMA_URL: 'http://host.docker.internal:11434'` — this works on Docker Desktop but **not on Linux VMs** (which is the likely deployment target). LocalMemGPT uses `OLLAMA_URL: 'http://host.docker.internal:11434'` — this works on Docker Desktop but **not on Linux VMs** (which is the likely deployment target).
**Fix on Linux:** Add `extra_hosts: ['host.docker.internal:host-gateway']` to the service, or use `network_mode: host`. **Status:** ✅ Addressed in `docker-compose.ecosystem.yml` via `extra_hosts: ['host.docker.internal:host-gateway']` on `localmemgpt-backend`.
### Summary of Required Work Before Compose Works ### Summary of Required Work Before Compose Works
@ -1171,311 +1388,33 @@ LocalMemGPT uses `OLLAMA_URL: 'http://host.docker.internal:11434'` — this work
| **P0** | Add `output: 'standalone'` to next.config.ts | 3 webs | ✅ Done (4 webs: ChronoMind, JarvisJr, FlowMonk, MindLyst) | | **P0** | Add `output: 'standalone'` to next.config.ts | 3 webs | ✅ Done (4 webs: ChronoMind, JarvisJr, FlowMonk, MindLyst) |
| **P1** | Fix NomGap backend Dockerfile (add `.tarballs/` COPY) | 1 file | ✅ Done | | **P1** | Fix NomGap backend Dockerfile (add `.tarballs/` COPY) | 1 file | ✅ Done |
| **P1** | Fix NoteLett backend Dockerfile (explicit COPY, not `.`) | 1 file | ✅ Done | | **P1** | Fix NoteLett backend Dockerfile (explicit COPY, not `.`) | 1 file | ✅ Done |
| **P1** | Create `.env.ecosystem` template | 1 file | Pending | | **P1** | Create `.env.ecosystem` template | 1 file | ✅ Done |
| **P2** | Standardize Node.js version to 22-alpine | 4 Dockerfiles | ✅ Done (all new Dockerfiles use 22-alpine) | | **P2** | Standardize Node.js version to 22-alpine | 4 Dockerfiles | ✅ Done (all new Dockerfiles use 22-alpine) |
| **P2** | Add `extra_hosts` for Linux VM Ollama access | 1 service | Pending | | **P2** | Add `extra_hosts` for Linux VM Ollama access | 1 service | ✅ Done (`localmemgpt-backend`) |
--- ---
## 13. K8s & Docker Best Practices (from Production Comparisons) ## 13. Kubernetes Roadmap Reference
> Derived from comparing three production K8s deployments: a Go-based Call Controller (Paladin), a Python/FastAPI streaming agent platform (NetBond), and a Python/FastAPI voice agent (Welcome Agent). These patterns should be adopted when ByteLyst moves from Docker Compose → K3s → managed K8s. Kubernetes planning has been split into a standalone roadmap:
### 13.1 Deployment — Zero-Downtime Rolling Updates - `docs/devops/KUBERNETES_ROADMAP.md`
**Do this (NetBond pattern):** Use that document for:
```yaml - Docker Compose → Docker Desktop K8s / K3s transition planning
spec: - K8s best practices and production comparison takeaways
strategy: - Helm values layering and namespace strategy
type: RollingUpdate - secrets progression
rollingUpdate: - CI/CD guidance for chart/image promotion
maxUnavailable: 0 # Never kill a pod before its replacement is ready - local K8s deployment workflow shape
maxSurge: 1 # Only 1 extra pod during rollout
template:
spec:
terminationGracePeriodSeconds: 45 # Match your app's drain timeout
containers:
- lifecycle:
preStop:
exec:
command: ['sleep', '5'] # Let load balancer deregister before SIGTERM
```
**Don't do this (Paladin anti-pattern):** `SINGLE_VM_DEPLOYMENT.md` remains the source of truth for:
```yaml - single-VM deployment scope
maxUnavailable: 50% # Half your pods die instantly — users get errors - Docker Compose ecosystem architecture
maxSurge: 50% # Wastes resources by doubling pod count - Dockerization/package-manager deployment guidance
``` - current implementation status and audit findings
**ByteLyst action:** Every deployment template should use `maxUnavailable: 0` + preStop sleep + explicit `terminationGracePeriodSeconds` matching the Fastify graceful shutdown timeout.
### 13.2 Pod Security Context
**Always set (NetBond pattern):**
```yaml
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
```
If the app needs writable paths (e.g., `/tmp`, cache dirs), use `emptyDir` volumes:
```yaml
volumes:
- name: tmp
emptyDir: {}
- name: cache
emptyDir: {}
volumeMounts:
- name: tmp
mountPath: /tmp
- name: cache
mountPath: /home/node/.cache
```
**ByteLyst action:** All Fastify backends are stateless — `readOnlyRootFilesystem: true` works. Next.js standalone servers may need `/tmp` writable.
### 13.3 Health Probes — Dedicated Endpoints
**Do this:**
```yaml
livenessProbe:
httpGet:
path: /health # Dedicated lightweight endpoint
port: 4003
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5 # Fast fail — 5s max
readinessProbe:
httpGet:
path: /health
port: 4003
initialDelaySeconds: 5
periodSeconds: 5
timeoutSeconds: 5
```
**Don't do this (Welcome Agent anti-pattern):**
```yaml
livenessProbe:
httpGet:
path: /openapi.json # Heavy endpoint, not a health check
timeoutSeconds: 60 # Masks real failures for a full minute
```
**ByteLyst action:** All backends already expose `GET /health` → `{ status: "ok" }`. Use it. Set timeout to 5s.
### 13.4 Ingress — WebSocket Support
If any service uses WebSocket or SSE (FlowMonk SSE, LocalMemGPT streaming, future real-time features):
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/proxy-read-timeout: '1800'
nginx.ingress.kubernetes.io/proxy-send-timeout: '1800'
nginx.ingress.kubernetes.io/proxy-buffering: 'off'
nginx.ingress.kubernetes.io/proxy-http-version: '1.1'
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
```
Missing WebSocket headers is a silent failure — connections drop after 60s with no error.
### 13.5 HPA — Use `autoscaling/v2`
**Do this:**
```yaml
apiVersion: autoscaling/v2 # Current API, supports multiple metrics
```
**Don't do this:**
```yaml
apiVersion: autoscaling/v1 # Deprecated, CPU-only, will be removed
```
### 13.6 Dockerfile Best Practices
| Practice | Do | Don't |
| ------------------- | ---------------------------------------------------------- | ---------------------------------------------------------------------------------- |
| **ENTRYPOINT form** | `ENTRYPOINT ["node", "dist/server.js"]` (exec form) | `ENTRYPOINT node dist/server.js` (shell form — PID 1 is `/bin/sh`, signals broken) |
| **COPY scope** | `COPY package.json ./` then `COPY src/ ./src/` (selective) | `COPY . .` (copies node_modules, .git, tests, everything) |
| **Layer count** | Combine related `RUN` steps | 3 separate `RUN pip install` / `RUN npm install` steps |
| **Non-root** | `USER node` (Node.js images have a `node` user) | Running as root in production |
| **Local variant** | Provide `local.Dockerfile` without corp proxy/JFrog deps | Single Dockerfile that only works behind corporate proxy |
| **Build args** | `ARG NODE_ENV=production` for conditional behavior | Hardcoded env in Dockerfile |
### 13.7 Helm Values Layering
Use 3 layers for environment management:
```
values.yaml # Base defaults (image, port, probes, resources)
├── env/local.yaml # Local K3s overrides (lower resources, NodePort, no TLS)
├── env/dev.yaml # Dev cluster overrides (replicas, hostnames, secrets)
└── env/prod.yaml # Prod overrides (more replicas, real TLS, HPA limits)
```
Deploy with layered `-f` flags:
```bash
# Local
helm upgrade --install myapp ./charts -f charts/values.yaml -f charts/env/local.yaml
# Dev
helm upgrade --install myapp ./charts -f charts/values.yaml -f charts/env/dev.yaml
# Prod
helm upgrade --install myapp ./charts -f charts/values.yaml -f charts/env/prod.yaml
```
### 13.8 Namespace Strategy
Use Helm `_helpers.tpl` for namespace — never hardcode:
```yaml
# ✅ Standard pattern — respects --namespace flag
{{ include "myapp.namespace" . }}
# ❌ Anti-pattern — ignores helm --namespace, causes confusion
{{ .Values.namespace }}
```
### 13.9 Secrets Management Progression
| Phase | Strategy | Complexity |
| ------------------------ | ----------------------------------------------------- | ---------- |
| **Phase 1** (Compose) | `.env.ecosystem` file (gitignored) | Trivial |
| **Phase 2** (K3s) | Native K8s `Secret` objects + `kubectl create secret` | Low |
| **Phase 3** (Production) | Azure Key Vault via `SecretProviderClass` CSI driver | Medium |
| **Phase 4** (Enterprise) | AKV + `AzureKeyVaultSecret` CRD with auto-sync | High |
ByteLyst already uses AKV in production (platform-service) — the CSI driver pattern is the natural next step.
### 13.10 CI/CD Best Practices (Lessons from Production Pipelines)
| Practice | Description |
| ---------------------- | ------------------------------------------------------------------------------------------------------------ |
| **Semantic release** | Auto-version from commit messages (`feat:` → minor, `fix:` → patch). ByteLyst already uses this convention. |
| **Image promotion** | Build once → push to staging repo → promote to gold/prod repo (never rebuild for prod). |
| **Branch pipelines** | Different CI stages per branch: feature (lint+test), develop (build+deploy-dev), main (promote+deploy-prod). |
| **Security gates** | SAST + SCA scans on every build. Block merges on critical findings. |
| **Quality gates** | Unit tests + coverage + SonarQube. Fail pipeline if coverage drops. |
| **Auto-deploy to dev** | Pipeline trigger: when build completes → auto-deploy to dev. Manual gate for prod. |
| **Chart versioning** | Publish Helm chart to OCI registry (ACR) with semantic version. Pull by version during deploy. |
### 13.11 Local K8s Development Script Template
A good local K8s deploy script should handle both Docker Desktop K8s (kind) and K3s:
```bash
#!/usr/bin/env bash
# deploy-local-k8s.sh — Full local K8s deployment for ByteLyst ecosystem
# Works with both Docker Desktop Kubernetes and K3s.
set -euo pipefail
NAMESPACE="bytelyst"
ACTION="${1:-deploy}" # deploy | teardown
# Detect K8s runtime
detect_runtime() {
local ctx
ctx=$(kubectl config current-context 2>/dev/null || echo "")
if [[ "$ctx" == "docker-desktop" ]]; then
echo "docker-desktop" # kind cluster inside Docker Desktop
elif command -v k3s &>/dev/null; then
echo "k3s"
else
echo "unknown"
fi
}
case "$ACTION" in
deploy)
RUNTIME=$(detect_runtime)
echo "Detected K8s runtime: $RUNTIME"
# 1. Build all Docker images
echo "Building images..."
for svc in platform-service extraction-service mcp-server; do
docker build -t bytelyst/$svc:local ./learning_ai_common_plat/services/$svc
done
# 2. Load images into K8s runtime
if [[ "$RUNTIME" == "docker-desktop" ]]; then
echo "Docker Desktop: images are already available to K8s (shared daemon)."
elif [[ "$RUNTIME" == "k3s" ]]; then
echo "K3s: importing images into containerd..."
for img in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep bytelyst); do
sudo k3s ctr images import <(docker save "$img")
done
else
echo "WARNING: Unknown K8s runtime. You may need to load images manually."
fi
# 3. Create namespace + secrets
kubectl create namespace "$NAMESPACE" --dry-run=client -o yaml | kubectl apply -f -
kubectl create secret generic bytelyst-secrets \
--from-env-file=.env.ecosystem \
-n "$NAMESPACE" --dry-run=client -o yaml | kubectl apply -f -
# 4. Deploy via Helm with local overlay
helm upgrade --install bytelyst ./helm/bytelyst-ecosystem \
-f helm/bytelyst-ecosystem/values.yaml \
-f helm/bytelyst-ecosystem/env/local.yaml \
-n "$NAMESPACE"
# 5. Wait + verify
kubectl rollout status deploy -n "$NAMESPACE" --timeout=120s
echo ""
echo "All pods:"
kubectl get pods -n "$NAMESPACE"
echo ""
if [[ "$RUNTIME" == "docker-desktop" ]]; then
echo "View in Docker Desktop: Kubernetes tab → namespace: $NAMESPACE"
fi
echo "Port-forward: kubectl port-forward svc/platform-service 4003:4003 -n $NAMESPACE"
;;
teardown)
helm uninstall bytelyst -n "$NAMESPACE" 2>/dev/null || true
kubectl delete namespace "$NAMESPACE" 2>/dev/null || true
echo "Teardown complete."
;;
esac
```
### 13.12 Quick Reference — What to Apply at Each Phase
| Best Practice | Phase 1 (Compose) | Phase 2 (K3s) | Phase 3 (Prod K8s) |
| ---------------------------- | ------------------------ | ------------------ | ------------------ |
| Zero-downtime rolling update | N/A | ✅ Apply | ✅ Apply |
| Pod security context | N/A | ✅ Apply | ✅ Apply |
| Health probes | N/A (use `healthcheck:`) | ✅ Apply | ✅ Apply |
| WebSocket ingress headers | N/A | ✅ If using SSE/WS | ✅ Apply |
| HPA v2 | N/A | Optional | ✅ Apply |
| Exec-form ENTRYPOINT | ✅ Apply now | ✅ | ✅ |
| Selective COPY | ✅ Apply now | ✅ | ✅ |
| Non-root user | ✅ Apply now | ✅ | ✅ |
| Values layering | N/A | ✅ Apply | ✅ Apply |
| Secrets via AKV CSI | N/A | N/A | ✅ Apply |
| Semantic release | ✅ Apply now | ✅ | ✅ |
| Image promotion | N/A | N/A | ✅ Apply |
| Local deploy script | N/A | ✅ Apply | ✅ Adapt |
--- ---
@ -1483,9 +1422,11 @@ esac
| Question | Answer | | Question | Answer |
| ------------------------------ | -------------------------------------------------------------------------------------------------------------- | | ------------------------------ | -------------------------------------------------------------------------------------------------------------- |
| **Can deploy on single VM?** | **Yes.** All ~25 services fit in 32 GB RAM. | | **Can deploy on single VM?** | **Yes.** All ~27 services fit in 32 GB RAM. |
| **All Dockerized?** | **Yes.** All 10 product repos now have Dockerfiles + docker-prep.sh. | | **All Dockerized?** | **Yes.** All 10 product repos now have Dockerfiles + docker-prep.sh. |
| **Package-manager direction?** | **`pnpm` is the long-term standard** for Node/TS repos, but migration is phased repo-by-repo, not big-bang. | | **Package-manager direction?** | **`pnpm` is the long-term standard** for Node/TS repos, but migration is phased repo-by-repo, not big-bang. |
| **Self-hosted CI?** | **Yes.** Gitea + act_runner (~250 MB combined). GitHub Actions–compatible workflows. |
| **Package registry?** | **Gitea npm registry** replaces `docker-prep.sh` + `.tgz` tarballs. Also has Docker container registry. |
| **K8s practice on single VM?** | **Docker Desktop K8s** (Mac/Windows) or **K3s** (Linux). Same manifests scale to AKS/EKS/GKE. | | **K8s practice on single VM?** | **Docker Desktop K8s** (Mac/Windows) or **K3s** (Linux). Same manifests scale to AKS/EKS/GKE. |
| **Recommended VM?** | 8 vCPU / 32 GB (min) or 16 vCPU / 64 GB (with Ollama). Hetzner ~$45/mo for dev. | | **Recommended VM?** | 8 vCPU / 32 GB (min) or 16 vCPU / 64 GB (with Ollama). Hetzner ~$45/mo for dev. |
| **Time to production K8s?** | Phase 1 (compose) → Phase 2 (Docker Desktop / K3s) → Phase 3 (multi-node) → Phase 4 (managed). Same manifests. | | **Time to production K8s?** | Phase 1 (compose) → Phase 2 (Docker Desktop / K3s) → Phase 3 (multi-node) → Phase 4 (managed). Same manifests. |

77
scripts/publish-local-gitea-packages.sh Normal file
View File

@ -0,0 +1,77 @@
#!/usr/bin/env bash
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
PACKAGES_DIR="$REPO_ROOT/packages"
TMP_DIR="${TMPDIR:-/tmp}/bytelyst-gitea-publish"
REGISTRY_URL="${GITEA_NPM_REGISTRY_URL:-http://localhost:3300/api/packages/bytelyst/npm/}"
TOKEN="${GITEA_NPM_TOKEN:-}"
PACKAGE_FILTER="${1:-}"
if [ -z "$TOKEN" ]; then
echo "❌ GITEA_NPM_TOKEN is required"
exit 1
fi
rm -rf "$TMP_DIR"
mkdir -p "$TMP_DIR"
publish_package() {
local pkg_dir="$1"
local package_name
local package_version
package_name="$(node -e "const fs=require('fs'); const pkg=JSON.parse(fs.readFileSync(process.argv[1], 'utf8')); process.stdout.write(pkg.name);" "$pkg_dir/package.json")"
package_version="$(node -e "const fs=require('fs'); const pkg=JSON.parse(fs.readFileSync(process.argv[1], 'utf8')); process.stdout.write(pkg.version);" "$pkg_dir/package.json")"
local safe_name
safe_name="${package_name//@/}"
safe_name="${safe_name//\//-}"
local work_dir="$TMP_DIR/$safe_name-$package_version"
local packed_tgz
local final_tgz
rm -rf "$work_dir"
mkdir -p "$work_dir"
echo "📦 Packing $package_name@$package_version"
(
cd "$pkg_dir"
pnpm pack --pack-destination "$work_dir" >/dev/null
)
packed_tgz="$(find "$work_dir" -maxdepth 1 -name '*.tgz' | head -1)"
if [ -z "$packed_tgz" ]; then
echo "❌ Failed to pack $package_name@$package_version"
exit 1
fi
mkdir -p "$work_dir/unpacked"
tar -xzf "$packed_tgz" -C "$work_dir/unpacked"
(
cd "$work_dir/unpacked/package"
npm pack --pack-destination "$work_dir" >/dev/null
)
final_tgz="$(find "$work_dir" -maxdepth 1 -name '*.tgz' | sort | tail -1)"
if [ -z "$final_tgz" ]; then
echo "❌ Failed to repack $package_name@$package_version"
exit 1
fi
echo "🚀 Publishing $package_name@$package_version to $REGISTRY_URL"
if ! npm publish "$final_tgz" \
--registry "$REGISTRY_URL" \
--//localhost:3300/api/packages/bytelyst/npm/:_authToken="$TOKEN"; then
echo "⚠️ Publish failed for $package_name@$package_version (possibly already published)"
fi
}
while IFS= read -r -d '' pkg_json; do
pkg_dir="$(dirname "$pkg_json")"
pkg_name="$(node -e "const fs=require('fs'); const pkg=JSON.parse(fs.readFileSync(process.argv[1], 'utf8')); process.stdout.write(pkg.name);" "$pkg_json")"
if [ -n "$PACKAGE_FILTER" ] && [ "$pkg_name" != "$PACKAGE_FILTER" ]; then
continue
fi
publish_package "$pkg_dir"
done < <(find "$PACKAGES_DIR" -mindepth 2 -maxdepth 2 -name package.json -print0 | sort -z)
echo "✅ Local Gitea package publish complete"