# Gitea npm Package Registry Migration

> **Goal:** Replace the `docker-prep.sh` + tarball workflow with Gitea's built-in npm package registry for `@bytelyst/*` shared packages.
>
> **Phase policy:** use **this Mac as the future single VM** and validate the entire flow locally first. No Azure rollout and no publishing to any external or corporate registry in this phase.

---

## 1. Decision

We should adopt the local Gitea npm registry as the long-term package-distribution path for `@bytelyst/*` packages.

We should **not** switch directly in Azure first.

Instead, we should validate the exact end-to-end flow locally on this Mac:

- local Gitea git hosting
- local Gitea Actions
- local Gitea npm package registry
- local Docker / Docker Compose builds
- local product repos consuming `@bytelyst/*` from local Gitea

Once that is stable, we replicate the same validated shape on the Azure single VM.

---

## 2. Why We Want This

### Previous state (resolved)

Before this migration, repos depended on Docker-time workarounds:

- `file:` references to sibling `learning_ai_common_plat/packages/*`
- `docker-prep.sh` + `.docker-deps` or `.tarballs` copy steps
- temporary `package.json` rewriting before builds

These are now eliminated. All repos use the Gitea npm registry.

### Target outcome

We want one dependency source for shared packages:

- local development
- local CI via Gitea Actions
- local Docker builds
- later Azure single-VM builds

That dependency source should be the **local Gitea npm registry**.

---

## 3. Local-First Target Architecture

### 3.1 This Mac as the rehearsal VM

```text
This Mac
├── local Gitea (git + actions + npm registry)
├── local Docker / Docker Compose
├── learning_ai_common_plat
└── product repos
```

Flow:

```text
learning_ai_common_plat/packages/*
  ↓ build locally
  ↓ publish to local Gitea npm registry
local Gitea npm registry (${GITEA_NPM_HOST}:3300)
  ↓ install via semver refs
product repos / Docker builds / local Gitea CI
```

### 3.2 Future Azure single VM

After the local rehearsal is proven, Azure should mirror the same shape:

```text
Azure VM
├── Gitea
├── Gitea Actions runner
├── Gitea npm registry
├── Docker / Compose
└── same repo + build flow
```

The Azure plan should reuse the validated local approach, not invent a second design.

---

## 4. What Is In Scope Right Now

### Included

- validate local Gitea package-registry access
- validate local Gitea Actions availability
- define local-only package publish flow to local Gitea
- define local-only consumer install flow from local Gitea
- define local-only Docker build flow from local Gitea
- define the later Azure single-VM replication steps

### Excluded

- Azure changes right now
- external npm registry publishing
- corporate registry publishing
- non-local deployment actions
- K3s / Helm work

---

## 5. Current Local Readiness Snapshot

Validated locally on this Mac:

- local Gitea health endpoint responds
- local Gitea version endpoint responds
- local `bytelyst` user auth works
- local Gitea Actions workflow exists for `learning_ai_common_plat`
- local Gitea package registry endpoint is reachable
- local pilot packages have been published to local Gitea successfully
- a clean scratch consumer install from local Gitea works on the host

That is enough to treat this Mac as the single-VM rehearsal environment.

---

## 5.1 Validation Results So Far

### Proven locally on this Mac

- shared packages build cleanly in `learning_ai_common_plat`
- a local-only Gitea package token was created and used successfully for local package publishing
- **56 `@bytelyst/*` packages** published to local Gitea npm registry
- a clean scratch `pnpm install` from the local Gitea registry works on all migrated repos
- **all 10 product repos migrated** from `file:` refs + `docker-prep.sh` to semver `^0.1.0` + registry-backed Dockerfiles
- all migrated repos pass host-side `pnpm install` and backend typecheck
- Docker builds (backend + web) verified for all repos with Dockerfiles
- local Gitea CI green for 10/10 repos with CI workflows (FlowMonk, NoteLett, ActionTrail, LocalMemGPT, NomGap, ChronoMind, JarvisJr, PeakPulse, MindLyst, LysnrAI)
- `docker-prep.sh` removed from all 10 repos
- `GITEA_NPM_TOKEN` added to act_runner config for CI registry access
- stale "Build @bytelyst/\* packages" step removed from all CI workflows

### Important implementation finding

Publishing `@bytelyst/*` workspace packages with raw `npm publish` is not sufficient for this migration.

Why:

- packages with internal `workspace:*` dependencies can be published with unresolved workspace specifiers
- those published artifacts break downstream consumers

What worked:

- `pnpm pack` produces normalized package metadata for workspace dependencies
- publishing the normalized tarball to Gitea works

This is now codified in the local-only helper script:

`scripts/publish-local-gitea-packages.sh`

That script is currently the authoritative local rehearsal path for publishing `@bytelyst/*` packages to local Gitea.

### No remaining blockers

Host-side installs, Docker builds, and local CI are all validated.

### Network topology determines the Docker recipe

| Topology            | `NETWORK=` | Gitea host                                       | Docker reaches Gitea via                                     | `--add-host` needed?                  |
| ------------------- | ---------- | ------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------- |
| **Local dev (Mac)** | `corp`     | `localhost:3300` (GITEA_NPM_HOST=localhost)      | `host.docker.internal` + `--add-host localhost:host-gateway` | **Yes** — expected                    |
| **Local dev (Mac)** | `home`     | `<vm-host>:3300` (from `~/.gitea_vm_host`)       | Direct internet (no proxy)                                   | **No** — direct connection            |
| **VM deployment**   | N/A        | `localhost:3300` or compose service `gitea:3300` | Native Docker networking                                     | **No** — `ROOT_URL` resolves natively |

The `--add-host localhost:host-gateway` workaround is the **correct local-dev pattern** when `NETWORK=corp`, not a generic blocker. On a VM, Gitea's `ROOT_URL` will be set to the real hostname or compose service name, so tarball URLs resolve without any host-mapping workaround.

At the time of writing (2026-03-24):

- host-side registry usage is validated across all 10 migrated repos
- Docker builds verified for all repos with Dockerfiles
- local Gitea CI green for all 10 repos with CI workflows
- 56 `@bytelyst/*` packages published to local Gitea
- `docker-prep.sh` retired from all repos

Verified local FlowMonk Docker recipe (`NETWORK=corp`):

```bash
export TOKEN='<local-gitea-package-token>'

docker build \
  --add-host localhost:host-gateway \
  --build-arg GITEA_NPM_HOST=host.docker.internal \
  --secret id=gitea_npm_token,env=TOKEN \
  -f backend/Dockerfile /Users/sd9235/code/mygh/learning_ai_flowmonk

docker build \
  --add-host localhost:host-gateway \
  --build-arg GITEA_NPM_HOST=host.docker.internal \
  --secret id=gitea_npm_token,env=TOKEN \
  -f web/Dockerfile /Users/sd9235/code/mygh/learning_ai_flowmonk
```

On the target VM, the recipe simplifies to just `--build-arg GITEA_NPM_HOST=<gitea-host>` plus the BuildKit secret — no `--add-host` needed.

---

## 5.2 Remaining Gaps From Local Mac Validation

The local rehearsal on this Mac has validated the **host-side** registry model, **Docker builds** (backend + web), and **local Gitea CI** across all 10 product repos.

### Validated ✅

- Host-side `pnpm install` against local Gitea registry (all 10 repos)
- Backend + web Docker builds with BuildKit secret + `--add-host` (all repos with Dockerfiles)
- Local Gitea CI green for all 10 repos with CI workflows
- Runner re-registered against `127.0.0.1` (avoids IPv6 `[::1]` declaration failure)
- `GITEA_NPM_TOKEN` added to act_runner config for CI registry access
- 56 `@bytelyst/*` packages published and consumed successfully
- Backend typecheck passes for all 10 migrated repos
- `docker-prep.sh` removed from all 10 repos

### Gaps closed

- ~~2 repos not yet migrated~~ → MindLyst and LysnrAI migrated (2026-03-24)
- ~~Gitea CI queue stuck~~ → Runner restarted with `GITEA_NPM_TOKEN`, all 8 CI repos green
- ~~Stale common-plat build step in CI~~ → Removed from all 7 CI workflows

### Remaining minor items

- Mobile workspace members remain on `file:` refs where present (NomGap, NoteLett) — acceptable for now
- `@bytelyst/create-app` is marked `private: true` and cannot be published (scaffolding CLI tool, not consumed by product repos)

### Why this matters for Azure

Azure should be a replication step, not a redesign step.

On the VM, Gitea's `ROOT_URL` will point to the real hostname, so Docker containers will resolve tarball URLs natively — the `--add-host` workaround is local-dev only. The Azure VM rollout should wait until the remaining local gaps are cleared for:

- package metadata correctness
- Docker consumer correctness across all product repos (not just FlowMonk pilot)
- local CI correctness for the full build → publish → consume loop

If we skip those validations locally, the first Azure VM trial becomes a debugging environment instead of a deployment environment.

### Concrete local-exit criteria before Azure

Before starting Azure VM rollout, we should be able to demonstrate all of the following on this Mac:

- one Gitea deployment shape whose package URLs work for both host installs and Docker builds
- one publish path for `@bytelyst/*` packages that works repeatably with `pnpm pack`
- one pilot repo that installs from the registry on the host and inside Docker without fallback tarballs
- one local Gitea Actions path that can build/publish/install with the same registry assumptions
- one documented rollback path that cleanly returns a pilot repo to tarball-based Docker consumption if needed

---

## 6. Migration Strategy

### Stage A — Local registry rehearsal

Validate the package-registry path locally before changing every repo.

Required outcomes:

1. local package token exists
2. a small pilot set of `@bytelyst/*` packages can be published to local Gitea
3. a local consumer can install them via semver refs
4. a Docker build can install them without `docker-prep.sh`
5. local Gitea Actions can run the same package build/publish path

### Stage B — Pilot repo migration

Migrate one repo first.

Recommended pilot:

- `learning_ai_flowmonk`

Reason:

- recent Docker/build work already exposed its edge cases
- both backend and web surfaces exist
- it is a strong canary for dependency-distribution changes

### Stage C — Sequential ecosystem rollout

After the pilot is stable, migrate remaining repos sequentially.

---

## 7. Local Implementation Plan

### 7.1 Verify package metadata in `learning_ai_common_plat`

Every `packages/*/package.json` should have:

- a valid `name`
- a valid `version`
- a valid `files` / `exports` setup
- a successful local `build`

Local validation command:

```bash
cd /Users/sd9235/code/mygh/learning_ai_common_plat
pnpm -r --filter './packages/*' build
```

### 7.2 Create a local-only Gitea package token

Use a token intended only for this Mac rehearsal.

Required scopes:

- package read
- package write

Do not use Azure or external registry credentials here.

### 7.3 Publish only to local Gitea

In this phase, all `npm publish` activity must point only to:

`http://${GITEA_NPM_HOST}:3300/api/packages/bytelyst/npm/`

(`GITEA_NPM_HOST` is set by `switch-network.sh` — `localhost` on corp, Azure VM host on home.)

No Azure Artifacts.
No npmjs.org.
No external corporate npm registry.

Use the local-only helper script for this rehearsal:

```bash
cd /Users/sd9235/code/mygh/learning_ai_common_plat
GITEA_NPM_TOKEN='<local-token>' bash ./scripts/publish-local-gitea-packages.sh
```

For a single package:

```bash
cd /Users/sd9235/code/mygh/learning_ai_common_plat
GITEA_NPM_TOKEN='<local-token>' bash ./scripts/publish-local-gitea-packages.sh '@bytelyst/errors'
```

### 7.4 Start with a minimal package pilot set

Publish a small set first:

- `@bytelyst/errors`
- `@bytelyst/config`
- `@bytelyst/api-client`

Then expand once the local install path is proven.

### 7.5 Consumer dependency model after migration

Consumers move from local file refs:

```json
"@bytelyst/auth": "file:../../learning_ai_common_plat/packages/auth"
```

to semver refs:

```json
"@bytelyst/auth": "^0.1.0"
```

And resolve via scoped `.npmrc` or `.pnpmrc` config pointing `@bytelyst` to local Gitea.

### 7.6 Docker model after migration

After a repo is migrated to the registry model, its Dockerfiles should:

- stop copying `.docker-deps` or `.tarballs`
- stop depending on `docker-prep.sh`
- install from local Gitea through scoped registry config

That is the main operational simplification we want.

---

## 8. Local Validation Sequence

### Step 1 — Validate local Gitea surfaces

Confirm locally:

- Gitea UI is reachable
- package registry API is reachable with auth
- Actions workflow is visible and active

### Step 2 — Validate package build output

Run local builds for the shared packages and confirm pack/publish inputs are clean.

### Step 3 — Publish a minimal package pilot set to local Gitea only

Success means:

- packages appear in local Gitea package list
- metadata looks correct
- no external registry interaction occurred

### Step 4 — Validate local install from Gitea

Use a scratch consumer or a pilot repo surface and verify:

- `pnpm install` resolves `@bytelyst/*` from local Gitea
- lockfile updates are clean
- no fallback to tarballs is required for the tested packages

### Step 5 — Validate Docker build from local Gitea

For the pilot repo:

- remove the tested surface's dependency on `docker-prep.sh`
- add scoped registry config for Docker
- build backend and web without tarball prep

Success means:

- image build works without `.docker-deps` or `.tarballs`
- build does not require local package rewrite logic

Current status on this Mac:

- **fully green** — host-side installs, Docker builds, and local CI all validated

### Step 6 — Validate local Gitea Actions path

Add or adapt a local-only workflow in `learning_ai_common_plat` that:

- builds packages
- optionally publishes to local Gitea npm registry
- never targets an external registry

### Step 7 — Validate full local E2E

At the end of the local rehearsal we should be able to say:

- shared packages build locally
- shared packages publish to local Gitea registry
- a pilot consumer installs from local Gitea
- a pilot Docker build succeeds from local Gitea
- local Gitea Actions can drive the same path

Only then should we expand repo-by-repo.

---

## 9. Recommended Rollout Order

### All 10 repos migrated ✅

1. ✅ `learning_ai_flowmonk` — pilot repo, CI green
2. ✅ `learning_ai_notes` — CI green
3. ✅ `learning_ai_trails` — CI green
4. ✅ `learning_ai_local_memory_gpt` — CI green
5. ✅ `learning_ai_fastgap` — CI green (fixed missing `@expo/vector-icons`)
6. ✅ `learning_ai_clock` — CI green
7. ✅ `learning_ai_jarvis_jr` — CI green
8. ✅ `learning_ai_peakpulse` — CI green (backend only, no web Dockerfile)
9. ✅ `learning_multimodal_memory_agents` — non-standard layout (KMP + `mindlyst-native/web/`), typecheck pass
10. ✅ `learning_voice_ai_agent` — non-standard layout (Python desktop + `user-dashboard-web/`), typecheck pass

---

## 10. CI Model For The Local Rehearsal

For the local rehearsal phase, Gitea Actions should do only local work:

- build packages
- test packages
- typecheck packages
- optionally publish to **local Gitea only**

This is intentionally different from a final enterprise rollout.

The objective is to prove the local VM pattern first.

---

## 11. Risks And Guardrails

### Risks

- version drift between local source and published package versions
- peer dependency mismatches hidden by current tarball workflow
- Docker auth/config differences from shell installs
- accidental publishing to the wrong registry
- switching too many repos before the pilot is stable

### Guardrails

- use local-only credentials for this phase
- keep `@bytelyst` scoped registry config explicit
- keep rollback steps documented per repo (§12)

---

## 12. Rollback Plan

The migration is complete for all 10 repos. If a repo needs to revert to the old `file:` + `docker-prep.sh` pattern:

### Per-repo rollback steps

1. `git revert <migration-commit>` — reverts `package.json`, `Dockerfile`, `.npmrc`, `pnpm-workspace.yaml`
2. Restore `scripts/docker-prep.sh` from the commit just before `chore: remove docker-prep.sh`
3. Re-add the sibling workspace reference in `pnpm-workspace.yaml`:
   ```yaml
   packages:
     - ../learning_ai_common_plat/packages/*
   ```
4. Restore `file:` refs in `package.json` — sed: `sed -i '' 's|"\^0.1.0"|"file:../../learning_ai_common_plat/packages/PKG_NAME"|g'`
5. Run `pnpm install` to regenerate the lockfile
6. Verify `pnpm run typecheck` and `pnpm run test` pass
7. Commit and push

### Registry-side rollback

No registry-side action is needed. The Gitea packages remain published and don't interfere with `file:` refs. The `.npmrc` scoped registry config only activates when the `@bytelyst` scope resolves through the registry.

### When rollback is NOT needed

- If a package is missing from the registry → publish it with `scripts/publish-local-gitea-packages.sh`
- If a transitive dep is missing → add it explicitly to `package.json` (this surfaced for `@expo/vector-icons` in NomGap)

---

## 13. Azure Single-VM Follow-Through

After the local rehearsal is green, Azure should follow the same validated recipe:

1. provision one VM
2. install Docker and Gitea
3. enable Gitea Actions runner
4. enable Gitea packages
5. clone the same repos onto the VM
6. apply the same local registry/token/package flow
7. re-run the pilot repo first
8. then expand sequentially across the ecosystem

Azure should be a replication step, not a redesign step.

### What Azure still needs beyond current local proof

Even though the codebase is already designed to be highly configurable, Azure VM rollout still requires a few validations that have not been fully cleared by the local Mac rehearsal yet:

- Gitea running in a deployment shape where package tarball URLs are stable for both host consumers and containerized consumers
- registry-backed Docker builds for the pilot repo without `docker-prep.sh`
- local Gitea Actions or equivalent host-runner proof for package build/publish/install
- a final decision on whether mobile-facing `@bytelyst/*` packages stay on local `file:` links during the pilot phase or join the registry migration in a later wave
- one Azure-ready secrets model for Gitea token handling, service envs, and registry auth that maps cleanly from local env vars to VM secrets/config files

### What should require minimal change when moving to Azure

The good news is that most of the ecosystem has already been implemented in a way that keeps scale-up mostly configuration-driven once the registry and image flows are proven.

Expected low-change areas:

- service code already externalizes most environment-specific values via env/config
- Compose and K8s models already map cleanly to Deployments, Services, Ingress, ConfigMaps, and Secrets
- Traefik, readiness probes, service ports, and namespace separation are already documented in a K8s-friendly way
- scaling stateless services should mostly mean changing replica counts, resource requests/limits, and HPA settings
- moving from single-node K3s to multi-node K3s or managed Kubernetes should mostly reuse the same manifests with infra-level adjustments

All package-distribution concerns are resolved:

- All 10 repos consume `@bytelyst/*` from Gitea registry
- Docker builds verified for all repos with Dockerfiles
- `docker-prep.sh` removed from all repos

---

## 14. Definition Of Done

This migration plan is locally validated only when all are true:

- [x] local Gitea package publish auth verified
- [x] local package publish path verified — 56 packages published
- [x] local consumer install path verified on the host — all 10 repos
- [x] Docker build path verified without `docker-prep.sh` — all repos with Dockerfiles
- [x] local Gitea CI verified — 10/10 repos with CI workflows green
- [x] pilot repo migrated successfully end-to-end including Docker (FlowMonk)
- [x] 7 standard repos migrated and verified (NoteLett, ActionTrail, LocalMemGPT, NomGap, ChronoMind, JarvisJr, PeakPulse)
- [x] remaining 2 non-standard repos migrated (MindLyst, LysnrAI)
- [x] rollback path documented (§12)
- [x] Azure single-VM reproduction steps documented (§13)

---

## 15. Migration Complete — Summary

1. ~~create or verify a local-only Gitea package token~~ ✅
2. ~~publish `@bytelyst/*` packages to local Gitea~~ ✅ (56 packages)
3. ~~migrate pilot repo (FlowMonk) end-to-end~~ ✅
4. ~~expand to remaining standard-layout repos~~ ✅ (7 repos)
5. ~~confirm Gitea CI green for all repos~~ ✅ (10/10 with CI workflows)
6. ~~migrate remaining 2 non-standard repos (MindLyst, LysnrAI)~~ ✅
7. ~~document rollback path~~ ✅ (§12)
8. ~~document Azure single-VM reproduction steps~~ ✅ (§13)
9. ~~remove `docker-prep.sh` from all 10 repos~~ ✅
10. ~~add `GITEA_NPM_TOKEN` to act_runner config~~ ✅
11. ~~remove stale common-plat build step from CI workflows~~ ✅

**Migration completed 2026-03-24. All 10 product repos now consume `@bytelyst/*` packages from the local Gitea npm registry.**

---

## 15.1 Package Inventory (verified 2026-03-26)

58 package directories exist in `packages/`. Of those:

- **2 are native SDKs** (not npm): `kotlin-platform-sdk`, `swift-platform-sdk`
- **56 are published** to Gitea npm registry (55 at `0.1.0`, `react-auth` also at `0.1.1`)
- **1 is private** (`@bytelyst/create-app`) — scaffolding CLI tool, not publishable

### Not published (1 package)

```
@bytelyst/create-app             # private: true in package.json — scaffolding CLI, not consumed by product repos
```

---

## 16. Post-Migration Audit (2026-03-24)

After the initial migration was marked complete, a systematic audit was run across all 10 repos. 5 bugs/gaps were found and fixed:

| #   | Finding                                                                                      | Severity | Fix                                                                        | Repo                                |
| --- | -------------------------------------------------------------------------------------------- | -------- | -------------------------------------------------------------------------- | ----------------------------------- |
| 1   | FlowMonk `mobile/package.json` had 6 stale `file:` refs                                      | **BUG**  | Converted to `^0.1.0` + regenerated lockfile                               | `learning_ai_flowmonk`              |
| 2   | MindLyst had 3 stale `package-lock.json` files (npm artifact, pnpm doesn't use them)         | **BUG**  | Deleted all 3                                                              | `learning_multimodal_memory_agents` |
| 3   | MindLyst missing `.dockerignore`                                                             | **GAP**  | Created with proper exclusions (node_modules, .next, dist, KMP build dirs) | `learning_multimodal_memory_agents` |
| 4   | MindLyst CI workflow had stale "Build @bytelyst/\* packages" step + used npm instead of pnpm | **BUG**  | Removed step, added `pnpm install`, switched to `pnpm --filter`            | `learning_multimodal_memory_agents` |
| 5   | LysnrAI CI workflow had stale "Build @bytelyst/\* packages" step                             | **BUG**  | Removed step, added `pnpm install`                                         | `learning_voice_ai_agent`           |

### Dead file sweep — confirmed clean

- `docker-prep.sh` — removed from all 10 repos ✅
- `.docker-deps/` dirs — none tracked ✅
- `*.tgz` tarballs — none tracked ✅
- `package-lock.json` — none tracked (all pnpm repos) ✅
- jfrog refs in Dockerfiles — intentional for corp proxy, not dead ✅

### Lesson learned

Mobile workspace members (FlowMonk, NomGap, NoteLett) that were previously insulated from the `file:` → registry change because their lockfiles still resolved transitively will break when the lockfile is regenerated. Always run `GITEA_NPM_TOKEN=... pnpm install --no-frozen-lockfile` after converting mobile `package.json` refs.

---

## 17. Docker Build Verification (2026-03-24)

After completing the migration and audit, all Dockerfiles were tested with actual `docker build` commands using the Gitea registry.

### Bugs found and fixed during Docker verification

| #   | Finding                                                                                               | Severity | Fix                                                   | Repo                                |
| --- | ----------------------------------------------------------------------------------------------------- | -------- | ----------------------------------------------------- | ----------------------------------- |
| 6   | MindLyst + LysnrAI Dockerfiles used shell syntax (`2>/dev/null \|\| true`) in COPY instructions       | **BUG**  | Removed — Docker COPY doesn't support shell redirects | both                                |
| 7   | MindLyst web Dockerfile used `pnpm run build -- --webpack` — pnpm passes `--webpack` as directory arg | **BUG**  | Changed to `npx next build --webpack`                 | `learning_multimodal_memory_agents` |
| 8   | `@bytelyst/extraction` was missing from Gitea registry                                                | **GAP**  | Published                                             | common-plat                         |

### Docker build results — all PASS

| Repo     | Image                    | Status |
| -------- | ------------------------ | ------ |
| MindLyst | `mindlyst-backend:test`  | ✅     |
| MindLyst | `mindlyst-web:test`      | ✅     |
| LysnrAI  | `lysnrai-backend:test`   | ✅     |
| LysnrAI  | `lysnrai-dashboard:test` | ✅     |

(Standard repos were already Docker-verified during the initial migration.)

### Full verification sweep

- **Backend tests**: 1,700+ tests passing across all 12 backend services (platform 1,483 + extraction 136 + 10 products)
- **Web typecheck**: 9/9 web apps typecheck clean
- **Backend typecheck**: 10/10 repos typecheck clean
