From 9d871282c3f615dbe6bb6955e39c333d87140786 Mon Sep 17 00:00:00 2001 From: Saravanakumar D Date: Sun, 31 May 2026 00:05:19 -0700 Subject: [PATCH] docs: explain Gitea registry vs workspace package resolution + the registry-offline trap Document the two ways @bytelyst/* packages resolve (local workspace links vs Gitea npm registry for Docker/CI), the common 'registry offline' local-dev failure and its fix (sibling directory layout, not a token), and the deploy-side 'package not published' / token issues with remediation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- docs/gitea-registry-and-package-resolution.md | 189 ++++++++++++++++++ docs/repo-map.md | 1 + 2 files changed, 190 insertions(+) create mode 100644 docs/gitea-registry-and-package-resolution.md diff --git a/docs/gitea-registry-and-package-resolution.md b/docs/gitea-registry-and-package-resolution.md new file mode 100644 index 0000000..eead286 --- /dev/null +++ b/docs/gitea-registry-and-package-resolution.md @@ -0,0 +1,189 @@ +# Gitea Registry & `@bytelyst/*` Package Resolution + +> **TL;DR** — `@bytelyst/*` packages resolve **two different ways** depending on +> context, and conflating them causes the most common onboarding failure: +> +> | Context | How `@bytelyst/*` resolves | Needs Gitea registry? | Needs `GITEA_NPM_TOKEN`? | +> |---|---|---|---| +> | **Local dev** (`pnpm install` on a laptop / dev box) | pnpm **workspace links** to the sibling `learning_ai_common_plat` checkout | **No** | **No** | +> | **Docker / CI build** (`docker build`, `deploy-*.sh`) | Pulled from the **Gitea npm registry** (`localhost:3300`) | **Yes** | **Yes** | +> +> If a local install is trying to reach `localhost:3300` or asking for a token, +> the repo isn't sitting beside `learning_ai_common_plat` — fix the layout, don't +> chase a token. See [The local-dev "registry offline" trap](#the-local-dev-registry-offline-trap). + +--- + +## 1. What is Gitea here? + +[Gitea](https://gitea.io) is the **self-hosted Git server + package registry** that +backs the ByteLyst ecosystem. It runs on the production VM and exposes a private +**npm registry** for the shared `@bytelyst/*` packages (`@bytelyst/cosmos`, +`@bytelyst/config`, `@bytelyst/ui`, `@bytelyst/telemetry`, etc.) that live in +`learning_ai_common_plat`. + +- **Public host:** `https://gitea.bytelyst.com/` +- **npm registry API (on the VM / via tunnel):** `http://localhost:3300/api/packages/bytelyst/npm/` +- **Auth:** a personal access token exported as `GITEA_NPM_TOKEN` + +The registry exists so that **Docker images and CI builds** can install +`@bytelyst/*` without needing the `learning_ai_common_plat` source tree copied +into the build context. + +## 2. The two resolution paths (this is the whole story) + +Every ByteLyst product repo (`learning_ai_invt_trdg`, `learning_ai_clock`, …) is a +pnpm workspace that depends on `@bytelyst/*`. There are exactly two ways those +deps get satisfied: + +### Path A — Local dev: workspace links (no registry, no token) + +Each product repo ships a `.pnpmfile.cjs` hook and a `pnpm-workspace.yaml` that, +**by default**, rewrite every `@bytelyst/*` dependency to `workspace:*` and point +it at the sibling `learning_ai_common_plat/packages/*` source on disk. + +``` +/ +├── learning_ai_common_plat/ # source of @bytelyst/* packages +└── learning_ai_invt_trdg/ # product repo — resolves siblings via workspace links +``` + +- `pnpm-workspace.yaml` globs `../learning_ai_common_plat/packages/*` +- `.pnpmfile.cjs` (default `BYTELYST_PACKAGE_SOURCE=common-plat`) rewrites + `@bytelyst/foo` → `workspace:*` **when it finds the package on disk** +- Both lookups are **relative to the repo's parent directory** + +Result: `pnpm install` links packages straight from source. **No `localhost:3300`, +no `GITEA_NPM_TOKEN`, no published artifacts required.** This is the correct path +for all laptop / dev-box work. + +> Override the sibling location if your checkout lives elsewhere: +> `BYTELYST_COMMON_PLAT_ROOT=/abs/path/to/learning_ai_common_plat pnpm install` + +### Path B — Docker / CI build: the Gitea registry (token required) + +Inside a Docker build there is no sibling source tree, so the build pulls +`@bytelyst/*` from the Gitea npm registry instead. Example +(`learning_ai_invt_trdg/backend/Dockerfile`): + +```dockerfile +ARG GITEA_NPM_TOKEN +RUN printf '@bytelyst:registry=http://localhost:3300/api/packages/bytelyst/npm/\n//localhost:3300/api/packages/bytelyst/npm/:_authToken=${GITEA_NPM_TOKEN}\n' > .npmrc \ + && pnpm install --ignore-scripts --lockfile=false +``` + +The deploy wrapper `deploy-invttrdg.sh` (in this repo) sets this up: + +1. **Pre-flight check** — verifies each required `@bytelyst/*` package is published + (`curl -sf "${GITEA_REGISTRY}${package}"`); fails early with remediation if not. +2. **Token resolution** — reads `GITEA_NPM_TOKEN` from `/opt/bytelyst/.gitea_token` + or `~/.gitea_npm_token`. +3. **Build** — `docker build --network host --build-arg GITEA_NPM_TOKEN=... …` + (`--network host` lets the container reach `localhost:3300` on the VM). + +## 3. The local-dev "registry offline" trap + +**Symptom** + +``` + WARN GET http://localhost:3300/api/packages/.../@bytelyst%2Fcosmos error (ECONNREFUSED) + ERR_PNPM_FETCH GITEA_NPM_TOKEN ... 401 / registry offline +``` + +…on a plain `pnpm install`, on a laptop, with no Docker involved. + +**Root cause** + +The repo was cloned **standalone** (e.g. `D:\SANDBOX\learning_ai_invt_trdg`) with +**no sibling `learning_ai_common_plat`**. `.pnpmfile.cjs` couldn't find the package +source on disk, so it left the `@bytelyst/*` specifiers untouched and pnpm fell +back to the registry in `.npmrc` — which is the **Path B (Docker/CI) registry**, +not meant for local dev. With the VM tunnel down and no token, the install dies. + +**Fix (layout, not token)** + +Place the product repo **beside** `learning_ai_common_plat` under a common parent, +then reinstall: + +```bash +# from the product repo +pnpm install # or: pnpm run install:common-plat +``` + +Or, if you can't move the checkout, point at the sibling explicitly: + +```bash +BYTELYST_COMMON_PLAT_ROOT=/abs/path/to/learning_ai_common_plat pnpm install +``` + +That's it — no token, no running registry needed for local work. + +> **How we hit & solved this (May 2026):** a Cosmos cost-optimization task required +> building/testing `learning_ai_invt_trdg`, but it had been cloned standalone under +> `D:\SANDBOX\`. `pnpm install` fell back to the offline `localhost:3300` registry +> and failed. The fix was **not** to restore the registry or mint a token — it was +> to move the repo beside `learning_ai_common_plat` (into the same workspace parent) +> so `.pnpmfile.cjs` could rewrite `@bytelyst/*` → `workspace:*`. Install then +> succeeded in seconds with zero registry involvement. The product repos' +> `AGENTS.md` / `README.md` were updated to document the required sibling layout +> and to stop implying `GITEA_NPM_TOKEN` is needed for local dev. + +## 4. The deploy-side "package not published" issue + +**Symptom** — `deploy-invttrdg.sh` aborts: + +``` +Required @bytelyst packages not published to Gitea registry: +@bytelyst/cosmos @bytelyst/config ... +``` + +**Root cause** — `learning_ai_common_plat` published new/changed packages that were +never pushed to the Gitea registry the Docker build pulls from. + +**Fix** — republish the packages to Gitea, then re-run the deploy: + +```bash +python3 /opt/bytelyst/republish_packages.py # publishes all @bytelyst/* to Gitea +./deploy-invttrdg.sh +``` + +**Token-side failures** (`GITEA_NPM_TOKEN not set and no token file found`) — ensure +the token file exists at `/opt/bytelyst/.gitea_token` (preferred) or +`~/.gitea_npm_token`. The deploy script `unset`s any inherited env var first and +reads from file, so a stale shell export won't mask a missing/wrong file. + +## 5. Troubleshooting quick reference + +| Symptom | Context | Cause | Fix | +|---|---|---|---| +| `pnpm install` hits `localhost:3300` / ECONNREFUSED / 401 | Local laptop | Repo not beside `learning_ai_common_plat`; `.pnpmfile.cjs` fell back to registry | Move repo into the workspace parent, or set `BYTELYST_COMMON_PLAT_ROOT`; re-run `pnpm install` | +| `Cannot find module @bytelyst/...` after install | Local | `learning_ai_common_plat` packages not built | `cd ../learning_ai_common_plat && pnpm install && pnpm build` | +| `Required @bytelyst packages not published` | Deploy/CI | Packages missing from Gitea registry | `python3 /opt/bytelyst/republish_packages.py`, then redeploy | +| `GITEA_NPM_TOKEN not set / no token file` | Deploy/CI | Missing token file | Create `/opt/bytelyst/.gitea_token` or `~/.gitea_npm_token` | +| Docker build can't reach registry | Docker | Missing `--network host` | Build with `docker build --network host …` (deploy scripts already do) | + +## 6. Mental model + +``` + @bytelyst/* dependency + │ + ┌───────────────┴────────────────┐ + │ │ + LOCAL DEV (laptop) DOCKER / CI BUILD + .pnpmfile.cjs → workspace:* .npmrc → Gitea registry + from sibling common_plat http://localhost:3300 + token + │ │ + no registry, no token needs published pkgs + GITEA_NPM_TOKEN + + docker build --network host +``` + +**Rule of thumb:** if you're on a laptop and you see `localhost:3300` or a token +prompt, you're on the wrong path — fix the **directory layout**, not the registry. + +## Related + +- Product repo guidance: `learning_ai_invt_trdg/AGENTS.md` → "Local Development Setup", + `learning_ai_invt_trdg/README.md` → "Shared Dependencies" +- Resolution hook: `/.pnpmfile.cjs`, `/pnpm-workspace.yaml` +- Deploy wrapper: `deploy-invttrdg.sh` (this repo) +- Docker prep (tarball alternative used by some repos): `learning_ai_common_plat/scripts/docker-prep.sh` diff --git a/docs/repo-map.md b/docs/repo-map.md index a78a1a2..e3bcfe6 100644 --- a/docs/repo-map.md +++ b/docs/repo-map.md @@ -53,6 +53,7 @@ Current key files: - `docs/hermes-setup-upgrade-roadmap.md` - `docs/hermes-operations.md` - `docs/llm-utility-workflows.md` +- `docs/gitea-registry-and-package-resolution.md` - `docs/vm-security-blind-spots-roadmap.md` - `docs/vm-exposure-inventory.md`