docs: explain Gitea registry vs workspace package resolution + the registry-offline trap
Document the two ways @bytelyst/* packages resolve (local workspace links vs Gitea npm registry for Docker/CI), the common 'registry offline' local-dev failure and its fix (sibling directory layout, not a token), and the deploy-side 'package not published' / token issues with remediation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Saravanakumar D
parent
dcf017a0de
commit
9d871282c3
189
docs/gitea-registry-and-package-resolution.md
Normal file
189
docs/gitea-registry-and-package-resolution.md
Normal file
@ -0,0 +1,189 @@
|
||||
# Gitea Registry & `@bytelyst/*` Package Resolution
|
||||
|
||||
> **TL;DR** — `@bytelyst/*` packages resolve **two different ways** depending on
|
||||
> context, and conflating them causes the most common onboarding failure:
|
||||
>
|
||||
> | Context | How `@bytelyst/*` resolves | Needs Gitea registry? | Needs `GITEA_NPM_TOKEN`? |
|
||||
> |---|---|---|---|
|
||||
> | **Local dev** (`pnpm install` on a laptop / dev box) | pnpm **workspace links** to the sibling `learning_ai_common_plat` checkout | **No** | **No** |
|
||||
> | **Docker / CI build** (`docker build`, `deploy-*.sh`) | Pulled from the **Gitea npm registry** (`localhost:3300`) | **Yes** | **Yes** |
|
||||
>
|
||||
> If a local install is trying to reach `localhost:3300` or asking for a token,
|
||||
> the repo isn't sitting beside `learning_ai_common_plat` — fix the layout, don't
|
||||
> chase a token. See [The local-dev "registry offline" trap](#the-local-dev-registry-offline-trap).
|
||||
|
||||
---
|
||||
|
||||
## 1. What is Gitea here?
|
||||
|
||||
[Gitea](https://gitea.io) is the **self-hosted Git server + package registry** that
|
||||
backs the ByteLyst ecosystem. It runs on the production VM and exposes a private
|
||||
**npm registry** for the shared `@bytelyst/*` packages (`@bytelyst/cosmos`,
|
||||
`@bytelyst/config`, `@bytelyst/ui`, `@bytelyst/telemetry`, etc.) that live in
|
||||
`learning_ai_common_plat`.
|
||||
|
||||
- **Public host:** `https://gitea.bytelyst.com/`
|
||||
- **npm registry API (on the VM / via tunnel):** `http://localhost:3300/api/packages/bytelyst/npm/`
|
||||
- **Auth:** a personal access token exported as `GITEA_NPM_TOKEN`
|
||||
|
||||
The registry exists so that **Docker images and CI builds** can install
|
||||
`@bytelyst/*` without needing the `learning_ai_common_plat` source tree copied
|
||||
into the build context.
|
||||
|
||||
## 2. The two resolution paths (this is the whole story)
|
||||
|
||||
Every ByteLyst product repo (`learning_ai_invt_trdg`, `learning_ai_clock`, …) is a
|
||||
pnpm workspace that depends on `@bytelyst/*`. There are exactly two ways those
|
||||
deps get satisfied:
|
||||
|
||||
### Path A — Local dev: workspace links (no registry, no token)
|
||||
|
||||
Each product repo ships a `.pnpmfile.cjs` hook and a `pnpm-workspace.yaml` that,
|
||||
**by default**, rewrite every `@bytelyst/*` dependency to `workspace:*` and point
|
||||
it at the sibling `learning_ai_common_plat/packages/*` source on disk.
|
||||
|
||||
```
|
||||
<parent>/
|
||||
├── learning_ai_common_plat/ # source of @bytelyst/* packages
|
||||
└── learning_ai_invt_trdg/ # product repo — resolves siblings via workspace links
|
||||
```
|
||||
|
||||
- `pnpm-workspace.yaml` globs `../learning_ai_common_plat/packages/*`
|
||||
- `.pnpmfile.cjs` (default `BYTELYST_PACKAGE_SOURCE=common-plat`) rewrites
|
||||
`@bytelyst/foo` → `workspace:*` **when it finds the package on disk**
|
||||
- Both lookups are **relative to the repo's parent directory**
|
||||
|
||||
Result: `pnpm install` links packages straight from source. **No `localhost:3300`,
|
||||
no `GITEA_NPM_TOKEN`, no published artifacts required.** This is the correct path
|
||||
for all laptop / dev-box work.
|
||||
|
||||
> Override the sibling location if your checkout lives elsewhere:
|
||||
> `BYTELYST_COMMON_PLAT_ROOT=/abs/path/to/learning_ai_common_plat pnpm install`
|
||||
|
||||
### Path B — Docker / CI build: the Gitea registry (token required)
|
||||
|
||||
Inside a Docker build there is no sibling source tree, so the build pulls
|
||||
`@bytelyst/*` from the Gitea npm registry instead. Example
|
||||
(`learning_ai_invt_trdg/backend/Dockerfile`):
|
||||
|
||||
```dockerfile
|
||||
ARG GITEA_NPM_TOKEN
|
||||
RUN printf '@bytelyst:registry=http://localhost:3300/api/packages/bytelyst/npm/\n//localhost:3300/api/packages/bytelyst/npm/:_authToken=${GITEA_NPM_TOKEN}\n' > .npmrc \
|
||||
&& pnpm install --ignore-scripts --lockfile=false
|
||||
```
|
||||
|
||||
The deploy wrapper `deploy-invttrdg.sh` (in this repo) sets this up:
|
||||
|
||||
1. **Pre-flight check** — verifies each required `@bytelyst/*` package is published
|
||||
(`curl -sf "${GITEA_REGISTRY}${package}"`); fails early with remediation if not.
|
||||
2. **Token resolution** — reads `GITEA_NPM_TOKEN` from `/opt/bytelyst/.gitea_token`
|
||||
or `~/.gitea_npm_token`.
|
||||
3. **Build** — `docker build --network host --build-arg GITEA_NPM_TOKEN=... …`
|
||||
(`--network host` lets the container reach `localhost:3300` on the VM).
|
||||
|
||||
## 3. The local-dev "registry offline" trap
|
||||
|
||||
**Symptom**
|
||||
|
||||
```
|
||||
WARN GET http://localhost:3300/api/packages/.../@bytelyst%2Fcosmos error (ECONNREFUSED)
|
||||
ERR_PNPM_FETCH GITEA_NPM_TOKEN ... 401 / registry offline
|
||||
```
|
||||
|
||||
…on a plain `pnpm install`, on a laptop, with no Docker involved.
|
||||
|
||||
**Root cause**
|
||||
|
||||
The repo was cloned **standalone** (e.g. `D:\SANDBOX\learning_ai_invt_trdg`) with
|
||||
**no sibling `learning_ai_common_plat`**. `.pnpmfile.cjs` couldn't find the package
|
||||
source on disk, so it left the `@bytelyst/*` specifiers untouched and pnpm fell
|
||||
back to the registry in `.npmrc` — which is the **Path B (Docker/CI) registry**,
|
||||
not meant for local dev. With the VM tunnel down and no token, the install dies.
|
||||
|
||||
**Fix (layout, not token)**
|
||||
|
||||
Place the product repo **beside** `learning_ai_common_plat` under a common parent,
|
||||
then reinstall:
|
||||
|
||||
```bash
|
||||
# from the product repo
|
||||
pnpm install # or: pnpm run install:common-plat
|
||||
```
|
||||
|
||||
Or, if you can't move the checkout, point at the sibling explicitly:
|
||||
|
||||
```bash
|
||||
BYTELYST_COMMON_PLAT_ROOT=/abs/path/to/learning_ai_common_plat pnpm install
|
||||
```
|
||||
|
||||
That's it — no token, no running registry needed for local work.
|
||||
|
||||
> **How we hit & solved this (May 2026):** a Cosmos cost-optimization task required
|
||||
> building/testing `learning_ai_invt_trdg`, but it had been cloned standalone under
|
||||
> `D:\SANDBOX\`. `pnpm install` fell back to the offline `localhost:3300` registry
|
||||
> and failed. The fix was **not** to restore the registry or mint a token — it was
|
||||
> to move the repo beside `learning_ai_common_plat` (into the same workspace parent)
|
||||
> so `.pnpmfile.cjs` could rewrite `@bytelyst/*` → `workspace:*`. Install then
|
||||
> succeeded in seconds with zero registry involvement. The product repos'
|
||||
> `AGENTS.md` / `README.md` were updated to document the required sibling layout
|
||||
> and to stop implying `GITEA_NPM_TOKEN` is needed for local dev.
|
||||
|
||||
## 4. The deploy-side "package not published" issue
|
||||
|
||||
**Symptom** — `deploy-invttrdg.sh` aborts:
|
||||
|
||||
```
|
||||
Required @bytelyst packages not published to Gitea registry:
|
||||
@bytelyst/cosmos @bytelyst/config ...
|
||||
```
|
||||
|
||||
**Root cause** — `learning_ai_common_plat` published new/changed packages that were
|
||||
never pushed to the Gitea registry the Docker build pulls from.
|
||||
|
||||
**Fix** — republish the packages to Gitea, then re-run the deploy:
|
||||
|
||||
```bash
|
||||
python3 /opt/bytelyst/republish_packages.py # publishes all @bytelyst/* to Gitea
|
||||
./deploy-invttrdg.sh
|
||||
```
|
||||
|
||||
**Token-side failures** (`GITEA_NPM_TOKEN not set and no token file found`) — ensure
|
||||
the token file exists at `/opt/bytelyst/.gitea_token` (preferred) or
|
||||
`~/.gitea_npm_token`. The deploy script `unset`s any inherited env var first and
|
||||
reads from file, so a stale shell export won't mask a missing/wrong file.
|
||||
|
||||
## 5. Troubleshooting quick reference
|
||||
|
||||
| Symptom | Context | Cause | Fix |
|
||||
|---|---|---|---|
|
||||
| `pnpm install` hits `localhost:3300` / ECONNREFUSED / 401 | Local laptop | Repo not beside `learning_ai_common_plat`; `.pnpmfile.cjs` fell back to registry | Move repo into the workspace parent, or set `BYTELYST_COMMON_PLAT_ROOT`; re-run `pnpm install` |
|
||||
| `Cannot find module @bytelyst/...` after install | Local | `learning_ai_common_plat` packages not built | `cd ../learning_ai_common_plat && pnpm install && pnpm build` |
|
||||
| `Required @bytelyst packages not published` | Deploy/CI | Packages missing from Gitea registry | `python3 /opt/bytelyst/republish_packages.py`, then redeploy |
|
||||
| `GITEA_NPM_TOKEN not set / no token file` | Deploy/CI | Missing token file | Create `/opt/bytelyst/.gitea_token` or `~/.gitea_npm_token` |
|
||||
| Docker build can't reach registry | Docker | Missing `--network host` | Build with `docker build --network host …` (deploy scripts already do) |
|
||||
|
||||
## 6. Mental model
|
||||
|
||||
```
|
||||
@bytelyst/* dependency
|
||||
│
|
||||
┌───────────────┴────────────────┐
|
||||
│ │
|
||||
LOCAL DEV (laptop) DOCKER / CI BUILD
|
||||
.pnpmfile.cjs → workspace:* .npmrc → Gitea registry
|
||||
from sibling common_plat http://localhost:3300 + token
|
||||
│ │
|
||||
no registry, no token needs published pkgs + GITEA_NPM_TOKEN
|
||||
+ docker build --network host
|
||||
```
|
||||
|
||||
**Rule of thumb:** if you're on a laptop and you see `localhost:3300` or a token
|
||||
prompt, you're on the wrong path — fix the **directory layout**, not the registry.
|
||||
|
||||
## Related
|
||||
|
||||
- Product repo guidance: `learning_ai_invt_trdg/AGENTS.md` → "Local Development Setup",
|
||||
`learning_ai_invt_trdg/README.md` → "Shared Dependencies"
|
||||
- Resolution hook: `<product-repo>/.pnpmfile.cjs`, `<product-repo>/pnpm-workspace.yaml`
|
||||
- Deploy wrapper: `deploy-invttrdg.sh` (this repo)
|
||||
- Docker prep (tarball alternative used by some repos): `learning_ai_common_plat/scripts/docker-prep.sh`
|
||||
@ -53,6 +53,7 @@ Current key files:
|
||||
- `docs/hermes-setup-upgrade-roadmap.md`
|
||||
- `docs/hermes-operations.md`
|
||||
- `docs/llm-utility-workflows.md`
|
||||
- `docs/gitea-registry-and-package-resolution.md`
|
||||
- `docs/vm-security-blind-spots-roadmap.md`
|
||||
- `docs/vm-exposure-inventory.md`
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user