docs(devops): add repo context note

REPO_CONTEXT.md

@@ -0,0 +1,355 @@
+# ByteLyst DevOps Tools - Repository Context
+
+> **Purpose**: This file provides quick context for AI coding agents about what this repository contains and how to use it effectively.
+
+## What This Repository Provides
+
+This is the **operational tools repository** for the ByteLyst ecosystem. It contains:
+
+- **Deployment scripts** for all products (deploy-{project}.sh pattern)
+- **GitHub administration tools** and multi-repo safety helpers
+- **Operational automation** scripts
+- **Internal DevOps dashboard** (dashboard/ - full product)
+- **Cross-repo utilities** and maintenance scripts
+
+## Key Locations
+
+### Deployment Scripts (MOST IMPORTANT FOR PRODUCT DEPLOYMENTS)
+- **Location**: Root level `deploy-*.sh` files
+- **Pattern**: `deploy-{project}.sh`
+- **Available Scripts**:
+  - `deploy-invttrdg.sh` - Investment Trading deployment
+  - `deploy-notes.sh` - Notes deployment
+  - `deploy-clock.sh` - Clock deployment
+  - `deploy-all.sh` - Deploy all products
+- **Always check here first** before attempting custom deployment solutions
+
+### Deployment Script Pattern
+All deployment scripts follow this standard pattern:
+1. Dirty checks (uncommitted changes, unpushed commits)
+2. Pull and rebase from origin/main
+3. Run smoke tests (if available)
+4. Build Docker images with `--network host` (critical for Gitea registry access)
+5. Deploy containers via docker-compose
+6. Health checks and endpoint verification
+
+### Critical Deployment Configuration
+- **Docker Build**: Uses `--network host` flag for Gitea registry access
+- **Gitea Token**: Required for @bytelyst/* package installation
+- **Token Location**: 
+  - Environment variable: `GITEA_NPM_TOKEN`
+  - Fallback 1: `/opt/bytelyst/.gitea_token`
+  - Fallback 2: `$HOME/.gitea_npm_token`
+- **Build Metadata**: Automatically includes commit SHA, branch, author, build time
+
+### GitHub Administration Tools
+- **bytelyst-cli.sh**: Main CLI for common GitHub operations
+- **remove_user_interactive.sh**: Interactive collaborator removal
+- **remove_user_guided.sh**: Guided removal workflow
+- **remove_user_from_repos.sh**: Scripted removal flow
+
+### Multi-Repo Git Safety Tools
+- **Location**: `git-work-safety-tools/`
+- **Purpose**: Safe bulk git workflows across multiple repositories
+- **Key Scripts**:
+  - `git_repos_status.sh` - Scan many repos for dirty state
+  - `git_repos_rebase_commit_push.sh` - Safe rebase + commit + push
+  - `multi_repo_safe_push.sh` - Safe multi-repo pushing
+
+### Scripts Directory
+- **Location**: `scripts/`
+- **Purpose**: Self-contained operational scripts
+- **Pattern**: More focused than root-level helpers
+
+### Documentation
+- **Location**: `docs/`
+- **Key Files**:
+  - `getting-started.md` - Onboarding guide
+  - `repo-map.md` - Repository structure map
+  - `tooling-status.md` - Tool availability status
+  - `operations.md` - Operational patterns
+
+## Common Usage Patterns
+
+### Deployment (Most Common Use Case)
+```bash
+# Deploy specific product
+./deploy-invttrdg.sh
+
+# Force deployment (skip dirty checks)
+./deploy-invttrdg.sh --force
+
+# Skip health checks
+./deploy-invttrdg.sh --skip-health-check
+
+# Deploy all products
+./deploy-all.sh
+```
+
+### GitHub Operations
+```bash
+# List public repos for a user
+./bytelyst-cli.sh list-public-repos --user <username>
+
+# List private repos for an org
+./bytelyst-cli.sh list-private-repos --org <orgname>
+
+# Interactive user removal
+./remove_user_interactive.sh
+```
+
+### Multi-Repo Safety
+```bash
+# Check status of multiple repos
+./git-work-safety-tools/git_repos_status.sh
+
+# Safe rebase + commit + push across repos
+./git-work-safety-tools/git_repos_rebase_commit_push.sh
+```
+
+## Deployment Script Details
+
+### Standard Deployment Flow
+1. **Prerequisites Check**: Verify repo directory exists
+2. **Dirty Checks**: 
+   - Uncommitted changes (fail unless --force)
+   - Untracked files (fail unless --force)
+   - Unpushed commits (fail unless --force)
+3. **Git Sync**: Pull and rebase origin/main
+4. **Smoke Tests**: Run `scripts/smoke-release.sh` if available
+5. **Docker Build**:
+   - Build backend and web images
+   - Use `--network host` for Gitea registry access
+   - Pass GITEA_NPM_TOKEN via BuildKit secrets
+   - Include build metadata (commit SHA, branch, etc.)
+6. **Deployment**: `docker compose up -d --no-build`
+7. **Health Checks**: Verify local and production endpoints
+
+### Docker Build Configuration
+```bash
+docker build --network host \
+  --secret id=gitea_npm_token,env=GITEA_NPM_TOKEN \
+  --build-arg "BYTELYST_COMMIT_SHA=${COMMIT_SHA}" \
+  --build-arg "BYTELYST_BRANCH=${BRANCH}" \
+  -f Dockerfile -t image:tag .
+```
+
+### Health Check Endpoints
+- **Backend**: `http://localhost:4025/health/live`
+- **Web**: `http://localhost:3085`
+- **Production API**: `https://api.bytelyst.com/invttrdg`
+- **Production Web**: `https://invttrdg.bytelyst.com`
+
+## Important Notes
+
+### Before Deployment
+1. **Always check deployment scripts here first** - don't build custom deployment solutions
+2. Ensure GITEA_NPM_TOKEN is set correctly
+3. Run smoke tests if available
+4. Check git status for uncommitted changes
+
+### Common Deployment Issues
+- **"Unauthorized - 401" from Gitea**: Wrong token, check GITEA_NPM_TOKEN
+- **"Cannot find module @bytelyst/..."**: Package not built, check learning_ai_common_plat
+- **Docker build network errors**: Missing --network host flag
+- **Port conflicts**: Check if containers are already running
+
+### Gitea Registry Access
+- **Registry URL**: `http://localhost:3300/api/packages/bytelyst/npm/`
+- **Access Method**: --network host flag in Docker build
+- **Why this works**: Allows build container to access host's localhost:3300
+- **Alternative approaches** (network bridges, container names) typically fail
+
+### Build Metadata
+Deployment scripts automatically inject:
+- `BYTELYST_COMMIT_SHA`: Short commit hash
+- `BYTELYST_COMMIT_SHA_FULL`: Full commit hash
+- `BYTELYST_BRANCH`: Git branch name
+- `BYTELYST_BUILT_AT`: ISO timestamp
+- `BYTELYST_COMMIT_AUTHOR`: Commit author
+- `BYTELYST_COMMIT_MESSAGE`: Commit message (truncated)
+- `BYTELYST_DOCKER_IMAGE`: Docker image tag
+
+## Internal DevOps Dashboard
+
+### Dashboard Product
+- **Location**: `dashboard/`
+- **Purpose**: Internal deployment orchestration and service monitoring
+- **Architecture**: Full ByteLyst product (backend + web)
+- **Backend**: Fastify 5 (port 4004)
+- **Web**: Next.js 16 (port 3000)
+- **Integration**: Uses @bytelyst/* packages, links to admin-web
+
+### Dashboard Setup
+See `dashboard/README.md` for setup and usage instructions.
+
+## Safety and Operational Guidelines
+
+### Sensitive Files
+- `accounts.json` - GitHub account credentials
+- `.env` files - Environment configuration
+- Generated contributor/output data - Operational snapshots
+- **Never echo these files** unless task requires it
+
+### Destructive Operations
+- Many scripts are operational and potentially destructive
+- Preserve prompts, dry-run behavior, and confirmations
+- Review scripts before automation use
+- Some encode assumptions about ByteLyst org structure
+
+### Pre-commit Validation
+```bash
+pre-commit run --all-files
+```
+
+## Quick Reference for Common Tasks
+
+| Task | Command |
+|------|---------|
+| Deploy invttrdg | `./deploy-invttrdg.sh` |
+| Force deployment | `./deploy-invttrdg.sh --force` |
+| Deploy all products | `./deploy-all.sh` |
+| Check repo status | `./git-work-safety-tools/git_repos_status.sh` |
+| List user repos | `./bytelyst-cli.sh list-public-repos --user <user>` |
+| Remove user interactively | `./remove_user_interactive.sh` |
+| Run pre-commit checks | `pre-commit run --all-files` |
+
+## Related Repositories
+
+- **learning_ai_common_plat**: Shared platform packages consumed by products
+- **Product repos**: learning_ai_invt_trdg, learning_ai_notes, etc. (deployed via scripts here)
+
+## First Steps for Any Deployment Task
+
+1. **Check deployment scripts first** - look for `deploy-{project}.sh`
+2. **Read the script** - understand its flow and requirements
+3. **Check GITEA_NPM_TOKEN** - ensure it's set correctly
+4. **Verify companion repos** - ensure learning_ai_common_plat is accessible
+5. **Run the deployment script** - use appropriate flags (--force, --skip-health-check)
+6. **Never build custom deployment** unless script doesn't exist for your use case
+
+## New Deployment Pattern for @bytelyst/* Packages (Learned 2026-05)
+
+### Background
+Products that consume @bytelyst/* packages from learning_ai_common_plat require a special build preparation step because:
+- Gitea registry is not accessible from Docker build context
+- Direct npm install would fail during Docker build
+- Dependencies need to be pre-packaged as tarballs
+
+### Required Pre-Build Step
+**Products using @bytelyst/* packages MUST have a `scripts/docker-prep.sh` script:**
+
+```bash
+# Before building Docker images
+bash scripts/docker-prep.sh
+
+# After building Docker images
+bash scripts/docker-prep.sh --restore
+```
+
+### Dockerfile Changes Required
+**Dockerfiles MUST be updated to remove Gitea secret mounting:**
+
+**Before (incorrect):**
+```dockerfile
+RUN --mount=type=secret,id=gitea_npm_token \
+  TOKEN=$(cat /run/secrets/gitea_npm_token) && \
+  echo "//localhost:3300/api/packages/bytelyst/npm/:_authToken=$TOKEN" >> .npmrc && \
+  pnpm install --ignore-scripts --lockfile=false
+```
+
+**After (correct):**
+```dockerfile
+COPY .docker-deps/ ../.docker-deps/
+RUN pnpm install --ignore-scripts --lockfile=false
+```
+
+### Environment Variable Configuration
+**NEXT_PUBLIC_ variables MUST use public API URLs:**
+
+**Incorrect:**
+```yaml
+NEXT_PUBLIC_BACKEND_URL: http://backend:4011
+```
+
+**Correct:**
+```yaml
+NEXT_PUBLIC_BACKEND_URL: https://api.bytelyst.com/{product}
+```
+
+### Docker Compose Format
+**Use YAML mapping syntax, not array syntax:**
+
+**Incorrect:**
+```yaml
+environment:
+  - NODE_ENV=production
+  - PORT:4011
+```
+
+**Correct:**
+```yaml
+environment:
+  NODE_ENV: production
+  PORT: 4011
+```
+
+### Container Naming
+**Container names MUST match Caddyfile configuration:**
+
+```yaml
+services:
+  backend:
+    container_name: {product}-backend
+  web:
+    container_name: {product}-web
+```
+
+### DNS Setup
+**Add DNS records via GoDaddy API for new subdomains:**
+
+```bash
+curl -X PUT "https://api.godaddy.com/v1/domains/bytelyst.com/records/A/{subdomain}" \
+  -H "Authorization: sso-key $GODADDY_API_KEY:$GODADDY_API_SECRET" \
+  -H "Content-Type: application/json" \
+  -d '[{"data": "187.124.159.82", "name": "{subdomain}", "ttl": 600, "type": "A"}]'
+```
+
+### Caddy Configuration
+**Add subdomain routing to /opt/bytelyst/Caddyfile:**
+
+```caddy
+{subdomain}.bytelyst.com {
+  encode gzip
+  reverse_proxy {product}-web:{port}
+}
+```
+
+### Products Requiring This Pattern
+- learning_ai_notes (NoteLett)
+- learning_ai_clock (ChronoMind)
+- learning_ai_invttrdg (Investment Trading)
+- Any product consuming @bytelyst/* packages
+
+### Deployment Script Updates
+**Deployment scripts should be updated to:**
+1. Check for docker-prep.sh existence
+2. Run docker-prep.sh before Docker build
+3. Run docker-prep.sh --restore after successful deployment
+4. Provide option to skip docker-prep if already run
+
+### Health Check Requirements
+**Backend Dockerfiles MUST include wget for health checks:**
+
+```dockerfile
+RUN apk add --no-cache wget
+```
+
+Health check in docker-compose.yml:
+```yaml
+healthcheck:
+  test: ["CMD", "wget", "--spider", "-q", "http://localhost:{port}/health"]
+  interval: 30s
+  timeout: 5s
+  retries: 3
+```