learning_ai_common_plat/AGENTS.md
saravanakumardb1 a33f3cfbd6 docs: design system audit, onboarding guide, and AGENTS.md updates
- Comprehensive audit report: DESIGN_SYSTEM_AUDIT_2026-03-03.md
  - Token consumption across all 7 repos
  - 594 hardcoded colors identified
  - Per-product breakdown with migration priorities

- Developer onboarding guide: ONBOARDING.md
  - Platform-specific setup instructions
  - Migration steps for each product
  - Token cheat sheet and troubleshooting

- Updated AGENTS.md with Design System section
  - Never hardcode colors rule
  - Platform usage patterns (Web, iOS, Android, RN)
  - Current adoption status table
  - Validation commands

- Added design-tokens package README
  - Quick start for all platforms
  - Usage examples (CSS, TS, Swift, Kotlin, RN)
  - CI integration examples
2026-03-03 21:55:02 -08:00

423 lines
28 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# AGENTS.md — AI Coding Agent Instructions
> **For:** Claude Code, OpenAI Codex, Cursor, GitHub Copilot, Windsurf Cascade, and any AI coding agent.
> **Repo:** `learning_ai_common_plat` — Shared platform packages + microservices for the ByteLyst ecosystem.
> **See also:** [`README.md`](README.md) for quick start, [`docs/ECOSYSTEM_ARCHITECTURE.md`](docs/ECOSYSTEM_ARCHITECTURE.md) for full architecture.
---
## 1. Project Identity
| Key | Value |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Root package** | `@bytelyst/root` |
| **Library scope** | `@bytelyst/*` (packages/) |
| **Service scope** | `@lysnrai/*` (services/) |
| **Product consumers** | [LysnrAI](../learning_voice_ai_agent), [MindLyst](../learning_multimodal_memory_agents), [ChronoMind](../learning_ai_clock), [JarvisJr](../learning_ai_jarvis_jr), [NomGap](../learning_ai_fastgap), [PeakPulse](../learning_ai_peakpulse) |
| **Product-agnostic** | Yes — every Cosmos doc includes `productId` |
| **Runtime** | Node.js (ESM), TypeScript 5.7+ |
| **Package manager** | pnpm (workspace) |
| **Test runner** | Vitest |
## 2. Monorepo Layout
```
learning_ai_common_plat/
├── packages/ # @bytelyst/* shared libraries
│ ├── errors/ # Typed HTTP service errors (400429)
│ ├── cosmos/ # Azure Cosmos DB client singleton + container registry
│ ├── config/ # Zod-based env loader + product identity + AKV resolver
│ ├── auth/ # JWT utilities, auth middleware, password hashing
│ ├── api-client/ # Configurable fetch wrapper with auth token injection
│ ├── fastify-core/ # createServiceApp() factory + startService() helper
│ ├── react-auth/ # React auth context factory (typed provider + hook)
│ ├── logger/ # Structured logging wrapper (pino-based)
│ ├── testing/ # Shared test mocks, fixtures, Fastify inject helpers
│ ├── blob/ # Azure Blob Storage client + SAS token helpers
│ ├── extraction/ # createExtractionClient(), shared types for extraction consumers
│ ├── monitoring/ # Health-check utilities, Loki/Grafana helpers
│ └── design-tokens/ # Cross-platform tokens (JSON → CSS/TS/Kotlin/Swift)
│ ├── tokens/bytelyst.tokens.json # ← CANONICAL SOURCE
│ ├── scripts/generate.ts # Token generator
│ └── generated/ # Output: tokens.css, tokens.ts, MindLystTokens.kt, MindLystTheme.swift
├── dashboards/ # Product-agnostic web dashboards (Next.js)
│ ├── admin-web/ # Platform admin console (port 3001)
│ └── tracker-web/ # Issue tracker + public roadmap (port 3003)
├── services/ # @lysnrai/* product-agnostic microservices
│ ├── platform-service/ # Product-agnostic platform: auth, audit, flags, notifications, blob,
│ │ # invitations, referrals, promos, subscriptions, usage, plans,
│ │ # licenses, stripe, items, comments, votes, public (port 4003)
│ │ # NOTE: Product-specific modules migrated to product repos' backend/
│ ├── extraction-service/ # LangExtract text extraction + Python sidecar (port 4005)
│ └── monitoring/ # Loki + Grafana config, health-check script
├── docs/ # Architecture docs, roadmap, analysis
├── package.json # Root scripts: build, test, typecheck, clean
├── pnpm-workspace.yaml # Workspace: packages/* + services/* + dashboards/*
├── tsconfig.base.json # Shared TS config (ES2022, NodeNext, strict)
├── vitest.config.ts # Root vitest config (passWithNoTests)
├── docker-compose.yml # Full service stack + Loki + Grafana + Traefik
├── .env.example # Required env vars template
└── .editorconfig # Editor settings
```
## 3. Tech Stack Rules
### TypeScript (all packages + services)
- **Module system:** ESM (`"type": "module"` everywhere)
- **TS target:** ES2022, `moduleResolution: NodeNext`
- **Base config:** All packages/services extend `../../tsconfig.base.json`
- **Builds:** `tsc` — outputs to `dist/`
- **Tests:** Vitest with `passWithNoTests: true` at root
### Service framework (Fastify)
- **Framework:** Fastify 5 with Zod validation
- **Module pattern:** `types.ts``repository.ts``routes.ts` per module
- **Auth:** JWT via `jose` — platform-service issues, all others validate
- **Database:** Azure Cosmos DB via `@azure/cosmos`
- **Logging:** Fastify built-in `req.log` / `app.log` (never `console.log`)
- **Health:** Every service exposes `GET /health``{ status: "ok", service, requestId }`
- **Request tracing:** `x-request-id` header propagated across all services
- **Dev mode:** `tsx watch src/server.ts`
### Package conventions
- All packages export from `src/index.ts``dist/index.js`
- Use `exports` field in `package.json` (not just `main`)
- Peer dependencies for heavy/shared deps (`@azure/cosmos`, `jose`, `bcryptjs`, `react`, `zod`)
- Workspace deps: `"@bytelyst/errors": "workspace:*"`
## 4. Coding Conventions
### MUST follow
- Every Cosmos document MUST include a `productId` field
- Every REST endpoint MUST validate input with Zod schemas
- Every service MUST propagate `x-request-id` headers
- Use `PRODUCT_ID` from `@bytelyst/config` (`loadProductIdentity()`) — never hardcode
- Services use self-contained Zod config schemas in `src/lib/config.ts` (avoids zod version mismatch with shared packages)
- Services re-export from `@bytelyst/*` in their `src/lib/` files (`errors.ts`, `cosmos.ts`, `product-config.ts`)
- Commit messages: `type(scope): description` — types: `feat`, `fix`, `docs`, `refactor`, `test`, `chore`
### MUST NOT do
- Never use `console.log` in production code — use `req.log` or `app.log` in Fastify
- Never use `any` type — use Zod inference or explicit types
- Never hardcode secrets or API keys
- Secret guardrails: Husky runs `scripts/secret-scan-staged.sh` (pre-commit) and `scripts/secret-scan-repo.sh` (pre-push). See `docs/WINDSURF/CODEX_SESSION_SUMMARY_AND_PLAYBOOK.md`.
- Never modify tests to make them pass — fix the actual code
- Never delete existing comments or documentation unless explicitly asked
- Never add emojis to code unless explicitly asked
## 5. Design System (Critical)
### NEVER hardcode colors
**All colors MUST come from `@bytelyst/design-tokens`.** Hardcoded hex values create inconsistency and maintenance debt.
| Platform | ❌ DON'T | ✅ DO |
| ------------------ | ---------------------------- | ------------------------------------------- |
| **Web** | `color: '#5A8CFF'` | `color: var(--ml-accent-primary)` |
| **Web (Tailwind)** | `bg-[#5A8CFF]` | `bg-[var(--ml-accent-primary)]` |
| **iOS** | `Color(hex: 0x5A8CFF)` | `MindLystColors.darkAccentPrimary` |
| **Android/KMP** | `Color(0xFF5A8CFF)` | `Color(MindLystTokens.Dark.ACCENT_PRIMARY)` |
| **React Native** | `backgroundColor: '#5A8CFF'` | `tokens.colors.accentPrimary` |
### Token file locations
```
packages/design-tokens/
├── tokens/bytelyst.tokens.json # ← CANONICAL SOURCE (edit this)
├── generated/
│ ├── tokens.css # Web: CSS custom properties
│ ├── tokens.ts # TypeScript constants
│ ├── MindLystTokens.kt # Kotlin/KMP/Android
│ ├── MindLystTheme.swift # Swift/iOS
│ └── react-native/tokens.ts # React Native
└── scripts/
├── generate.ts # Main generator
├── generate-react-native.ts # RN generator
├── validate-tokens.cjs # Validation script
└── token-coverage.cjs # Coverage report
```
### How to update tokens
1. **Edit** `packages/design-tokens/tokens/bytelyst.tokens.json`
2. **Run** `pnpm --filter @bytelyst/design-tokens generate`
3. **Copy** generated files to consumer repos (see per-repo AGENTS.md)
4. **Commit** both the JSON source AND generated files
### Product-specific token sections
Each product has a dedicated section in `bytelyst.tokens.json`:
| Product | Token Section | Example |
| ---------- | ------------------ | ------------------------------------ |
| MindLyst | `color.brain` | `BRAIN_WORK`, `BRAIN_HOME` |
| JarvisJr | `color.jarvisjr` | `AGENT_COACH`, `AGENT_LINGUA` |
| PeakPulse | `color.peakpulse` | `ACTIVITY_HIKE`, `SPEED_ZONE_FAST` |
| ChronoMind | `color.chronomind` | `URGENCY_CRITICAL`, `FOCUS_MODE` |
| NomGap | `color.nomgap` | `STAGE_KETOSIS`, `AUTOPHAGY_METER` |
| LysnrAI | `color.lysnrai` | `RECORDING_ACTIVE`, `DICTATION_MODE` |
### Validation
Check for hardcoded colors before committing:
```bash
# In any product repo
node ../learning_ai_common_plat/packages/design-tokens/scripts/validate-tokens.cjs src/
# Get coverage report
node ../learning_ai_common_plat/packages/design-tokens/scripts/token-coverage.cjs src/
```
**CI Integration:** All PRs should run the validation script and fail if new hardcoded colors are introduced.
### Current adoption status (2026-03-03)
| Product | Token Adoption | Status |
| ---------------- | -------------- | --------------------------- |
| MindLyst iOS | 100% | ✅ Good |
| MindLyst Android | 100% | ✅ Good |
| ChronoMind Web | 0% | ❌ Critical |
| ChronoMind iOS | 0% | ❌ Critical |
| PeakPulse iOS | 0% | ❌ Needs tokens |
| NomGap | 0% | ❌ Critical (466 hardcoded) |
| LysnrAI iOS | 0% | ❌ Critical |
| JarvisJr iOS | 0% | ❌ Critical |
See full audit: [`docs/design-system/DESIGN_SYSTEM_AUDIT_2026-03-03.md`](docs/design-system/DESIGN_SYSTEM_AUDIT_2026-03-03.md)
## 6. File Ownership Map
| Domain | Location | Key Files |
| ----------------------- | ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------- |
| **Errors** | `packages/errors/` | `src/index.ts``BadRequestError`, `UnauthorizedError`, `ForbiddenError`, `NotFoundError`, `ConflictError`, `RateLimitError` |
| **Cosmos client** | `packages/cosmos/` | `src/index.ts``getCosmosClient()`, `getContainer()`, container registry |
| **Config / Product ID** | `packages/config/` | `src/index.ts``loadEnvConfig()`, `loadProductIdentity()`, `getProductId()` |
| **JWT / Auth** | `packages/auth/` | `src/index.ts``signJwt()`, `verifyJwt()`, `hashPassword()`, `verifyPassword()`, auth middleware |
| **API client** | `packages/api-client/` | `src/index.ts``createApiClient()` with token injection |
| **React auth** | `packages/react-auth/` | `src/index.ts``createAuthContext()` factory (provider + hook) |
| **Design tokens** | `packages/design-tokens/` | `tokens/bytelyst.tokens.json` (source), `scripts/generate.ts` (generator), `generated/` (output) |
| **Auth / JWT issue** | `services/platform-service/` | `src/modules/auth/` |
| **Feature flags** | `services/platform-service/` | `src/modules/flags/` — FNV-1a hash for deterministic rollout |
| **Blob storage** | `services/platform-service/` | `src/modules/blob/`, `src/lib/blob.ts` — SAS tokens, CRUD |
| **Audit log** | `services/platform-service/` | `src/modules/audit/` |
| **Notifications** | `services/platform-service/` | `src/modules/notifications/` |
| **Rate limiting** | `services/platform-service/` | `src/modules/rate-limit/` |
| **Subscriptions** | `services/platform-service/` | `src/modules/subscriptions/` |
| **Stripe webhooks** | `services/platform-service/` | `src/modules/stripe/` |
| **Usage tracking** | `services/platform-service/` | `src/modules/usage/` |
| **Plans** | `services/platform-service/` | `src/modules/plans/` |
| **Licenses** | `services/platform-service/` | `src/modules/licenses/` |
| **Invitations** | `services/platform-service/` | `src/modules/invitations/` |
| **Referrals** | `services/platform-service/` | `src/modules/referrals/` |
| **Promos** | `services/platform-service/` | `src/modules/promos/` |
| **Tracker items** | `services/platform-service/` | `src/modules/items/` |
| **Public roadmap** | `services/platform-service/` | `src/modules/public/` |
| **Tracker comments** | `services/platform-service/` | `src/modules/comments/` |
| **Tracker votes** | `services/platform-service/` | `src/modules/votes/` |
| **Extraction routes** | `services/extraction-service/` | `src/modules/extract/` — POST /extract, /extract/batch, /extract/jobs, /extract/models |
| **Extraction tasks** | `services/extraction-service/` | `src/modules/tasks/` — predefined task library (triage, transcript, memory-insight, etc.) |
| **Extraction Python** | `services/extraction-service/` | `python/src/` — LangExtract sidecar (FastAPI :4006), extractor, task registry, language detection |
| **Extraction package** | `packages/extraction/` | `src/index.ts``createExtractionClient()`, shared types for consumers |
| **Monitoring** | `services/monitoring/` | `health-check.ts`, `loki/`, `grafana/` |
### Dashboard Consumers (via `file:` refs)
The following dashboards consume `@bytelyst/*` packages:
| Dashboard | Location | Packages Used |
| -------------------- | ------------------------------------- | ------------------------------------------------------------ |
| `admin-web` | `dashboards/admin-web/` (this repo) | api-client, auth, config, cosmos, errors, logger, react-auth |
| `user-dashboard-web` | `../learning_voice_ai_agent/` | api-client, auth, config, cosmos, errors, logger, react-auth |
| `tracker-web` | `dashboards/tracker-web/` (this repo) | api-client, config, cosmos, errors |
**Prerequisite:** Run `pnpm build` in this repo before running `npm install` in any dashboard.
## 6. How to Run Things
```bash
# ── Install ────────────────────────────────────────
pnpm install
# ── Build all packages + services ──────────────────
pnpm build
# ── Run all tests ────────────────────────────────
pnpm test
# ── Type-check everything ──────────────────────────
pnpm typecheck
# ── Clean dist/ in all packages ────────────────────
pnpm clean
# ── Run a specific service in dev mode ─────────────
pnpm --filter @lysnrai/platform-service dev # port 4003
pnpm --filter @lysnrai/extraction-service dev # port 4005
# ── Run tests for one workspace ────────────────────
pnpm --filter @lysnrai/platform-service test
pnpm --filter @bytelyst/errors test
# ── Generate design tokens ─────────────────────────
pnpm --filter @bytelyst/design-tokens generate
# ── Docker Compose (all services + monitoring) ─────
docker compose up -d
docker compose down
# ── Portable builds for consumer repos ────────────
# Pack @bytelyst/* tarballs so Docker/CI builds don't need this sibling repo
./scripts/prep-consumer.sh /path/to/consumer-dir # pack + rewrite
./scripts/prep-consumer.sh /path/to/consumer-dir --restore # undo
# ── Health check all services ──────────────────────
pnpm --filter @lysnrai/monitoring check
```
## 7. Common Patterns
### Adding a new Fastify module
1. Create `services/<service>/src/modules/<name>/types.ts` — Zod schemas + TS interfaces
2. Create `services/<service>/src/modules/<name>/repository.ts` — Cosmos CRUD
3. Create `services/<service>/src/modules/<name>/routes.ts` — REST endpoints
4. Register in `services/<service>/src/server.ts`: `await app.register(routes, { prefix: "/api" })`
5. Add tests: `services/<service>/src/modules/<name>/<name>.test.ts`
### Adding a new shared package
1. Create `packages/<name>/` with `package.json` (`name: "@bytelyst/<name>"`)
2. Add `src/index.ts` with exports
3. Add `tsconfig.json` extending `../../tsconfig.base.json`
4. Heavy deps go in `peerDependencies`, not `dependencies`
5. Consumer services add `"@bytelyst/<name>": "workspace:*"` to their `dependencies`
### Using shared packages in services
Services re-export from `@bytelyst/*` in `src/lib/` for clean internal imports:
```typescript
// src/lib/errors.ts
export { BadRequestError, NotFoundError, ConflictError } from '@bytelyst/errors';
// src/lib/cosmos.ts
export { getCosmosClient, getContainer } from '@bytelyst/cosmos';
// src/lib/product-config.ts
import { loadProductIdentity } from '@bytelyst/config';
const identity = loadProductIdentity();
export const PRODUCT_ID = identity.productId;
```
### Design token workflow
1. Edit `packages/design-tokens/tokens/bytelyst.tokens.json`
2. Run `pnpm --filter @bytelyst/design-tokens generate`
3. Copy generated files to consumer repos:
- `MindLystTokens.kt``mindlyst-native/shared/.../theme/`
- `MindLystTheme.swift``mindlyst-native/iosApp/`
- Token CSS → `design-system/web/mindlyst.css` (token section only)
## 8. Environment Variables
Required by all services (see `.env.example`):
```
COSMOS_ENDPOINT=https://<account>.documents.azure.com:443/
COSMOS_KEY=<primary-key>
COSMOS_DATABASE=lysnrai
JWT_SECRET=<shared-secret>
```
Additional per-service:
```
# platform-service
STRIPE_SECRET_KEY=<stripe-key>
STRIPE_WEBHOOK_SECRET=<webhook-secret>
# platform-service (blob storage)
AZURE_BLOB_CONNECTION_STRING=<connection-string>
AZURE_BLOB_ACCOUNT_NAME=bytelystblobs
AZURE_BLOB_ACCOUNT_KEY=<account-key>
# All services
DEFAULT_PRODUCT_ID=lysnrai
```
## 9. Dependency Graph
```
@bytelyst/errors ← no deps (foundation)
@bytelyst/cosmos ← peers: @azure/cosmos
@bytelyst/config ← peers: zod; includes AKV resolver (resolveKeyVaultSecrets)
@bytelyst/auth ← peers: jose, bcryptjs
@bytelyst/api-client ← no deps
@bytelyst/react-auth ← deps: @bytelyst/api-client; peers: react
@bytelyst/blob ← peers: @azure/storage-blob
@bytelyst/extraction ← no deps (types + client factory)
@bytelyst/monitoring ← no deps (health-check utilities)
@bytelyst/fastify-core ← deps: fastify; createServiceApp() + startService()
@bytelyst/logger ← deps: pino
@bytelyst/testing ← deps: vitest; shared mocks + Fastify inject helpers
@bytelyst/design-tokens ← no deps (standalone generator)
@lysnrai/platform-service ← @bytelyst/{fastify-core, config, cosmos, errors, auth, blob}
@lysnrai/extraction-service ← @bytelyst/{fastify-core, config, cosmos, errors, auth}
```
Build order: packages first (they have no inter-deps), then services. `pnpm build` handles this automatically via workspace topology.
## 10. Key Documents
| When you need to... | Read this |
| -------------------------------- | -------------------------------------------------------------------------------------------------- |
| Understand the full architecture | [`docs/ECOSYSTEM_ARCHITECTURE.md`](docs/ECOSYSTEM_ARCHITECTURE.md) |
| See the extraction roadmap | [`docs/ROADMAP.md`](docs/ROADMAP.md) |
| Understand why this repo exists | [`docs/COMMON_PLATFORM_ANALYSIS.md`](docs/COMMON_PLATFORM_ANALYSIS.md) |
| Quick start / consuming packages | [`README.md`](README.md) |
| LysnrAI product repo conventions | [`../learning_voice_ai_agent/AGENTS.md`](../learning_voice_ai_agent/AGENTS.md) |
| MindLyst native repo conventions | [`../learning_multimodal_memory_agents/AGENTS.md`](../learning_multimodal_memory_agents/AGENTS.md) |
## 11. Service Test Counts
| Service / Package | Tests | Runner |
| ------------------ | -------- | ------ |
| platform-service | 800 | vitest |
| extraction-service | 46 | vitest |
| packages (various) | ~30 | vitest |
| **Total** | **876+** | |
> Note: billing-service, growth-service, and tracker-service were consolidated into platform-service (Feb 2026).
> Product-specific modules migrated to product repo backends (see below).
## 13. Product-Specific Backends
Product-specific API modules have been migrated from platform-service into each product repo's `backend/` directory:
| Product | Repo | Port | Modules | Tests |
| ---------- | ----------------------------------------------- | ---- | ----------------------------------------------------------------------------------------- | ----- |
| PeakPulse | `../learning_ai_peakpulse/backend/` | 4010 | peak-sessions, peak-routes | 32 |
| ChronoMind | `../learning_ai_clock/backend/` | 4011 | timers, routines, households, shared-timers | 130 |
| JarvisJr | `../learning_ai_jarvis_jr/backend/` | 4012 | jarvis-agents, jarvis-sessions, jarvis-memory, jarvis-teams, marketplace | 198 |
| NomGap | `../learning_ai_fastgap/backend/` | 4013 | fasting-sessions, fasting-protocols, body-stages, social-fasting, meal-log, push-triggers | 152 |
| MindLyst | `../learning_multimodal_memory_agents/backend/` | 4014 | brains, memory, reflections, daily-briefs, streaks | 59 |
Each product backend uses `@bytelyst/*` packages via `file:` refs and follows the same Fastify module pattern.
## 12. Common Pitfalls
1. **Don't import `zod` from `@bytelyst/config` in services** — services bundle their own zod version. Use self-contained Zod schemas in `src/lib/config.ts`.
2. **Don't forget `productId`** — every Cosmos document must include it.
3. **Don't use `dependencies` for heavy libs in packages** — use `peerDependencies` so consumers control the version.
4. **Don't mix up package scopes** — libraries are `@bytelyst/*`, services are `@lysnrai/*`.
5. **Don't run `npm` commands** — this is a pnpm workspace. Always use `pnpm`.
6. **Don't modify generated files directly** — edit `bytelyst.tokens.json` and re-run the generator.
7. **Build packages before testing services** — service tests may import from `@bytelyst/*` dist. Run `pnpm build` first if you get import errors.