learning_ai_common_plat/docs/COMMON_PLATFORM_ANALYSIS.md
saravanakumardb1 a874a4332b docs: add common platform analysis, ecosystem architecture, and drawio diagram
- COMMON_PLATFORM_ANALYSIS.md: identifies 8 shared packages to extract from LysnrAI and MindLyst repos (~1,580 LOC duplication eliminated)
- ECOSYSTEM_ARCHITECTURE.md: detailed post-refactor architecture with components, services, migration plan, advantages, cautions, versioning, testing, and CI/CD impact
- ecosystem-after-refactor.drawio: 4-layer architecture diagram (clients, repos, common platform, Azure infra)
2026-02-12 10:40:28 -08:00

382 lines
16 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Common Platform Analysis
> **Purpose:** Identify duplicated code across `learning_voice_ai_agent` (LysnrAI) and `learning_multimodal_memory_agents` (MindLyst) that should be extracted into `learning_ai_common_plat` as shared packages.
>
> **Date:** 2026-02-12
---
## Executive Summary
After scanning both repos, there are **8 high-value extraction candidates** across 3 tiers (TypeScript services, Next.js dashboards, design system). The heaviest duplication is in the LysnrAI monorepo where 4 Fastify microservices each contain near-identical copies of 5 foundational files. The MindLyst repo shares the same design token values and will benefit from a shared design system package.
---
## 1. Duplicated Components — TypeScript Microservices
### 1.1 Cosmos DB Client Singleton
**Duplication:** 4 near-identical copies across LysnrAI services + 2 in dashboards = **6 copies**
| Location | Lines |
|----------|-------|
| `services/platform-service/src/lib/cosmos.ts` | 25 |
| `services/billing-service/src/lib/cosmos.ts` | 25 |
| `services/growth-service/src/lib/cosmos.ts` | 25 |
| `services/tracker-service/src/lib/cosmos.ts` | 25 |
| `admin-dashboard-web/src/lib/cosmos.ts` | 112 |
| `user-dashboard-web/src/lib/cosmos.ts` | 94 |
**Pattern:** All do: lazy singleton `CosmosClient`, read `COSMOS_ENDPOINT` + `COSMOS_KEY` + `COSMOS_DATABASE` from env, expose `getContainer(name)`. Dashboard versions add container registry with partition key definitions and TTL config.
**Proposed shared package:** `@bytelyst/cosmos`
```
packages/cosmos/
src/
client.ts — getCosmosClient(), getDatabase(), getContainer()
containers.ts — container registry (names, partition keys, TTL)
types.ts — ContainerConfig interface
index.ts
package.json
```
---
### 1.2 Service Error Classes
**Duplication:** 4 copies across LysnrAI services, each slightly different
| Location | Error Classes |
|----------|--------------|
| `platform-service/src/lib/errors.ts` | ServiceError, NotFound, BadRequest, Unauthorized, Forbidden |
| `billing-service/src/lib/errors.ts` | ServiceError, NotFound, BadRequest, Forbidden, TooManyRequests |
| `growth-service/src/lib/errors.ts` | ServiceError, NotFound, BadRequest, Forbidden |
| `tracker-service/src/lib/errors.ts` | ServiceError, NotFound, BadRequest, Unauthorized, Forbidden, Conflict |
**Pattern:** All share the same `ServiceError` base class with `statusCode` + `message`. Subclasses vary per service (some have Unauthorized, some have Conflict, etc.).
**Proposed shared package:** `@bytelyst/errors`
```
packages/errors/
src/
service-error.ts — base ServiceError class
http-errors.ts — NotFound, BadRequest, Unauthorized, Forbidden, Conflict, TooManyRequests
index.ts
package.json
```
---
### 1.3 JWT Auth Middleware
**Duplication:** 3 copies in LysnrAI services + 2 in dashboards = **5 copies**
| Location | Variant |
|----------|---------|
| `platform-service/src/modules/auth/jwt.ts` | Full: create + verify (issuer service) |
| `tracker-service/src/lib/auth.ts` | Verify-only (consumer service) |
| `billing-service` (via internal key) | Different pattern |
| `admin-dashboard-web/src/lib/auth-server.ts` | create + verify + bcrypt + authenticateUser |
| `user-dashboard-web/src/lib/auth-server.ts` | create + verify + bcrypt + authenticateUser (near-identical) |
**Pattern:** All use `jose` library, HS256, same `getSecret()``TextEncoder.encode(JWT_SECRET)`. Dashboards add bcrypt password hashing and `getCurrentUser()`.
**Proposed shared package:** `@bytelyst/auth`
```
packages/auth/
src/
jwt.ts — createAccessToken, createRefreshToken, verifyToken (configurable issuer/expiry)
middleware.ts — extractAuth (Fastify), getCurrentUser (Next.js)
password.ts — hashPassword, verifyPassword (bcryptjs wrapper)
types.ts — AuthPayload, TokenPayload interfaces
index.ts
package.json
```
---
### 1.4 Fastify Server Bootstrap
**Duplication:** 4 copies in LysnrAI services
| Location | Lines |
|----------|-------|
| `platform-service/src/server.ts` | 87 |
| `billing-service/src/server.ts` | 101 |
| `growth-service/src/server.ts` | 79 |
| `tracker-service/src/server.ts` | 84 |
**Pattern:** All follow the same structure:
1. Create Fastify instance with logger
2. Register CORS (parse `CORS_ORIGIN`)
3. Register Swagger (service-specific title/description)
4. Register Prometheus metrics
5. Add `x-request-id` hook (propagate or generate UUID)
6. Add `/health` endpoint (same shape: `{ status, service, version, timestamp, requestId }`)
7. Set error handler (check `instanceof ServiceError`)
8. Register route modules with `/api` prefix
9. Start listener
**Proposed shared package:** `@bytelyst/fastify-core`
```
packages/fastify-core/
src/
create-app.ts — createServiceApp({ name, version, description }) → configured Fastify instance
request-id.ts — x-request-id hook (reusable plugin)
health.ts — health check route factory
error-handler.ts — ServiceError-aware error handler
index.ts
package.json
```
---
### 1.5 Zod Config Loader
**Duplication:** 4 copies in LysnrAI services
| Location | Shared Fields |
|----------|--------------|
| `platform-service/src/lib/config.ts` | PORT, HOST, NODE_ENV, CORS_ORIGIN, SERVICE_NAME, COSMOS_ENDPOINT, COSMOS_KEY, COSMOS_DATABASE |
| `billing-service/src/lib/config.ts` | Same base + STRIPE_*, BILLING_* |
| `growth-service/src/lib/config.ts` | Same base + STRIPE_*, WEBHOOK_* |
| `tracker-service/src/lib/config.ts` | Same base + JWT_SECRET, DEFAULT_PRODUCT_ID |
**Pattern:** Every service has 7 identical base fields (PORT, HOST, NODE_ENV, CORS_ORIGIN, SERVICE_NAME, COSMOS_ENDPOINT, COSMOS_KEY, COSMOS_DATABASE) and then service-specific extensions.
**Proposed shared package:** `@bytelyst/config`
```
packages/config/
src/
base-schema.ts — baseEnvSchema (Zod object with the 8 common fields)
loader.ts — loadConfig(extendedSchema?) merges base + service-specific
index.ts
package.json
```
---
### 1.6 Product Config / Identity
**Duplication:** 4 copies in LysnrAI services + 1 JSON canonical source
| Location | Content |
|----------|---------|
| `shared/product.json` | Canonical: `{ productId, displayName, licensePrefix, ... }` |
| `platform-service/src/lib/product-config.ts` | `PRODUCT_ID = "lysnrai"` (hardcoded) |
| `billing-service/src/lib/product-config.ts` | Same (hardcoded) |
| `growth-service/src/lib/product-config.ts` | Same (hardcoded) |
| `tracker-service/src/lib/product-config.ts` | `DEFAULT_PRODUCT_ID = env || "lysnrai"` |
**Pattern:** Every service hardcodes or reads the same product identity. Should read from a shared source.
**Proposed:** Include in `@bytelyst/config` — export `loadProductConfig()` that reads `shared/product.json` or env vars. Each consuming project provides its own `product.json`.
---
## 2. Duplicated Components — Next.js Dashboards
### 2.1 Auth Context (React)
**Duplication:** 3 copies across LysnrAI dashboards
| Location | Variant |
|----------|---------|
| `admin-dashboard-web/src/lib/auth-context.tsx` | AdminUser, localStorage keys: `admin_*` |
| `user-dashboard-web/src/lib/auth-context.tsx` | PortalUser, localStorage keys: `portal_*`, SSO cookie support |
| `tracker-dashboard-web/src/lib/` (auth context) | TrackerUser, localStorage keys: `tracker_*` |
**Pattern:** All implement: `AuthProvider`, `useAuth()`, localStorage-backed user + token storage, login/logout. Differ only in: user type, storage key prefix, and SSO support.
**Proposed shared package:** `@bytelyst/react-auth`
```
packages/react-auth/
src/
auth-context.tsx — generic AuthProvider<TUser>, configurable storage keys + login endpoint
use-auth.ts — typed useAuth() hook
types.ts — BaseUser interface, AuthConfig
index.ts
package.json
```
---
### 2.2 API Fetch Utility
**Duplication:** 4+ copies across dashboards and service clients
| Location | Pattern |
|----------|---------|
| `admin-dashboard-web/src/lib/api.ts` | `apiFetch<T>(path, options)``{ data, error }` |
| `user-dashboard-web/src/lib/platform-client.ts` | `request<T>(path, options)``T` (throws) |
| `user-dashboard-web/src/lib/billing-client.ts` | Same `request<T>` pattern |
| `user-dashboard-web/src/lib/growth-client.ts` | Same `request<T>` pattern |
| `tracker-dashboard-web/src/lib/tracker-client.ts` | `apiFetch<T>` with Bearer token |
**Pattern:** All wrap `fetch()` with: base URL, JSON headers, auth header injection, error parsing. Two variants: result-tuple `{ data, error }` vs throw-on-error.
**Proposed shared package:** `@bytelyst/api-client`
```
packages/api-client/
src/
client.ts — createApiClient({ baseUrl, getToken?, headers? }) → { fetch, safeFetch }
types.ts — ApiResult<T>, ApiError
index.ts
package.json
```
---
### 2.3 Utility Functions
**Duplication:** Identical `cn()` function in 2 dashboards
| Location |
|----------|
| `admin-dashboard-web/src/lib/utils.ts` |
| `user-dashboard-web/src/lib/utils.ts` |
**Pattern:** `cn(...inputs) → twMerge(clsx(inputs))` — standard shadcn/ui utility.
**Proposed:** Include in `@bytelyst/ui-utils` or just in `@bytelyst/react-auth` as a peer export.
---
## 3. Duplicated Components — Design System
### 3.1 Design Tokens
**Duplication:** Same color/spacing/typography values maintained across **5 formats** in MindLyst + referenced in LysnrAI dashboards
| Location | Format |
|----------|--------|
| `design-system/tokens/mindlyst.tokens.json` | Canonical JSON |
| `shared/src/.../theme/MindLystTokens.kt` | KMP Kotlin object |
| `iosApp/MindLystTheme.swift` | SwiftUI structs |
| `androidApp/.../MindLystTheme.kt` | Compose theme |
| `web/src/styles/globals.css` | CSS custom properties |
| `design-system/web/mindlyst.css` | CSS custom properties (duplicate of above) |
**Note:** LysnrAI's admin/user dashboards use TailwindCSS + shadcn/ui with a different color system currently. If both products converge on a shared design language, the token JSON can be the single source.
**Proposed shared package:** `@bytelyst/design-tokens`
```
packages/design-tokens/
tokens/
colors.json — palette, semantic dark/light, brain gradients
typography.json — font families, sizes, weights, line heights
spacing.json — 8pt grid scale
radius.json — border radius scale
motion.json — duration + easing
layout.json — breakpoints, gutter, max-width
generated/ — auto-generated from tokens JSON
tokens.css — CSS custom properties (--ml-*)
tokens.ts — TypeScript constants
tokens.kt — Kotlin object (for KMP)
tokens.swift — Swift structs (for iOS)
scripts/
generate.ts — reads JSON, outputs all formats
package.json
```
---
## 4. Potential Future Shared Components
These are not exact duplicates yet but represent patterns that will diverge if not unified:
| Component | LysnrAI | MindLyst | Shared Potential |
|-----------|---------|----------|------------------|
| **OpenAI API client** | Python `src/llm/` + Azure OpenAI | KMP `api/OpenAIClient.kt` | Shared TS client for backend services |
| **Whisper/STT client** | Python `src/audio/azure_stt.py` | KMP `api/WhisperClient.kt` | Both need speech-to-text |
| **Health check aggregator** | `services/monitoring/health-check.ts` | N/A (could reuse) | Generic multi-service health poller |
| **Docker Compose patterns** | Full stack compose | Planned | Shared compose fragments |
| **CI workflow templates** | 9 GitHub Actions | 1 CI workflow | Reusable workflow templates |
| **Python Cosmos client** | `src/cloud/cosmos_client.py` | N/A | If MindLyst adds a Python backend |
---
## 5. Recommended Package Structure
```
learning_ai_common_plat/
├── packages/
│ ├── cosmos/ — Azure Cosmos DB client singleton + container registry
│ ├── errors/ — Typed HTTP service errors (ServiceError hierarchy)
│ ├── auth/ — JWT create/verify + bcrypt + Fastify middleware
│ ├── fastify-core/ — Server bootstrap, request-id, health check, error handler
│ ├── config/ — Zod base env schema + product identity loader
│ ├── api-client/ — Typed fetch wrapper for dashboards/service clients
│ ├── react-auth/ — AuthProvider + useAuth() for Next.js dashboards
│ └── design-tokens/ — Canonical token JSON + generators for CSS/TS/Kotlin/Swift
├── docs/
│ ├── COMMON_PLATFORM_ANALYSIS.md (this file)
│ └── MIGRATION_GUIDE.md (how to adopt in each repo)
├── package.json — workspace root (npm/pnpm workspaces)
├── tsconfig.base.json — shared TypeScript config
└── README.md
```
---
## 6. Migration Priority
Ordered by **impact × ease**:
| Priority | Package | Impact | Effort | Reason |
|----------|---------|--------|--------|--------|
| **P0** | `@bytelyst/errors` | High | Low | Drop-in, no config, 6 consumers |
| **P0** | `@bytelyst/cosmos` | High | Low | 6 identical files, most-used utility |
| **P1** | `@bytelyst/config` | High | Medium | Eliminates 4 config files + 4 product-config files |
| **P1** | `@bytelyst/auth` | High | Medium | 5 copies of JWT logic, security-critical |
| **P1** | `@bytelyst/fastify-core` | High | Medium | 4 nearly identical server.ts files (~350 lines saved) |
| **P2** | `@bytelyst/api-client` | Medium | Low | 5+ fetch wrappers in dashboards |
| **P2** | `@bytelyst/react-auth` | Medium | Medium | 3 auth contexts, saves ~400 lines |
| **P3** | `@bytelyst/design-tokens` | Medium | High | Cross-platform token generation pipeline |
---
## 7. Consumption Model
### TypeScript packages (services + dashboards)
- Publish as **npm workspace packages** or use **git submodule** + `file:` references
- Each consuming repo adds `"@bytelyst/<pkg>": "file:../learning_ai_common_plat/packages/<pkg>"` to `package.json`
- Alternatively, publish to a private npm registry (GitHub Packages)
### Design tokens
- `design-tokens` package exports a CLI: `npx @bytelyst/design-tokens generate --format css,ts,kt,swift`
- Each consuming repo runs the generator as a build step or commits the generated output
### KMP (Kotlin Multiplatform)
- MindLyst's `MindLystTokens.kt` could be auto-generated from `design-tokens/tokens/*.json`
- Add a Gradle task in `shared/build.gradle.kts` that reads the JSON and emits Kotlin
---
## 8. Lines of Code Impact
| Category | Current Duplicated LOC | After Extraction |
|----------|----------------------|------------------|
| Cosmos client (6 files) | ~306 | ~50 (one shared) |
| Error classes (4 files) | ~157 | ~45 (one shared) |
| JWT auth (5 files) | ~250 | ~60 (one shared) |
| Server bootstrap (4 files) | ~351 | ~30 (per-service config only) |
| Config loader (4 files) | ~107 | ~10 (per-service extension only) |
| Product config (4 files) | ~35 | 0 (read from shared) |
| API fetch utils (5 files) | ~200 | ~30 (one shared) |
| Auth context (3 files) | ~450 | ~50 (one shared, configured per-app) |
| **Total** | **~1,856 LOC** | **~275 LOC** |
**Net savings: ~1,580 lines of duplicated code eliminated**, plus single-point maintenance for security-critical auth and database infrastructure.
---
## 9. Next Steps
1. Initialize `learning_ai_common_plat` as a pnpm/npm workspace monorepo
2. Start with P0 packages (`errors`, `cosmos`) — lowest risk, highest copy count
3. Add tests (port existing service tests that cover these utilities)
4. Update LysnrAI services to import from `@bytelyst/*` instead of local `./lib/*`
5. Wire MindLyst's Next.js web app to use shared design tokens
6. Set up CI to test the common platform packages independently