From e4b55a042436427c0643bd539dbd028e538614a2 Mon Sep 17 00:00:00 2001 From: saravanakumardb1 Date: Sat, 14 Feb 2026 21:04:28 -0800 Subject: [PATCH] docs: add service consolidation roadmap, archive auth service design MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - SERVICE_CONSOLIDATION_ROADMAP.md: plan to merge billing+growth+tracker into platform-service (5→2 services) - AUTH_SERVICE_DESIGN.md: renamed with OUTDATED_CANCELED suffix, auth handled by platform-service --- ...AUTH_SERVICE_DESIGN__OUTDATED_CANCELED.md} | 8 +- .../WINDSURF/SERVICE_CONSOLIDATION_ROADMAP.md | 328 ++++++++++++++++++ 2 files changed, 334 insertions(+), 2 deletions(-) rename docs/WINDSURF/{AUTH_SERVICE_DESIGN.md => AUTH_SERVICE_DESIGN__OUTDATED_CANCELED.md} (93%) create mode 100644 docs/WINDSURF/SERVICE_CONSOLIDATION_ROADMAP.md diff --git a/docs/WINDSURF/AUTH_SERVICE_DESIGN.md b/docs/WINDSURF/AUTH_SERVICE_DESIGN__OUTDATED_CANCELED.md similarity index 93% rename from docs/WINDSURF/AUTH_SERVICE_DESIGN.md rename to docs/WINDSURF/AUTH_SERVICE_DESIGN__OUTDATED_CANCELED.md index 24b1feb2..95d5fc23 100644 --- a/docs/WINDSURF/AUTH_SERVICE_DESIGN.md +++ b/docs/WINDSURF/AUTH_SERVICE_DESIGN__OUTDATED_CANCELED.md @@ -1,6 +1,10 @@ -# Auth Service Design (WindSurf) +# ⛔ OUTDATED & CANCELED — Auth Service Design (WindSurf) -## Goal +> **Status:** CANCELED — Auth is handled by `platform-service/src/modules/auth/` and `@bytelyst/auth` package. +> This doc predates the current architecture. See `SERVICE_CONSOLIDATION_ROADMAP.md` for the current plan. +> **Canceled:** 2026-02-14 + +## Goal (Original — No Longer Applicable) Create a shared authentication microservice under `learning_ai_common_plat/services/auth-service` so all BytelystAI apps (MindLyst, LysnrAI, Windhawk extensions, etc.) rely on one canonical JWT-based auth + licensing stack rather than duplicating logic in each client. This service will become the source of truth for user identity, subscriptions, JWT minting/refreshing, and Azure resource guardianship. diff --git a/docs/WINDSURF/SERVICE_CONSOLIDATION_ROADMAP.md b/docs/WINDSURF/SERVICE_CONSOLIDATION_ROADMAP.md new file mode 100644 index 00000000..563429cc --- /dev/null +++ b/docs/WINDSURF/SERVICE_CONSOLIDATION_ROADMAP.md @@ -0,0 +1,328 @@ +# Service Consolidation Roadmap — 5 Services → 2 + +> **Goal:** Merge `billing-service`, `growth-service`, and `tracker-service` into `platform-service` so we have one unified Fastify service for all common platform concerns. `extraction-service` stays separate (Python sidecar). +> +> **Created:** 2026-02-14 +> **Estimated effort:** 3–4 days +> **Blocked by:** Nothing — can start immediately + +--- + +## Why Consolidate + +| Problem | Impact | +|---------|--------| +| 5 separate Node processes for 2 products | Unnecessary operational overhead | +| 5 ports to manage (4001–4005) | Complex docker-compose, run scripts, env files | +| 5 separate Cosmos connections | Wasted connection pool resources | +| 5 CI pipelines | Slow feedback, more config to maintain | +| 5 config schemas with duplicate env vars | Inconsistent config, easy to miss vars | + +**After consolidation:** 2 services — `platform-service` (port 4003) + `extraction-service` (port 4005) + +--- + +## Current State + +``` +services/ + ├── platform-service/ (port 4003) — 6 modules, ~55 tests + │ auth, audit, notifications, flags, ratelimit, blob + │ + ├── billing-service/ (port 4002) — 5 modules, ~11 tests + │ subscriptions, usage, plans, licenses, stripe + │ + ├── growth-service/ (port 4001) — 3 modules, ~14 tests + │ invitations, referrals, promos + │ + ├── tracker-service/ (port 4004) — 4 modules, ~45 tests + │ items, comments, votes, public + │ + └── extraction-service/ (port 4005) — stays separate (Python sidecar) +``` + +## Target State + +``` +services/ + ├── platform-service/ (port 4003) — 18 modules, ~125+ tests + │ ── existing ── + │ auth, audit, notifications, flags, ratelimit, blob + │ ── from billing ── + │ subscriptions, usage, plans, licenses, stripe + │ ── from growth ── + │ invitations, referrals, promos + │ ── from tracker ── + │ items, comments, votes, public + │ + └── extraction-service/ (port 4005) — unchanged +``` + +--- + +## Cosmos Containers (Unified) + +All containers served by one Cosmos client in platform-service: + +| Origin | Containers | +|--------|-----------| +| **platform** (existing) | `users`, `audit_log`, `feature_flags`, `notification_devices`, `notification_prefs` | +| **billing** → platform | `subscriptions`, `payments`, `plans`, `licenses`, `usage_daily` | +| **growth** → platform | `invitation_codes`, `referrals`, `promo_codes` | +| **tracker** → platform | `tracker_items`, `tracker_comments`, `tracker_votes` | + +--- + +## Phase 0 — Preparation + +> **Goal:** Backup, verify tests pass before any changes. + +- [ ] **0.1** Backup all 3 repos via `/repo_backup-main-branch` +- [ ] **0.2** Verify all services build: `pnpm build` +- [ ] **0.3** Verify all tests pass: `pnpm test` +- [ ] **0.4** Document current test counts per service (baseline) +- [ ] **0.5** Create `consolidation` branch (or work on main with incremental commits) + +--- + +## Phase 1 — Merge Growth Service (Smallest First) + +> **Goal:** Move invitations, referrals, promos modules into platform-service. Remove growth-service. + +### 1.1 Copy modules + +- [ ] **1.1.1** Copy `growth-service/src/modules/invitations/` → `platform-service/src/modules/invitations/` +- [ ] **1.1.2** Copy `growth-service/src/modules/referrals/` → `platform-service/src/modules/referrals/` +- [ ] **1.1.3** Copy `growth-service/src/modules/promos/` → `platform-service/src/modules/promos/` + +### 1.2 Fix imports + +- [ ] **1.2.1** Update import paths in copied modules: `../../lib/errors.js`, `../../lib/product-config.js`, `../../lib/cosmos.js` +- [ ] **1.2.2** Copy `growth-service/src/lib/webhooks.ts` → `platform-service/src/lib/webhooks.ts` (used by invitation redeem) +- [ ] **1.2.3** Verify no growth-specific lib files are missed + +### 1.3 Register routes + +- [ ] **1.3.1** Add imports to `platform-service/src/server.ts`: `invitationRoutes`, `referralRoutes`, `promoRoutes` +- [ ] **1.3.2** Register routes with `/api` prefix (same as growth-service) + +### 1.4 Merge config + +- [ ] **1.4.1** Add any growth-specific env vars to `platform-service/src/lib/config.ts` (check for `WEBHOOK_URL`, etc.) +- [ ] **1.4.2** Add growth Cosmos containers to platform-service container registry + +### 1.5 Copy tests + +- [ ] **1.5.1** Copy `growth-service/src/modules/invitations/invitations.test.ts` → platform-service +- [ ] **1.5.2** Copy any other test files from growth-service modules +- [ ] **1.5.3** Fix test import paths + +### 1.6 Verify + +- [ ] **1.6.1** `pnpm --filter @lysnrai/platform-service build` — clean +- [ ] **1.6.2** `pnpm --filter @lysnrai/platform-service test` — all tests pass (existing + growth) +- [ ] **1.6.3** Remove `services/growth-service/` directory +- [ ] **1.6.4** Commit: `refactor: merge growth-service into platform-service` + +--- + +## Phase 2 — Merge Billing Service + +> **Goal:** Move subscriptions, usage, plans, licenses, stripe modules into platform-service. Remove billing-service. + +### 2.1 Copy modules + +- [ ] **2.1.1** Copy `billing-service/src/modules/subscriptions/` → `platform-service/src/modules/subscriptions/` +- [ ] **2.1.2** Copy `billing-service/src/modules/usage/` → `platform-service/src/modules/usage/` +- [ ] **2.1.3** Copy `billing-service/src/modules/plans/` → `platform-service/src/modules/plans/` +- [ ] **2.1.4** Copy `billing-service/src/modules/licenses/` → `platform-service/src/modules/licenses/` +- [ ] **2.1.5** Copy `billing-service/src/modules/stripe/` → `platform-service/src/modules/stripe/` + +### 2.2 Fix imports + +- [ ] **2.2.1** Update all import paths in copied modules +- [ ] **2.2.2** Handle billing-specific internal key auth — convert `BILLING_INTERNAL_KEY` hook to route-level guard on billing endpoints (instead of global onRequest hook) + +### 2.3 Merge config + +- [ ] **2.3.1** Add billing env vars to `platform-service/src/lib/config.ts`: + - `STRIPE_SECRET_KEY`, `STRIPE_WEBHOOK_SECRET`, `STRIPE_PRICE_PRO`, `STRIPE_PRICE_ENTERPRISE` + - `BILLING_INTERNAL_KEY`, `BACKEND_URL`, `PLAN_LIMITS_JSON`, `USAGE_WARN_THRESHOLD` +- [ ] **2.3.2** Add billing Cosmos containers to platform-service container registry +- [ ] **2.3.3** Add `stripe` to platform-service `package.json` dependencies + +### 2.4 Register routes + +- [ ] **2.4.1** Add imports to `platform-service/src/server.ts` +- [ ] **2.4.2** Register: `subscriptionRoutes`, `usageRoutes`, `planRoutes`, `licenseRoutes`, `stripeRoutes` + +### 2.5 Copy tests + +- [ ] **2.5.1** Copy all billing test files to platform-service +- [ ] **2.5.2** Fix test import paths + +### 2.6 Verify + +- [ ] **2.6.1** `pnpm --filter @lysnrai/platform-service build` — clean +- [ ] **2.6.2** `pnpm --filter @lysnrai/platform-service test` — all tests pass +- [ ] **2.6.3** Remove `services/billing-service/` directory +- [ ] **2.6.4** Commit: `refactor: merge billing-service into platform-service` + +--- + +## Phase 3 — Merge Tracker Service + +> **Goal:** Move items, comments, votes, public modules into platform-service. Remove tracker-service. + +### 3.1 Copy modules + +- [ ] **3.1.1** Copy `tracker-service/src/modules/items/` → `platform-service/src/modules/items/` +- [ ] **3.1.2** Copy `tracker-service/src/modules/comments/` → `platform-service/src/modules/comments/` +- [ ] **3.1.3** Copy `tracker-service/src/modules/votes/` → `platform-service/src/modules/votes/` +- [ ] **3.1.4** Copy `tracker-service/src/modules/public/` → `platform-service/src/modules/public/` + +### 3.2 Fix imports + +- [ ] **3.2.1** Update all import paths in copied modules +- [ ] **3.2.2** Copy `tracker-service/src/lib/auth.ts` → merge into platform-service's existing auth helper (both use `extractAuth` from `@bytelyst/auth`) + +### 3.3 Merge config + +- [ ] **3.3.1** No new env vars needed (tracker uses same Cosmos + JWT vars as platform) +- [ ] **3.3.2** Add tracker Cosmos containers to container registry + +### 3.4 Register routes + +- [ ] **3.4.1** Add imports to `platform-service/src/server.ts` +- [ ] **3.4.2** Register: `itemRoutes`, `commentRoutes`, `voteRoutes`, `publicRoutes` +- [ ] **3.4.3** Ensure public routes skip auth (they already do internally — verify) + +### 3.5 Copy tests + +- [ ] **3.5.1** Copy all tracker test files to platform-service +- [ ] **3.5.2** Fix test import paths +- [ ] **3.5.3** Verify: 45+ tracker tests pass + +### 3.6 Verify + +- [ ] **3.6.1** `pnpm --filter @lysnrai/platform-service build` — clean +- [ ] **3.6.2** `pnpm --filter @lysnrai/platform-service test` — all 125+ tests pass +- [ ] **3.6.3** Remove `services/tracker-service/` directory +- [ ] **3.6.4** Commit: `refactor: merge tracker-service into platform-service` + +--- + +## Phase 4 — Update Consumers + +> **Goal:** Update all dashboards, scripts, configs, and docker files that reference the old services. + +### 4.1 Dashboard API clients + +- [ ] **4.1.1** `admin-dashboard-web/src/lib/billing-client.ts` — change `BILLING_API_URL` → `PLATFORM_API_URL` (port 4003) +- [ ] **4.1.2** `admin-dashboard-web/src/lib/growth-client.ts` — change `GROWTH_API_URL` → `PLATFORM_API_URL` +- [ ] **4.1.3** `user-dashboard-web/src/lib/billing-client.ts` — same change +- [ ] **4.1.4** `tracker-dashboard-web/src/lib/tracker-client.ts` — change `TRACKER_API_URL` → `PLATFORM_API_URL` +- [ ] **4.1.5** Update all dashboard `env.local.example` files — remove old service URLs, add note that everything goes through `PLATFORM_API_URL` + +### 4.2 Docker Compose + +- [ ] **4.2.1** `learning_ai_common_plat/docker-compose.yml` — remove billing, growth, tracker service entries; keep platform + extraction +- [ ] **4.2.2** `learning_voice_ai_agent/docker-compose.yml` — same cleanup +- [ ] **4.2.3** Update Traefik labels (all routes go to platform-service) + +### 4.3 Run scripts + +- [ ] **4.3.1** `learning_voice_ai_agent/run-local-all-services.sh` — remove billing/growth/tracker start commands; update health checks +- [ ] **4.3.2** `.windsurf/workflows/start-all-services.md` — update to reflect 2 services + +### 4.4 Environment files + +- [ ] **4.4.1** `learning_ai_common_plat/.env.example` — consolidate all env vars into one section for platform-service +- [ ] **4.4.2** Remove references to ports 4001, 4002, 4004 + +### 4.5 CI + +- [ ] **4.5.1** `.github/workflows/ci.yml.disabled` — remove billing/growth/tracker from matrix; only platform + extraction +- [ ] **4.5.2** Delete individual CI workflows: `ci-billing-service.yml`, `ci-growth-service.yml`, `ci-tracker-service.yml` (if they exist) + +### 4.6 Verify consumers + +- [ ] **4.6.1** `npx tsc --noEmit` in admin-dashboard-web +- [ ] **4.6.2** `npx tsc --noEmit` in user-dashboard-web +- [ ] **4.6.3** `npx tsc --noEmit` in tracker-dashboard-web +- [ ] **4.6.4** Commit: `refactor: update consumers for consolidated platform-service` + +--- + +## Phase 5 — Documentation & Cleanup + +> **Goal:** Update all docs, AGENTS.md, and clean up dead references. + +### 5.1 Documentation + +- [ ] **5.1.1** Update `AGENTS.md` in all 3 repos — new service layout (2 services, not 5) +- [ ] **5.1.2** Update `docs/ECOSYSTEM_ARCHITECTURE.md` — consolidated architecture diagram +- [ ] **5.1.3** Update `docs/MIGRATION_GUIDE.md` — single service URL for all API calls +- [ ] **5.1.4** Update `docs/ROADMAP.md` — add consolidation as completed item + +### 5.2 Package cleanup + +- [ ] **5.2.1** Remove `services/billing-service/` from git (already deleted in Phase 2) +- [ ] **5.2.2** Remove `services/growth-service/` from git (already deleted in Phase 1) +- [ ] **5.2.3** Remove `services/tracker-service/` from git (already deleted in Phase 3) +- [ ] **5.2.4** Run `pnpm install` — workspace resolution updated +- [ ] **5.2.5** Verify `pnpm build` passes across entire workspace + +### 5.3 Platform-service cleanup + +- [ ] **5.3.1** Update `platform-service/package.json` — description, add any missing deps +- [ ] **5.3.2** Update `platform-service/src/server.ts` — description comment +- [ ] **5.3.3** Swagger description reflects all 18 modules + +### 5.4 Final verification + +- [ ] **5.4.1** `pnpm build` — all packages + platform-service + extraction-service build +- [ ] **5.4.2** `pnpm test` — all 125+ tests pass +- [ ] **5.4.3** `pnpm typecheck` — clean across workspace +- [ ] **5.4.4** Commit: `docs: update all documentation for consolidated platform-service` + +--- + +## Summary + +| Phase | What | Effort | Tests Moved | +|-------|------|--------|-------------| +| **0** | Preparation & backup | 30 min | — | +| **1** | Merge growth-service (3 modules) | 2–3 hrs | ~14 | +| **2** | Merge billing-service (5 modules) | 3–4 hrs | ~11 | +| **3** | Merge tracker-service (4 modules) | 3–4 hrs | ~45 | +| **4** | Update consumers (dashboards, docker, scripts, CI) | 2–3 hrs | — | +| **5** | Documentation & cleanup | 1–2 hrs | — | +| **Total** | **5 services → 2** | **~3 days** | **~125+ tests** | + +## Port Allocation (After) + +| Service | Port | +|---------|------| +| **platform-service** | **4003** | +| **extraction-service** | **4005** | +| extraction-service python sidecar (internal) | 4006 | + +Ports 4001, 4002, 4004 freed up. + +## Rollback Strategy + +Each phase has its own commit. If a phase breaks something: +1. `git revert ` to undo that phase +2. The old service code is in git history +3. Backup branches created in Phase 0 + +## Risks & Mitigations + +| Risk | Mitigation | +|------|-----------| +| Route path collisions between services | All services already use unique prefixes (`/auth/*`, `/subscriptions/*`, `/items/*`, etc.) | +| Config schema gets large | Group env vars by domain in config.ts with clear comments | +| Stripe webhook handler needs raw body | Fastify already handles this — just verify after move | +| Billing internal key auth | Convert from global hook to route-level guard on billing endpoints only | +| Public tracker routes skip auth | They already handle auth internally — just verify |