learning_ai_invt_trdg/docs/ROADMAP.md

# Investment Trading Monorepo Roadmap

## 1. Purpose

This roadmap is the execution tracker for `learning_ai_invt_trdg`, the new canonical monorepo for the trading product.

It assumes:

- `learning_ai_fastgap` is the structural reference monorepo
- `learning_ai_common_plat` is the shared ecosystem dependency source
- `bytelyst-trading-dashboard-web`, `bytelyst-trading-bot-service`, and `bytelyst-trading-dashboard-mob` are code-reference inputs, not the target architecture
- `learning_ai_invt_trdg` is a greenfield deployment with Cosmos DB as the only supported target persistence system

## 2. Status Model

### Overall status

- Current phase: `Phase 6`
- Overall state: `Done`

### Legend

- `[ ]` not started
- `[-]` in progress
- `[x]` done
- `[!]` blocked / decision needed

### Implementation snapshot as of `2026-04-05`

- [x] Monorepo foundation scaffolded with root workspace config, shared runtime, shared product identity, local package linking, and verification scripts
- [x] Backend migrated into `backend/` and passing typecheck, build, test, and backend verification gates
- [x] Web migrated into `web/` with shared runtime, shared kill-switch gate, shared telemetry bootstrap, normalized backend URL resolution, and common-platform-native session handling
- [x] Mobile migrated into `mobile/` with product identity, shared runtime bootstrap, launch-time kill-switch gate, platform-service auth, live backend polling plus websocket-backed updates, startup/error telemetry capture, secure session storage with invalidation handling, and explicit degraded/offline status surfacing
- [x] Backend now accepts common-platform JWTs and persists global trading-control state through Cosmos-backed control storage
- [x] Dynamic config now flows through backend control-plane APIs with Cosmos-backed storage
- [x] Dynamic config runtime loading and admin writes are now Cosmos-only in the new backend runtime path
- [x] Backend snapshots now use a Cosmos-backed repository
- [x] Distributed entry and reconciliation locks now use a Cosmos-backed repository
- [x] Capital ledger persistence now uses a Cosmos-backed repository
- [x] Mobile platform auth requests now use the common React Native platform SDK
- [x] Backend risk and PnL aggregate reads now flow through repository abstractions instead of direct legacy service calls
- [x] Web history, profile, marketplace, config, and manual-entry flows now run through backend APIs instead of browser-side table access
- [x] Release smoke coverage now exists for web auth and product accessibility flows, with a tracked mobile release smoke checklist in operations
- [x] Request ID propagation is now standardized across the main web/mobile API paths, operator actions, lifecycle fetches, and backend HTTP responses
- [x] Backtest feature access now reads from an explicit backend feature-flags contract instead of scraping generic runtime config
- [x] Feature-flag key ownership and response shape for backtest are now shared across backend and web through a common product contract
- [x] Trading user profiles and marketplace presets now have Cosmos-backed authority paths
- [x] Runtime order, trade-history, manual-entry, order-activity, and reconciliation-audit repositories now use Cosmos-backed trading-record storage instead of the legacy service layer
- [x] Runtime sub-tag repair now operates through the Cosmos-backed order repository
- [x] Root verification and lint flows now run successfully without sandbox-hostile script harness behavior
- [x] Web runtime no longer instantiates a live Supabase browser client; legacy import paths fail fast with an explicit unsupported message
- [x] Backend startup validation treats Cosmos as a first-class per-user credential store (alongside optional legacy Supabase) when `.env` broker keys are placeholders
- [x] Cosmos-first repositories no longer take an injected legacy service handle for users, profiles, presets, snapshots, capital ledger, or strategy presets; legacy Postgres reads/writes go through `legacySupabaseClient.ts`
- [x] `apiServer` no longer imports `SupabaseService`; admin-scoped HTTP behavior uses `isTradingAdmin` (platform JWT role first, then legacy user-store admin flag)
- [x] Shared order/reconciliation row types live in `tradingPersistenceTypes.ts`; trading user row shape lives in `tradingUserTypes.ts` (with `SupabaseService` re-exporting for compatibility)
- [x] Root-level `backend/*.ts` maintenance scripts: raw Postgres access uses `getLegacySupabaseClient()` where practical; `loadDynamicConfig()` calls no longer pass a dead `supabaseService` argument; scripts that must keep `supabaseService` (reconciliation helpers, monkey-patches, `subscribeToProfiles`, etc.) are listed in `docs/BACKEND_LEGACY_SUPABASE_SCRIPTS.md`
- [x] DRY cleanup is complete for runtime/config/bootstrap, websocket auth helpers, platform-session handling, request tracing, persistence type layering, legacy access surface, and the feature-flag contract (backtest + tab visibility); admin audit schema formalized in `docs/BACKEND_AUDIT_SCHEMA.md`; deprecated endpoint strategy in `docs/BACKEND_API_DEPRECATION.md`
- [x] Cosmos-only execution persistence is in place for main `backend/src` runtime paths; remaining direct `supabaseService` imports in one-off root scripts are intentional and documented (see `docs/BACKEND_LEGACY_SUPABASE_SCRIPTS.md`)

## 3. Guiding Rules

1. Build the target repo cleanly first.
2. Do not duplicate auth, kill switch, telemetry, or config bootstrap.
3. Do not move core trading moat into common-platform packages.
4. Do not preserve legacy repo boundaries inside the new monorepo if they block clarity.
5. Rebuild by contract, not by uncontrolled file copying.
6. Do not preserve migration-only fallback behavior in the new runtime.

## 4. Critical Path

- [x] Canonical product identity
- [x] Shared platform contract
- [x] Backend authority and contracts
- [x] Web migration
- [x] Mobile migration
- [x] Verification and cutover

## 5. Target Repository Shape

```text
learning_ai_invt_trdg/
├── mobile/                     # Expo mobile app
├── web/                        # Web dashboard
├── backend/                    # Trading backend
├── shared/
│   └── product.json
├── docs/
├── scripts/
├── pnpm-workspace.yaml
├── package.json
├── docker-compose.yml
└── .env.example
```

## 6. Dependency Matrix

### Upstream dependencies

- [x] `learning_ai_common_plat` package compatibility confirmed
- [x] platform-service auth behavior confirmed
- [x] platform-service kill-switch behavior confirmed
- [x] canonical product identity schema confirmed

### Internal dependencies

- [x] backend contracts available before web integration
- [x] backend contracts available before mobile integration
- [x] auth/session semantics finalized before surface wiring
- [x] kill-switch semantics finalized before release readiness

## 7. Legacy Repo Migration Map

### `bytelyst-trading-dashboard-web`

- Destination: `web/`
- Rule: preserve valuable product UI and domain-facing client logic
- Rule: replace custom auth/bootstrap and duplicated platform plumbing

### `bytelyst-trading-bot-service`

- Destination: `backend/`
- Rule: preserve domain and safety logic
- Rule: normalize config, auth boundaries, telemetry, and runtime control

### `bytelyst-trading-dashboard-mob`

- Destination: `mobile/`
- Rule: treat current mobile repo as design/input source, not final architecture

## 8. Phase Tracker

## Phase 0: Product Definition and Monorepo Foundation

### Status

- State: `[x] Done`
- Priority: `Critical`
- Depends on: none

### Objective

Create a credible target repository with clear product identity and explicit integration contracts before any code migration starts.

### Checklist

- [x] Create canonical product identity in `shared/product.json`
- [x] Define root `package.json`
- [x] Define `pnpm-workspace.yaml`
- [x] Define top-level scripts
- [x] Define root docs and local-dev model
- [x] Define environment model and `.env.example`
- [x] Define shared dependency strategy on local common-platform package links
- [x] Define repo conventions based on FastGap where useful
- [x] Define product IDs, ports, env prefixes, domains, and package naming
- [x] Decide mobile root folder convention
- [x] Define root workspace naming conventions
- [x] Define migration guardrails

### Deliverables

- [x] Root product metadata
- [x] Root package/workspace config
- [x] Initial repo skeleton
- [x] Environment contract
- [x] Migration design notes

### Exit Criteria

- [x] Repo structure approved
- [x] Product identity approved
- [x] Platform integration boundaries approved
- [x] Migration guardrails approved

## Phase 1: Shared Platform Contract Layer

### Status

- State: `[x] Done`
- Priority: `Critical`
- Depends on: `Phase 0`

### Objective

Ensure all surfaces adopt one consistent platform model for auth, kill switch, telemetry, flags, diagnostics, and config.

### Checklist

- [x] Define web auth pattern using `@bytelyst/react-auth`
- [x] Define mobile auth pattern using shared platform-service session contracts
- [x] Define backend auth boundary and middleware strategy
- [x] Define kill-switch semantics across web, mobile, and backend
- [x] Define ownership split between product accessibility controls and trading-behavior controls
- [x] Define telemetry envelope fields
- [x] Define correlation ID and request propagation strategy
- [x] Define feature flag ownership and evaluation model
- [x] Define system-of-record ownership by concern
- [x] Define degraded-platform behavior
- [x] Define compatibility boundaries only where required to preserve domain behavior during derivation

### Deliverables

- [x] Auth integration spec
- [x] Kill-switch behavior spec
- [x] Telemetry and diagnostics contract
- [x] Shared config conventions

### Exit Criteria

- [x] No ambiguity remains between platform-service and product-backend responsibilities
- [x] All three surfaces have approved integration patterns
- [x] Auth and kill-switch semantics are implementation-ready

## Phase 2: Backend First Migration

### Status

- State: `[x] Done`
- Priority: `Critical`
- Depends on: `Phase 1`

### Objective

Make backend the stable authority before web and mobile migrate heavily onto it.

### Checklist

- [x] Create `backend/` workspace
- [x] Define module layout under `backend/src`
- [x] Classify legacy backend modules as keep, refactor, or drop
- [x] Migrate core trading service modules selectively
- [x] Split generic lib concerns from trading-domain modules
- [x] Define typed API contracts for status, alerts, config, lifecycle, trade, admin control, and health
- [x] Define websocket auth model and namespaces (auth middleware + /trading and /admin named namespaces)
- [x] Define websocket scoping model
- [x] Normalize config loading and schema validation
- [x] Integrate platform-aware telemetry and diagnostics — `tradingTelemetry` singleton wired in `backend/src/services/tradingTelemetry.ts` (2026-04-07)
- [x] Integrate explicit kill-switch and maintenance semantics
- [x] Assign backend enforcement for global trade halt, tenant disable, and profile disable
- [x] Add runtime control endpoints
- [x] Add platform-JWT verification
- [x] Add Cosmos-backed global trading-control persistence
- [x] Move snapshots to Cosmos-backed repository flow
- [x] Move distributed runtime locks to Cosmos-backed repository flow
- [x] Move capital ledger persistence to Cosmos-backed repository flow
- [x] Standardize admin controls and audit logging
- [x] Define admin audit event schema (see docs/BACKEND_AUDIT_SCHEMA.md)
- [x] Define durable state ownership between memory, database, and exchange sync
- [x] Document deprecated endpoints and legacy compatibility strategy (see docs/BACKEND_API_DEPRECATION.md)
- [x] Add reconciliation and safety docs

### Keep Local

- [x] Trade execution
- [x] Strategy logic
- [x] Lifecycle and reconciliation
- [x] Capital ledger
- [x] Market and exchange integration

### Reuse from Common Platform

- [x] Auth middleware patterns
- [x] Config conventions
- [x] Telemetry infrastructure — backend singleton using `@bytelyst/telemetry-client` with Map-based Node.js storage adapter
- [x] Diagnostics patterns

### Exit Criteria

- [x] Backend has stable typed contracts
- [x] Auth is enforced consistently
- [x] Kill-switch and control semantics are defined and testable
- [x] Backend is ready to anchor web and mobile integration

## Phase 3: Web Dashboard Migration

### Status

- State: `[x] Done`
- Priority: `High`
- Depends on: `Phase 2`

### Objective

Move the web dashboard onto the new repo and onto shared platform bootstrap patterns.

### Checklist

- [x] Create `web/` workspace
- [x] Define app shell
- [x] Replace custom auth provider with shared auth pattern
- [x] Move public auth boundary to common-platform-native session handling
- [x] Define route guards and role-aware rendering
- [x] Move runtime config to common conventions
- [x] Define product config
- [x] Define API client and websocket client
- [x] Standardize websocket token propagation
- [x] Integrate maintenance and kill-switch UX states
- [x] Define shell-level maintenance and kill-switch behavior
- [x] Classify each current web tab as ship, defer, or redesign
- [x] Migrate UI modules by priority, not blindly
- [x] Gate unfinished tabs/features behind flags (backtest, marketplace, membership, signals, entries via feature-flags contract)
- [x] Define admin/operator routes and role-based controls
- [x] Normalize terminology, models, and UI behavior around backend authority
- [x] Remove legacy bootstrap duplication instead of porting it

### Priority Order

- [x] Auth shell and session restore
- [x] Overview, positions, history
- [x] Config and settings
- [x] Admin and runtime controls
- [x] Advanced tabs such as marketplace and backtesting

### Exit Criteria

- [x] Web is no longer dependent on legacy custom auth context
- [x] Web contracts align with new backend
- [x] Kill-switch and maintenance states are integrated
- [x] Web feels like one coherent product surface

## Phase 4: Mobile Rebuild

### Status

- State: `[x] Done`
- Priority: `High`
- Depends on: `Phase 2`

### Objective

Build mobile as a real ecosystem surface, not a mock UI shell.

### Checklist

- [x] Create Expo app structure following FastGap-style monorepo conventions
- [x] Add product config bootstrap
- [x] Integrate `@bytelyst/react-native-platform-sdk`
- [x] Implement auth flow and session restore
- [x] Define secure storage and session invalidation behavior
- [x] Implement launch-time kill-switch and maintenance handling
- [x] Add telemetry startup and error capture
- [x] Define initial mobile scope
- [x] Connect to backend and websocket/status contracts
- [ ] Add push-notification-ready architecture (deferred — no push provider selected for v1)
- [x] Define mobile action policy for monitor-first versus control-first flows
- [x] Define alert and incident UX
- [x] Define operator-safe interventions (limited to read-heavy screens; no destructive actions in v1)
- [x] Define offline and degraded-state behavior

### Mobile v1 Scope

- [x] Sign in / restore session
- [x] Portfolio overview
- [x] Alerts and critical incidents
- [x] Positions
- [x] Recent history
- [x] Settings and sign out
- [x] Live state refresh via websocket with polling fallback
- [x] Safe operator controls limited to explicitly approved actions
- [x] Maintain monitor-first, but not monitor-only scope

### Do Not Do in Mobile v1

- [x] Do not pursue full parity with advanced web configuration — honoured; mobile is monitor-first
- [x] Do not add highly complex strategy editing — honoured; strategies managed via web only
- [x] Do not add admin-only deep diagnostics UI unless clearly justified — honoured; no admin diagnostics in mobile v1

### Exit Criteria

- [x] Mobile is integrated with platform auth and kill switch
- [x] Mobile consumes the same product contracts as web
- [x] Mobile scope is honest and operationally safe

## Phase 5: Cross-Repo DRY Consolidation

### Status

- State: `[x] Done`
- Priority: `Medium`
- Depends on: `Phases 2-4`

### Objective

Remove duplicated implementation patterns exposed during derivation from the legacy repos.

### Checklist

- [x] Consolidate auth/session bootstrap
- [x] Consolidate product config resolution
- [x] Consolidate request headers and token propagation helpers
- [x] Consolidate telemetry boot and event fields
- [x] Consolidate kill-switch UX and service-state handling
- [x] Consolidate shared types for product contracts
- [x] Remove temporary derivation-only adapters that are no longer needed

### Guardrail

- [x] Only extract code reused by at least two surfaces or clearly generic across ByteLyst products

### Exit Criteria

- [x] No duplicate platform bootstrap flows remain
- [x] Common code lives in the right place with clear ownership
- [x] Extracted code respects the generic-versus-domain ownership rule

## Phase 6: Verification, Release Readiness, and Cutover

### Status

- State: `[x] Done`
- Priority: `Critical`
- Depends on: `Phases 2-5`

### Objective

Validate that the new monorepo is safer and more coherent than the legacy setup before full adoption.

### Checklist

- [x] Add root verify scripts
- [x] Add backend contract tests (verifyApiContract.ts)
- [x] Add web auth and kill-switch smoke tests
- [x] Add mobile launch/auth/kill-switch smoke coverage
- [x] Add docs for local dev, CI, Docker, and degraded-platform behaviors
- [x] Define cutover sequencing from legacy repos
- [x] Define rollback paths
- [x] Define release go/no-go checklist
- [x] Define post-cutover monitoring checks

### Exit Criteria

- [x] New monorepo is production-ready for staged adoption
- [x] Rollback and cutover are documented
- [x] Engineers and operators can run the new repo confidently

## 9. Detailed Work Breakdown

## 9.1 Root / Repository Tasks

- [x] Create root `package.json`
- [x] Create `pnpm-workspace.yaml`
- [x] Create `.env.example`
- [x] Create `shared/product.json`
- [x] Create `scripts/verify.sh` or equivalent
- [x] Create root README
- [x] Create docker/dev orchestration model (docker-compose.yml + docker-compose.dev.yml)
- [x] Define naming conventions and import boundaries (see docs/CONVENTIONS.md)

## 9.2 Backend Tasks

- [x] Define module layout under `backend/src`
- [x] Split generic lib concerns from trading-domain modules
- [x] Add typed request/response schemas
- [x] Add websocket session/auth model
- [x] Add websocket auth model and namespaces
- [x] Add runtime control endpoints
- [x] Add telemetry and health integration — `tradingTelemetry` singleton in `backend/src/services/tradingTelemetry.ts`; Map-based Node.js storage adapter; init in `main()` after Key Vault resolution; tracks server_start and fatal_error events
- [x] Add Cosmos-first repository layer for snapshots, distributed locks, and capital ledger persistence
- [x] Add reconciliation and safety docs
- [x] Define admin audit event schema (see docs/BACKEND_AUDIT_SCHEMA.md)

## 9.3 Web Tasks

- [x] Define app shell
- [x] Define auth bootstrap
- [x] Define product config
- [x] Define API client and websocket client
- [x] Port prioritized UI modules
- [x] Integrate admin/operator states and surface messaging
- [x] Define shell-level maintenance and kill-switch behavior

## 9.4 Mobile Tasks

- [x] Define Expo structure
- [x] Define navigation shell
- [x] Define auth bootstrap and secure storage
- [x] Define status polling/live update strategy
- [x] Define alert/incident UX
- [x] Define operator-safe interventions
- [x] Define offline and degraded-state behavior

## 10. Sequencing Recommendations

### Recommended Commit Order

- [x] Repo scaffold and product identity
- [x] Backend skeleton and config/auth contracts
- [x] Backend runtime control and health contracts
- [x] Web shell and auth migration
- [x] Web dashboard migration by tab priority
- [x] Mobile bootstrap and auth
- [x] Mobile overview/alerts/positions/history
- [x] DRY cleanup
- [x] Verification and cutover docs
- [x] Backend Cosmos-authoritative repository implementation for safety-critical persistence

### Recommended Rollout Order

- [x] Backend internal validation
- [-] Web internal adoption (see docs/CUTOVER_WEB.md)
- [ ] Mobile internal beta
- [ ] External / staged rollout

## 11. Definition of Done by Phase

- [x] Docs are updated
- [x] Contracts are explicit
- [x] Verification has been run or the remaining gap is documented
- [x] Ownership boundaries are clearer than before the phase started

## 12. Risks and Mitigations

### Risk: legacy assumptions leak into the new architecture

- [x] Mitigation: define contracts first
- [x] Mitigation: migrate by module and purpose
- [x] Mitigation: reject dead or duplicate bootstrap code

### Risk: auth model remains split between compatibility layers and the target platform-service session contract

- [x] Mitigation: preserve domain behavior while removing migration-only storage fallbacks
- [x] Mitigation: define the platform-service session model as the authoritative web/mobile auth contract
- [x] Mitigation: document transitional behavior explicitly

### Risk: repo-level verification stays red due to test-harness drift instead of product regressions

- [x] Mitigation: keep backend safety gates green while cutting over persistence
- [x] Mitigation: fix the web Vitest `window.localStorage` harness issue and keep root `pnpm verify` green while API contracts evolve

### Risk: kill switch becomes semantically overloaded

- [x] Mitigation: separate product maintenance mode from trade-halt control
- [x] Mitigation: separate product-level access disable from profile-level disable

### Risk: mobile becomes a weak afterthought

- [x] Mitigation: scope it honestly as monitor/intervene-first
- [x] Mitigation: wire full platform integration from day one

### Risk: over-extraction into common platform

- [x] Mitigation: keep trading logic local unless there is proven reuse

## 13. Cutover Strategy

### Recommended Cutover Approach

- [x] Build target contracts in the new repo
- [x] Validate backend behavior in isolation
- [-] Migrate internal web usage (see docs/CUTOVER_WEB.md)
- [ ] Release mobile in controlled beta
- [ ] Switch operational ownership only after monitoring and support confidence is established

### Avoid

- [x] Big-bang replacement — honoured; staged cutover approach adopted
- [x] Silent endpoint swaps — honoured; all API changes documented in docs/BACKEND_API_DEPRECATION.md
- [x] Prolonged dual-maintenance of business logic — honoured; legacy repos are reference-only

## 14. Decision Log

### Decision 1

- [x] Use a new monorepo instead of incremental retrofitting of three existing repos

Reason:

- lower long-term complexity
- cleaner product identity
- fewer transitional glue layers

### Decision 2

- [x] Use `learning_ai_fastgap` as structural reference, not as a cloning template

Reason:

- the surface layout pattern is proven
- the domain is different and must not inherit unrelated product assumptions

### Decision 3

- [x] Use `learning_ai_common_plat` as ecosystem backbone for generic platform concerns

Reason:

- avoids duplicated auth, kill switch, telemetry, config, and diagnostics work

### Decision 4

- [x] Treat mobile as monitor/intervene-first for initial scope

Reason:

- improves delivery realism
- reduces safety risk
- avoids false parity with web

### Decision 5

- [x] Assign product-level kill switch to platform-service and trading-behavior controls to the trading backend

Reason:

- keeps trading enforcement with the backend authority
- avoids split ownership for safety-critical controls

## 15. Roadmap Acceptance Criteria

- [x] Repo structure is unambiguous
- [x] Product identity is unambiguous
- [x] Platform integration boundaries are unambiguous
- [x] Backend authority model is unambiguous
- [x] Migration sequence is unambiguous
- [x] DRY extraction rules are unambiguous
- [x] Risk controls are explicit

## 16. Immediate Next Steps

- [x] Finish profile risk/PnL aggregate repository migration off legacy Supabase
- [x] Finish remaining web direct legacy data-table reads and writes behind backend APIs
- [x] Replace remaining transitional web auth compatibility surfaces with fully common-platform-native session handling
- [x] Add release smoke coverage for web auth/kill-switch and mobile auth/kill-switch flows
- [x] Audit root-level `backend/*.ts` scripts: migrate `getClient`-only usage to `legacySupabaseClient` where practical, or document intentional direct `supabaseService` use (see `docs/BACKEND_LEGACY_SUPABASE_SCRIPTS.md`, 2026-04-05)
- [x] Phase 2 backlog resolved (2026-04-07): module layout confirmed, legacy classification done, typed contracts in place, websocket namespaces added, admin audit schema in docs/BACKEND_AUDIT_SCHEMA.md, deprecated-endpoint doc in docs/BACKEND_API_DEPRECATION.md
- [x] Backend telemetry wired (2026-04-07): `tradingTelemetry` singleton, Map-based Node.js storage adapter, `useTabFeatureFlags.dom.test.tsx` added to web smoke suite
- [ ] Remaining open: mobile push notifications (deferred — blocked on push provider selection)
- [x] Cosmos audit-events container: repository + GET /api/admin/audit implemented; activate by creating the container in Azure
- [-] Cutover in progress: web internal adoption (see docs/CUTOVER_WEB.md)