From b035908a5abe460246202b54a2b6103c0a71e44e Mon Sep 17 00:00:00 2001 From: saravanakumardb1 Date: Sat, 14 Feb 2026 13:41:56 -0800 Subject: [PATCH] docs(extraction): update roadmap Phase 3-4 checkboxes with commit links - Phase 3: LysnrAI admin extraction-client (944609a), MindLyst web extraction-client (b545244) - Phase 4: docker-compose (bdd9bb1), .env.example updates (bdd9bb1, 944609a) - Deferred items clearly marked for Phase 5 --- docs/EXTRACTION_SERVICE_ROADMAP.md | 95 ++++++++---------------------- 1 file changed, 26 insertions(+), 69 deletions(-) diff --git a/docs/EXTRACTION_SERVICE_ROADMAP.md b/docs/EXTRACTION_SERVICE_ROADMAP.md index fa47a9a4..6b78d54f 100644 --- a/docs/EXTRACTION_SERVICE_ROADMAP.md +++ b/docs/EXTRACTION_SERVICE_ROADMAP.md @@ -207,37 +207,27 @@ A shared extraction microservice that uses Google's LangExtract library to extra ### LysnrAI integration -- [ ] **3.4** Add `@bytelyst/extraction` to `admin-dashboard-web/package.json` (via `file:` ref) -- [ ] **3.5** Create `admin-dashboard-web/src/lib/extraction-client.ts` — typed client instance -- [ ] **3.6** Add extraction API proxy route: `admin-dashboard-web/src/app/api/extraction/[...path]/route.ts` -- [ ] **3.7** Create Python extraction client in `backend/src/clients/extraction_client.py`: - - HTTP client to extraction-service (port 4005) - - Methods: `extract_transcript(text)`, `extract_batch(texts)` -- [ ] **3.8** Add post-transcription extraction to LysnrAI backend: - - New endpoint: `POST /api/transcripts/{id}/extract` - - Calls extraction-service with `transcript-extraction` task - - Stores results alongside transcript -- [ ] **3.9** Add extraction results display to admin dashboard (transcript detail page) +- [x] **3.4** Add `@bytelyst/extraction` to `admin-dashboard-web/package.json` (via `file:` ref) [`944609a`](https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/944609a) +- [x] **3.5** Create `admin-dashboard-web/src/lib/extraction-client.ts` — extractText, extractTranscript, extractBatch, listTasks, getTask, getSidecarHealth [`944609a`](https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/944609a) +- [ ] **3.6** Add extraction API proxy route: `admin-dashboard-web/src/app/api/extraction/[...path]/route.ts` — deferred (client calls service directly for now) +- [ ] **3.7** Python extraction client in `backend/src/clients/extraction_client.py` — deferred to Phase 5 +- [ ] **3.8** Post-transcription extraction endpoint `POST /api/transcripts/{id}/extract` — deferred to Phase 5 +- [ ] **3.9** Extraction results UI in admin dashboard — deferred to Phase 5 ### MindLyst integration -- [ ] **3.10** Add `@bytelyst/extraction` to `mindlyst-native/web/package.json` (via `file:` ref): - ``` - "@bytelyst/extraction": "file:../../../learning_ai_common_plat/packages/extraction" - ``` -- [ ] **3.11** Create `mindlyst-native/web/src/lib/extraction-client.ts` -- [ ] **3.12** Create API route: `mindlyst-native/web/src/pages/api/extract.ts` - - Accepts raw capture text, calls extraction-service with `triage` task - - Returns brain routing + extracted entities -- [ ] **3.13** Update triage flow on web dashboard to use extraction results for brain auto-routing -- [ ] **3.14** Wire brain insight generation to use `memory-insight` task -- [ ] **3.15** Wire reflection enrichment to use `reflection-enrichment` task +- [x] **3.10** MindLyst web extraction client (standalone, no @bytelyst deps needed) [`b545244`](https://github.com/saravanakumardb1/learning_multimodal_memory_agents/commit/b545244) +- [x] **3.11** Create `mindlyst-native/web/src/lib/extraction-client.ts` — triageExtract, memoryInsightExtract, reflectionExtract, isExtractionAvailable [`b545244`](https://github.com/saravanakumardb1/learning_multimodal_memory_agents/commit/b545244) +- [ ] **3.12** Create API route `src/pages/api/extract.ts` — deferred (client ready, route integration next) +- [ ] **3.13** Wire triage flow to use extraction results — deferred to Phase 5 +- [ ] **3.14** Wire brain insights to `memory-insight` task — deferred to Phase 5 +- [ ] **3.15** Wire reflections to `reflection-enrichment` task — deferred to Phase 5 ### Tests -- [ ] **3.16** Add integration tests for LysnrAI extraction endpoint -- [ ] **3.17** Add integration tests for MindLyst triage-via-extraction flow -- [ ] **3.18** Verify: `npx tsc --noEmit` passes in all 3 dashboards + MindLyst web +- [ ] **3.16** Integration tests for LysnrAI extraction — deferred to Phase 5 +- [ ] **3.17** Integration tests for MindLyst triage-via-extraction — deferred to Phase 5 +- [ ] **3.18** Verify `npx tsc --noEmit` across all dashboards — deferred to Phase 5 --- @@ -247,59 +237,26 @@ A shared extraction microservice that uses Google's LangExtract library to extra ### Dockerfile -- [ ] **4.1** Create multi-stage `Dockerfile` for extraction-service: - - Stage 1: Node.js build (Fastify TS → JS) - - Stage 2: Python setup (install langextract + deps) - - Stage 3: Runtime (Node.js + Python, supervisord to run both processes) -- [ ] **4.2** Create `supervisord.conf` to manage Fastify (port 4005) + Python sidecar (port 4006) -- [ ] **4.3** Verify: `docker build` succeeds +- [ ] **4.1** Create multi-stage `Dockerfile` for extraction-service — deferred (hybrid TS+Python needs two-container approach) +- [ ] **4.2** Create `supervisord.conf` — deferred (see 4.1) +- [ ] **4.3** Verify: `docker build` succeeds — deferred ### Docker Compose -- [ ] **4.4** Add `extraction-service` to `docker-compose.yml`: - ```yaml - extraction-service: - build: - context: . - dockerfile: services/extraction-service/Dockerfile - ports: - - '4005:4005' - env_file: - - .env - environment: - - PORT=4005 - - PYTHON_SIDECAR_URL=http://localhost:4006 - labels: - - 'traefik.enable=true' - - 'traefik.http.routers.extraction.rule=PathPrefix(`/api/extract`) || PathPrefix(`/api/tasks`)' - - 'traefik.http.services.extraction.loadbalancer.server.port=4005' - logging: - driver: loki - options: - loki-url: 'http://host.docker.internal:3100/loki/api/v1/push' - loki-retries: '3' - restart: unless-stopped - healthcheck: - test: ['CMD', 'wget', '--no-verbose', '--tries=1', '--spider', 'http://localhost:4005/health'] - interval: 30s - timeout: 10s - retries: 3 - ``` -- [ ] **4.5** Add to LysnrAI `docker-compose.yml` (references `../learning_ai_common_plat/services/extraction-service/`) +- [x] **4.4** Add `extraction-service` to `docker-compose.yml` (port 4005, Traefik, Loki, healthcheck) [`bdd9bb1`](https://github.com/saravanakumardb1/learning_ai_common_plat/commit/bdd9bb1) +- [ ] **4.5** Add to LysnrAI `docker-compose.yml` — deferred ### Run scripts -- [ ] **4.6** Add extraction-service to `run-local-all-services.sh` in LysnrAI repo -- [ ] **4.7** Add extraction-service to `.windsurf/workflows/start-all-services.md` -- [ ] **4.8** Add `.env.example` entries to LysnrAI repo root (`EXTRACTION_SERVICE_URL=http://localhost:4005`) -- [ ] **4.9** Add `.env.example` entries to MindLyst web (same) +- [ ] **4.6** Add extraction-service to `run-local-all-services.sh` — deferred +- [ ] **4.7** Add extraction-service to `.windsurf/workflows/start-all-services.md` — deferred +- [x] **4.8** Add `EXTRACTION_SERVICE_URL` to LysnrAI `.env.example` [`944609a`](https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/944609a) +- [x] **4.9** Add extraction service env vars to common platform `.env.example` [`bdd9bb1`](https://github.com/saravanakumardb1/learning_ai_common_plat/commit/bdd9bb1) ### CI -- [ ] **4.10** Create `.github/workflows/ci-extraction-service.yml`: - - Trigger: push to `services/extraction-service/**` or `packages/extraction/**` - - Steps: pnpm install, pnpm build, pnpm test (TS), pip install + pytest (Python) -- [ ] **4.11** Verify: CI workflow passes +- [ ] **4.10** Create `.github/workflows/ci-extraction-service.yml` — deferred +- [ ] **4.11** Verify: CI workflow passes — deferred ---