bytelyst/learning_ai_common_plat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(roadmap): add Phase 4 DRY audit + fix 9 review gaps in Local AI consolidation roadmap

Browse Source 
Phase 4 — DRY Refactoring & Common Platform Extractions:
- 4.1 @bytelyst/ollama-client (NEW) — deduplicate Ollama API + NDJSON parsing (~350 lines)
- 4.2 Extend @bytelyst/fastify-sse with per-request SSE helpers (~26 lines)
- 4.3 @bytelyst/use-theme (NEW) — deduplicate across 6 web apps (~300 lines)
- 4.4 @bytelyst/use-keyboard-shortcuts (NEW) — deduplicate across 5 web apps (~250 lines)
- 4.5-4.7 Types, format utils, client-side stream parsers (~220 lines)
Total: ~1,150 lines of duplicated code identified for extraction

Review pass 2 fixes (R1-R9):
- R1: Fix misleading route group item count heading
- R2: Fix file inventory 6 pages+layout → 5 pages+1 layout
- R3: Fix LocalMemGPT test count 102 → 104 (field-encrypt)
- R4: Fix Phase 4.2 title to clarify extending existing package
- R5: Confirm ActionTrail use-theme.ts (was inferred)
- R6: Note post-Phase-1 stream parser copy reduction
- R7: Add addendum note for Phase 4 structure
- R8: Clarify both Attachment+ModelDefaults must be kept
- R9: Fix Mission Control API route count 11 → 13
This commit is contained in:
saravanakumardb1 2026-03-29 11:38:34 -07:00
parent 9e53a5cefa
commit 808f405889
1 changed files with 752 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

752
docs/roadmaps/not-started/LOCAL_AI_CONSOLIDATION_ROADMAP.md Normal file
View File

@ -0,0 +1,752 @@
# Local AI Repos — Consolidation Roadmap
> **Status:** Not Started
> **Created:** 2026-03-29
> **Scope:** `learning_ai_local_llms` + `learning_ai_local_memory_gpt`
> **Goal:** Eliminate feature overlap between the two repos by removing the chat workspace from Local LLM Lab, making Local Memory GPT the single chat surface.
---
## Executive Summary
Two repos in the ByteLyst ecosystem handle local AI inference:
| Repo | Product ID | Purpose |
| ------------------------------ | ------------- | ----------------------------------------------------------------------- |
| `learning_ai_local_llms` | `localllmlab` | Model management, hardware monitoring, benchmarks, STT/TTS, experiments |
| `learning_ai_local_memory_gpt` | `localmemgpt` | Persistent AI chat with RAG, full-text search, export/import |
Both repos have a **chat workspace** that talks to Ollama, stores conversations, and supports multi-model comparison. This duplication creates maintenance burden and user confusion about which surface to use for conversations.
### Decision
- **Local LLM Lab** keeps: Mission Control (model ops, hardware, benchmarks), STT/TTS, experiments
- **Local LLM Lab** removes: Chat workspace (conversations, agents, quick actions, orchestrations, projects, scheduled tasks, compare)
- **Local Memory GPT** stays as-is — it is already the better chat app (server-side persistence, RAG, FTS5, encryption)
- **Optional:** Port unique LLM Lab workspace features (agents, quick actions, orchestrations) into Local Memory GPT if they prove valuable
---
## Review Findings
> **Pass 1:** 13 bugs/gaps identified during systematic codebase verification (2026-03-29).
> All findings are incorporated inline below — see `[F1]`–`[F13]` markers.
>
> **Pass 2:** 9 additional issues found during Phase 4 review (2026-03-29).
> See `[R1]`–`[R9]` markers below.
| # | Severity | Finding | Section Fixed |
| --- | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- |
| F1 | **CRITICAL** | `api/ollama/chat`, `api/ollama/stream`, `api/ollama/logs` are used by Mission Control (46 whisper/tts/chat references in `page.tsx`) — cannot delete them | §1.2 |
| F2 | **CRITICAL** | `api/tts/` and `api/whisper/` are used by Mission Control (Whisper test, TTS engine status panel) — must keep, not "decision point" | §1.2 |
| F3 | **CRITICAL** | `(workspace)/page.tsx` is the **root page** (`/`) — no separate `src/app/page.tsx` exists. Deleting the workspace route group removes the root page; a redirect must be added | §1.5 |
| F4 | **HIGH** | `router.ts` (task classification) is imported by `api/ollama/chat/route.ts` which Mission Control calls — it is SHARED, not workspace-only. Was listed under "Lib files to KEEP" (correct) but also categorized workspace-only in the API routes section | §1.2 |
| F5 | **HIGH** | `api/ollama/title/route.ts` is the ONLY chat API route that is workspace-only (auto-generates conversation titles). The other 3 (`chat`, `stream`, `logs`) must stay | §1.2 |
| F6 | **MEDIUM** | E2E `chat.spec.ts` — workspace-specific, delete. `health.spec.ts` — has `/c/test-id` workspace test, needs update. `accessibility.spec.ts` — tests `/compare`, `/tts`, `/whisper` (workspace routes). `visual-regression.spec.ts` — tests `/` (workspace) and `/compare`. E2E impact is larger than "~3 specs" claimed | §1.4, Test Impact |
| F7 | **MEDIUM** | 3 npm dependencies are workspace-only and can be removed: `idb` (IndexedDB), `cron-parser` (scheduled tasks), `fuse.js` (CommandPalette search) | §1.3 |
| F8 | **MEDIUM** | `globals.css` has no workspace-specific tokens — all 319 lines are shared (card styles, responsive sidebar, light theme, utility classes). Entire step 1.6 was unnecessary | §1.6 |
| F9 | **MEDIUM** | `types.ts` workspace types include `ModelDefaults` (line 163–169) and `Attachment` (line 119–126) — `Attachment` is imported by `router.ts` (shared). Must keep `Attachment` type when trimming types.ts | §1.3 |
| F10 | **LOW** | Line count "~122,000" for Mission Control page.tsx is wrong — 121 KB ≈ 2,819 lines, not 122K lines | Files Kept table |
| F11 | **LOW** | Vitest test count "80" comes from AGENTS.md but 7 test files × ~10-15 tests each ≈ 70-80 is plausible. Removing 2 test files drops to ~65-73 range | Test Impact |
| F12 | **LOW** | Risk "Mission Control page.tsx has workspace imports" is NOT low — it imports `types.ts` types (OllamaData, WhisperData, etc.) which are in the KEEP section. Risk is actually about the `Attachment` type in the trimmed section | Risks |
| F13 | **LOW** | Open Question #1 is now RESOLVED — Mission Control definitively uses TTS/Whisper. Remove from open questions | Open Questions |
**Pass 2 (Phase 4 + consistency review):**
| # | Severity | Finding | Section Fixed |
| --- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- |
| R1 | **MEDIUM** | "Route group (22 items)" heading lists 22 but only enumerates 6 route files; 16 components listed separately — misleading count | §Current State |
| R2 | **MEDIUM** | File Inventory says "6 pages + layout" (=7 files) but there are 5 pages + 1 layout = 6 files | §File Inventory |
| R3 | **MEDIUM** | LocalMemGPT test count "102" is stale — field-encrypt integration added tests, current total is 104 | §Success Criteria, §Test Impact |
| R4 | **LOW** | Phase 4.2 title says `@bytelyst/sse-helpers` (new package) but body proposes extending existing `@bytelyst/fastify-sse` | §4.2 |
| R5 | **LOW** | Phase 4.3 ActionTrail listed as "(inferred, same pattern)" but actually confirmed at `web/src/lib/use-theme.ts` | §4.3 |
| R6 | **LOW** | Phase 4.7 says LLM Lab has 3 stream parser copies but 2 are in workspace files deleted in Phase 1 — post-consolidation savings differ | §4.7 |
| R7 | **LOW** | Phase 4 structurally placed after Summary/Success sections instead of grouped with Phases 1-3 — needs addendum note | §Phase 4 header |
| R8 | **LOW** | `router.ts` imports both `Attachment` AND `ModelDefaults` from types.ts — both must be kept (doc mentions Attachment but buries ModelDefaults) | §1.3 |
| R9 | **LOW** | Mission Control API route count says "11" but actual count is 10 (route.ts + pull + chat + stream + logs + tts + whisper + whisper/transcribe + system + system/memory + system/exec + health + extraction = 13) | §Files Kept |
---
## Current State Audit
### Local LLM Lab — Workspace (`(workspace)/`)
Files that constitute the chat workspace to be removed:
**Route pages + layout (6 files): [R1]**
- `dashboard/src/app/(workspace)/layout.tsx` — workspace shell (sidebar + routing)
- `dashboard/src/app/(workspace)/page.tsx` — workspace home **[F3: this is the root page `/` — must add a redirect after deletion]**
- `dashboard/src/app/(workspace)/c/[id]/page.tsx` — conversation detail
- `dashboard/src/app/(workspace)/compare/page.tsx` — side-by-side compare
- `dashboard/src/app/(workspace)/tts/page.tsx` — text-to-speech page
- `dashboard/src/app/(workspace)/whisper/page.tsx` — speech-to-text page
**Workspace components (16 files):**
- `ConversationView.tsx` — chat message display + streaming (13 KB)
- `Sidebar.tsx` — workspace nav (9 KB)
- `InputBar.tsx` — message input with attachments (9 KB)
- `AgentEditor.tsx` — create/edit custom agents (9 KB)
- `TaskEditor.tsx` — scheduled task CRUD (9 KB)
- `OrchestrationEditor.tsx` — chain/race/vote pipelines (9 KB)
- `OrchestrationRunner.tsx` — execute orchestrations (9 KB)
- `QuickActionEditor.tsx` — quick action CRUD (8 KB)
- `MessageBubble.tsx` — message rendering (6 KB)
- `CommandPalette.tsx` — Cmd+K action palette (6 KB)
- `ProjectSidebar.tsx` — project panel (5 KB)
- `TaskRunner.tsx` — execute scheduled tasks (4 KB)
- `ProjectSwitcher.tsx` — project switcher (3 KB)
- `ConversationList.tsx` — conversation list (3 KB)
- `MessageThread.tsx` — threaded messages (1.4 KB)
- `ContextBar.tsx` — context display (1.1 KB)
**Lib files (workspace-specific):**
- `db.ts` — IndexedDB schema + CRUD (14 KB, 494 lines) — conversations, messages, agents, quickActions, projects, scheduledTasks, taskRuns, modelRatings, orchestrations
- `agents.ts` — 6 built-in agent definitions (4 KB)
- `quick-actions.ts` — 30+ built-in quick actions (8 KB)
- `scheduled-tasks.ts` — task scheduler logic (2 KB)
- `cron.ts` — cron expression parser (1.5 KB)
- `migrate.ts` — v3→v4 IndexedDB migration (4 KB)
- `types.ts` — Conversation, Message, Agent, QuickAction, Project, ScheduledTask, Orchestration types (lines 78–221)
**API routes (workspace-only — safe to delete):**
- `api/ollama/title/route.ts` — auto-generate conversation title (only used by workspace `ConversationView.tsx`)
**API routes INCORRECTLY listed as workspace-only — must KEEP [F1, F2]:**
- ~~`api/ollama/chat/route.ts`~~ — **KEEP** — Mission Control calls this for model testing (line 719 of `page.tsx`)
- ~~`api/ollama/stream/route.ts`~~ — **KEEP** — Mission Control calls this for model comparison and benchmarks (lines 331, 538)
- ~~`api/ollama/logs/route.ts`~~ — **KEEP** — Mission Control has Ollama log viewer (line 297)
- ~~`api/tts/route.ts`~~ — **KEEP** — Mission Control has TTS engine status panel (line 179)
- ~~`api/whisper/route.ts`~~ — **KEEP** — Mission Control fetches Whisper status on load (line 165)
- ~~`api/whisper/transcribe/route.ts`~~ — **KEEP** — Mission Control has Whisper transcription test (line 313)
**Tests (workspace-specific):**
- `__tests__/agents.test.ts`
- `__tests__/cron.test.ts`
### Local LLM Lab — Mission Control (`(mission-control)/`)
Files to **KEEP** — these are the unique infrastructure value:
- `mission-control/page.tsx` — system overview, model management, benchmarks (121 KB)
- `mission-control/components/` — Mission Control UI components
**API routes to KEEP:**
- `api/ollama/route.ts` — model list, pull, delete
- `api/ollama/pull/route.ts` — model pull with progress
- `api/ollama/chat/route.ts` — chat completion proxy **[F1: used by Mission Control model testing]**
- `api/ollama/stream/route.ts` — raw streaming proxy **[F1: used by Mission Control comparison/benchmarks]**
- `api/ollama/logs/route.ts` — Ollama server logs **[F1: used by Mission Control log viewer]**
- `api/tts/route.ts` — TTS engine status + synthesis **[F2: used by Mission Control TTS panel]**
- `api/whisper/route.ts` — Whisper status **[F2: used by Mission Control Whisper card]**
- `api/whisper/transcribe/route.ts` — Whisper transcription test **[F2: used by Mission Control]**
- `api/system/route.ts` — hardware info (chip, GPU, memory, disk)
- `api/system/memory/route.ts` — memory pressure stats
- `api/system/exec/route.ts` — optional exec endpoint
- `api/health/route.ts` — health check
- `api/extraction/route.ts` — extraction proxy
**Lib files to KEEP:**
- `llm-router.ts` — `@bytelyst/llm-router` re-export
- `ollama-config.ts` — OLLAMA_HOST env loading
- `product-config.ts` — product identity
- `router.ts` — model routing logic
- `format.ts` — byte/duration formatters
- `use-theme.ts` — dark/light toggle
- `use-keyboard-shortcuts.ts` — global shortcuts
**Shared components to KEEP:**
- `src/components/MarkdownResponse.tsx` — markdown renderer
- `src/components/ProgressBar.tsx`
- `src/components/RamBudgetBar.tsx`
- `src/components/Sparkline.tsx`
- `src/components/StatusDot.tsx`
- `src/components/Toast.tsx`
**Tests to KEEP:**
- `__tests__/format.test.ts`
- `__tests__/ollama-config.test.ts`
- `__tests__/product-config.test.ts`
- `__tests__/use-keyboard-shortcuts.test.ts`
- `__tests__/use-theme.test.ts`
### Local LLM Lab — Non-Dashboard Assets (all KEEP)
- `docs/` — 11 setup guides + hardware specs
- `tts/` — TTS scripts + samples
- `scripts/` — start-dashboard.sh + windows setup
- `experiments/` — oss-llm, open-claw, voicebox
- `chat-history/windsurf/` — Windsurf IDE archive
### Local Memory GPT — Features LLM Lab Has But LocalMemGPT Doesn't
These are **optional ports** — only if they prove valuable:
| Feature | LLM Lab Implementation | Port Priority |
| ------------------------------------ | ------------------------------------ | ----------------- |
| **Agents** (custom personas) | 6 built-in + custom, IndexedDB | P2 — nice to have |
| **Quick Actions** (30+ templates) | Code review, explain, refactor, etc. | P2 — nice to have |
| **Orchestrations** (chain/race/vote) | Multi-model pipelines | P3 — advanced |
| **Projects** (conversation grouping) | Folder-like grouping | P1 — useful |
| **Scheduled Tasks** (cron) | Cron-based automated prompts | P3 — niche |
| **Model Ratings** (up/down) | Per-message thumbs up/down | P1 — useful |
| **STT input** (whisper) | whisper.cpp integration | P2 — voice input |
| **TTS output** (Orpheus) | Ollama-based speech synthesis | P3 — experimental |
| **Conversation branching** | parentId + branchIndex | P2 — nice to have |
| **Attachments** (image/file/audio) | Multi-modal input | P1 — useful |
---
## Phases
### Phase 1 — Remove Workspace from Local LLM Lab
> **Effort:** ~2 hours
> **Risk:** Low — deletion only, no logic changes
> **Repo:** `learning_ai_local_llms`
#### 1.1 Delete workspace route group
Delete the entire `(workspace)/` directory:
```
dashboard/src/app/(workspace)/ # 22 items — layout, pages, 16 components
```
#### 1.2 Delete workspace-specific API routes
Delete **only** the route that is workspace-only:
```
dashboard/src/app/api/ollama/title/ # Auto-title generation (workspace-only)
```
**[F1, F2] The following routes MUST be kept** — Mission Control uses them directly:
- `api/ollama/chat/` — model testing prompt (page.tsx line 719)
- `api/ollama/stream/` — model comparison + benchmarks (lines 331, 538)
- `api/ollama/logs/` — Ollama log viewer (line 297)
- `api/tts/` — TTS engine status panel (line 179)
- `api/whisper/` — Whisper status card (line 165)
- `api/whisper/transcribe/` — transcription test button (line 313)
#### 1.3 Clean up lib files
Delete workspace-only lib files:
```
dashboard/src/app/lib/db.ts # IndexedDB (conversations, messages, agents, etc.)
dashboard/src/app/lib/agents.ts # Built-in agent definitions
dashboard/src/app/lib/quick-actions.ts # Built-in quick action definitions
dashboard/src/app/lib/scheduled-tasks.ts # Task scheduler
dashboard/src/app/lib/cron.ts # Cron parser
dashboard/src/app/lib/migrate.ts # v3→v4 migration
```
Remove workspace types from `types.ts` (lines 78–221: Conversation, Message, Agent, QuickAction, Project, ScheduledTask, Orchestration). Keep lines 1–77 (OllamaModel, RunningModel, OllamaData, WhisperModel, WhisperData, SystemData, Toast, PullProgress, StreamMetrics). **[F9, R8] Also keep `Attachment` (lines 119–126) and `ModelDefaults` (lines 163–169)** — both are imported by `router.ts` (line 1: `import type { Attachment, ModelDefaults } from './types'`) which is used by `api/ollama/chat/route.ts` (shared).
#### 1.4 Delete workspace-specific tests
```
dashboard/src/app/lib/__tests__/agents.test.ts
dashboard/src/app/lib/__tests__/cron.test.ts
```
#### 1.5 Create root redirect page [F3]
**[F3] CRITICAL:** The workspace route group `(workspace)/page.tsx` is the current root page (`/`). There is no separate `src/app/page.tsx`. After deleting the workspace, `GET /` returns 404.
**Required:** Create `dashboard/src/app/page.tsx` that redirects to `/mission-control`:
```tsx
import { redirect } from 'next/navigation';
export default function Home() {
redirect('/mission-control');
}
```
- Verify `layout.tsx` root has no workspace-specific imports (confirmed: it only imports fonts, design tokens, product-config, and Providers — no workspace references)
- Update any shared components that reference workspace routes
#### 1.6 ~~Update globals.css~~ — NO ACTION NEEDED [F8]
**[F8]** Verified: `globals.css` (319 lines) contains no workspace-specific tokens. All styles (card, sidebar, light theme, utility classes, skeleton shimmer) are shared by Mission Control. Skip this step.
#### 1.6a Remove workspace-only npm dependencies [F7]
Remove from `dashboard/package.json`:
- `idb` — IndexedDB client, only used by `db.ts` (workspace)
- `cron-parser` — only used by `cron.ts` and `TaskEditor.tsx` / `TaskRunner.tsx` (workspace)
- `fuse.js` — only used by `CommandPalette.tsx` (workspace)
#### 1.7 Verify
```bash
cd dashboard && pnpm run typecheck && pnpm test && pnpm run build
```
**Commit:** `refactor(dashboard): remove chat workspace — LocalMemGPT is the chat surface`
#### 1.8 Update documentation
- Update `AGENTS.md` — remove workspace references, update repo layout, adjust test count
- Update `README.md` — remove chat workspace description, add note pointing to Local Memory GPT for chat
- Update `CLAUDE.md`, `.windsurfrules`, `.cursorrules` — align with new scope
**Commit:** `docs: update agent docs after workspace removal`
---
### Phase 2 — Add Cross-Link Between Repos
> **Effort:** ~30 minutes
> **Risk:** None
> **Repos:** Both
#### 2.1 Local LLM Lab → Local Memory GPT link
Add a prominent link in Mission Control dashboard pointing to Local Memory GPT:
- "Chat with your models →" card or banner
- Links to `http://localhost:3070` (LocalMemGPT web default port)
- Configurable via env var: `LOCALMEMGPT_URL`
**Commit:** `feat(dashboard): add LocalMemGPT cross-link in Mission Control`
#### 2.2 Local Memory GPT → Local LLM Lab link
Add a link in Settings panel:
- "Model Management →" pointing to `http://localhost:3000` (LLM Lab dashboard)
- Configurable via env var: `LOCALLLMLAB_URL`
**Commit:** `feat(web): add LLM Lab cross-link in settings`
---
### Phase 3 (Optional) — Port Valuable Features to Local Memory GPT
> **Effort:** ~3–5 days total (pick and choose)
> **Risk:** Medium — new features in a working app
> **Repo:** `learning_ai_local_memory_gpt`
Only pursue these if the features are missed after Phase 1.
#### 3.1 Conversation Attachments (P1)
LocalMemGPT currently has no attachment support. LLM Lab supports image/file/audio/url attachments.
- Add `attachments` field to messages table (SQLite JSON column)
- Update chat SSE handler to pass images to Ollama vision models
- Update `MessageBubble.tsx` to render attachments
- Update `ChatView.tsx` input to support file upload
**Effort:** ~1 day
#### 3.2 Model Ratings (P1)
LLM Lab has per-message thumbs up/down ratings with IndexedDB storage.
- Add `rating` column to messages table
- Add `PATCH /api/messages/:id/rate` endpoint
- Update `MessageBubble.tsx` with rating buttons
**Effort:** ~2 hours
#### 3.3 Projects / Folders (P1)
LLM Lab groups conversations into projects. LocalMemGPT has no grouping.
- Add `folders` table to SQLite
- Add `folderId` column to conversations table
- Add folder CRUD endpoints
- Update Sidebar to show folder tree
**Effort:** ~4 hours
#### 3.4 Custom Agents / System Prompts (P2)
LLM Lab has 6 built-in agents + custom agent editor. LocalMemGPT supports per-conversation system prompts but no reusable agent definitions.
- Add `agents` table to SQLite
- Add agent CRUD endpoints
- Add agent picker in new conversation flow
- Port 6 built-in agent definitions
**Effort:** ~4 hours
#### 3.5 Quick Actions (P2)
LLM Lab has 30+ built-in quick actions (code review, explain, summarize, etc.).
- Add `quick_actions` table to SQLite
- Add quick action CRUD + execute endpoints
- Add Cmd+K command palette component
- Port built-in quick action definitions
**Effort:** ~6 hours
#### 3.6 Voice Input — Whisper STT (P2)
LLM Lab has whisper.cpp integration for speech-to-text.
- Add `POST /api/whisper/transcribe` endpoint (shell out to whisper binary)
- Add microphone button in ChatView input bar
- Detect whisper binary availability
**Effort:** ~4 hours
#### 3.7 Orchestrations — Multi-Model Pipelines (P3)
LLM Lab supports chain/race/vote multi-model orchestrations. LocalMemGPT has compare (parallel) but not chain/vote.
- Add `orchestrations` table
- Add orchestration CRUD + execute endpoints
- Add orchestration editor UI
**Effort:** ~1 day
#### 3.8 Scheduled Tasks (P3)
LLM Lab has cron-based scheduled prompts. Very niche feature.
- Add `scheduled_tasks` table
- Add task scheduler (setInterval-based in backend)
- Add task editor UI
**Effort:** ~4 hours
---
## File Inventory Summary
### Phase 1 Deletions (Local LLM Lab)
| Category | Files | ~Lines |
| --------------------- | ----------------------- | ---------------- |
| Workspace route group | 5 pages + 1 layout [R2] | ~700 |
| Workspace components | 16 `.tsx` files | ~5,500 |
| Workspace API routes | 1 route dir (`title/`) | ~50 |
| Workspace lib files | 6 `.ts` files | ~1,800 |
| Workspace types | partial `types.ts` | ~140 |
| Workspace tests | 2 test files | ~120 |
| Workspace npm deps | 3 packages removed | — |
| **Total** | **~30 files** | **~8,310 lines** |
### Files Kept (Local LLM Lab)
| Category | Files | ~Lines |
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- |
| Mission Control | 1 page + components | ~2,819 lines (page.tsx is 121 KB / 2,819 lines) [F10] |
| Mission Control API routes | 13 route dirs/files [R9] (route.ts + pull + chat + stream + logs + tts + whisper + whisper/transcribe + system + system/memory + system/exec + health + extraction) | ~1,200 |
| Shared lib files | 7 `.ts` files | ~600 |
| Shared components | 6 `.tsx` files | ~400 |
| Shared tests | 5 test files | ~500 |
| Non-dashboard assets | docs, tts, scripts, experiments | ~250 files |
| **Total** | **~275 files** | — |
### Test Impact
| Metric | Before | After |
| ---------------------- | ----------------------------------------------- | ------------------------------------------------------ |
| LLM Lab Vitest tests | ~80 (7 test files) | ~65-73 (−agents.test, −cron.test; 5 test files remain) |
| LLM Lab Playwright E2E | 7 specs | **Need updates in 5 specs** [F6] — see below |
| LocalMemGPT tests | 79 backend + 23 web + field-encrypt = ~104 [R3] | unchanged |
**E2E spec details [F6]:**
| Spec | Action |
|------|--------|
| `chat.spec.ts` | **Delete** — tests workspace chat input/send at `/` |
| `health.spec.ts` | **Update** — remove `Conversation Routes` test that navigates to `/c/test-id` |
| `accessibility.spec.ts` | **Update** — remove `/compare`, `/tts`, `/whisper` from `routes` array (keep `/`, `/mission-control`) |
| `visual-regression.spec.ts` | **Update** — remove `workspace` and `compare` from ROUTES array (keep `mission-control`) |
| `navigation.spec.ts` | **Update** — test at `/` now redirects to `/mission-control` |
| `mission-control.spec.ts` | **Keep as-is** |
| `models.spec.ts` | **Update** — navigates to `/` which now redirects |
---
## Environment Variables
### New (Phase 2)
| Variable | Repo | Default | Description |
| ----------------- | ------------------------------ | ----------------------- | ---------------------------------- |
| `LOCALMEMGPT_URL` | `learning_ai_local_llms` | `http://localhost:3070` | Link target for "Chat with models" |
| `LOCALLLMLAB_URL` | `learning_ai_local_memory_gpt` | `http://localhost:3000` | Link target for "Model Management" |
---
## Risks & Mitigations
| Risk | Likelihood | Mitigation |
| ---------------------------------------------------------- | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Users have workspace conversations they want to keep | Medium | Provide a one-time export script (Phase 1, pre-deletion) that dumps IndexedDB conversations to JSON importable by LocalMemGPT |
| Mission Control page.tsx imports trimmed types | Low | **[F12]** Verified: Mission Control imports `OllamaData`, `WhisperData`, `SystemData`, `Toast`, `PullProgress`, `StreamMetrics` — all in the KEEP section. `Attachment` type must also be kept since `router.ts` uses it |
| ~~TTS/Whisper routes are workspace-only~~ | ~~Low~~ | **[F2] RESOLVED:** Mission Control definitively uses both TTS and Whisper routes (46 references). All kept. |
| Phase 3 feature ports introduce bugs in stable LocalMemGPT | Medium | Each port gets its own commit + test coverage; skip ports that aren't missed |
---
## Success Criteria
- [ ] Local LLM Lab dashboard loads with Mission Control as the default page
- [ ] No workspace routes, components, or lib files remain
- [ ] `pnpm run typecheck && pnpm test && pnpm run build` passes in LLM Lab
- [ ] LLM Lab AGENTS.md accurately reflects the new scope
- [ ] Cross-links work in both directions
- [ ] Local Memory GPT is unchanged and all ~104 tests pass [R3]
- [ ] (Phase 4) `@bytelyst/ollama-client` published to Gitea registry, consumed by both repos
- [ ] (Phase 4) `@bytelyst/fastify-sse` extended with per-request helpers, LocalMemGPT migrated
---
## Phase 4 — DRY Refactoring & Common Platform Extractions
> **Note:** [R7] This phase was added as an addendum during the DRY audit (2026-03-29). It is independent of Phases 1-3 and can be executed before, after, or in parallel.
>
> **Effort:** ~3–5 days total
> **Risk:** Low-Medium — new shared packages, consumers migrate incrementally
> **Repo:** `learning_ai_common_plat` (new packages) + both local AI repos + ecosystem-wide
Code audit identified **7 extraction candidates** — duplicated patterns across the two local AI repos and the broader ecosystem.
### 4.1 `@bytelyst/ollama-client` — Ollama API Client (NEW PACKAGE)
**Problem:** Both repos implement nearly identical Ollama API interaction code with duplicated NDJSON stream parsing.
| Repo | File | Functions | ~Lines |
| ----------- | ---------------------------------------------- | ----------------------------------------------------------------------------- | ------ |
| LocalMemGPT | `backend/src/lib/ollama.ts` | `listOllamaModels()`, `checkOllamaHealth()`, `streamChat()`, `getEmbedding()` | 117 |
| LLM Lab | `dashboard/src/app/api/ollama/route.ts` | `fetchOllama()`, model list/pull/delete/show/load/unload | 128 |
| LLM Lab | `dashboard/src/app/api/ollama/stream/route.ts` | raw streaming proxy | 37 |
| LLM Lab | `dashboard/src/app/api/ollama/chat/route.ts` | chat proxy with llm-router | 109 |
| LLM Lab | `dashboard/src/app/lib/ollama-config.ts` | `OLLAMA_URL` env resolution + WSL2 gateway detection | 40 |
**Shared NDJSON stream parsing pattern** (copy-pasted 5+ times across both repos):
```ts
const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const lines = buffer.split('\n');
buffer = lines.pop() ?? '';
for (const line of lines) {
if (!line.trim()) continue;
try {
yield JSON.parse(line);
} catch {
/* skip */
}
}
}
```
**Proposed package:** `packages/ollama-client/`
```
@bytelyst/ollama-client
├── src/
│ ├── config.ts # resolveOllamaUrl(env) — OLLAMA_URL/OLLAMA_HOST + WSL2 detection
│ ├── client.ts # OllamaClient class: tags, ps, show, pull, load, unload, delete, version
│ ├── stream.ts # streamChat(), streamGenerate() — AsyncGenerator<OllamaStreamChunk>
│ ├── embed.ts # getEmbedding()
│ ├── health.ts # checkHealth() with timeout
│ ├── ndjson.ts # parseNdjsonStream() — reusable NDJSON async generator
│ └── types.ts # OllamaModel, OllamaChatMessage, OllamaStreamChunk, etc.
└── package.json
```
**Consumers:**
- `learning_ai_local_memory_gpt/backend` — replace `lib/ollama.ts` (117 lines deleted)
- `learning_ai_local_llms/dashboard` — replace `lib/ollama-config.ts` + 4 API route files simplified
- `@bytelyst/llm-router` — could consume `@bytelyst/ollama-client` for its Ollama provider
**Effort:** ~1 day
**Priority:** P0 — eliminates the largest duplication
---
### 4.2 Extend `@bytelyst/fastify-sse` — Per-Request SSE Utilities [R4]
**Problem:** `@bytelyst/fastify-sse` provides `SSEHub` (broadcast to multiple clients) but NOT per-request SSE helpers. LocalMemGPT reinvented these as `sse-helpers.ts` (26 lines).
| Existing | What it does |
| ---------------------------- | -------------------------------------------------------------------------------------- |
| `@bytelyst/fastify-sse` | `SSEHub` — multi-client broadcast pattern (hub/spoke) |
| LocalMemGPT `sse-helpers.ts` | `startSSE()`, `sendSSEData()`, `sendSSEEvent()`, `endSSE()` — single-request streaming |
**Proposed:** Add per-request helpers to `@bytelyst/fastify-sse`:
```ts
// New exports in @bytelyst/fastify-sse
export { startSSE, sendSSEData, sendSSEEvent, endSSE } from './per-request.js';
```
**Consumers:**
- `learning_ai_local_memory_gpt/backend` — replace `lib/sse-helpers.ts` with import from package
- `learning_ai_trails/backend` — already uses `@bytelyst/fastify-sse` hub, could use per-request too
**Effort:** ~2 hours
**Priority:** P1
---
### 4.3 `@bytelyst/use-theme` — React Theme Toggle Hook (NEW PACKAGE)
**Problem:** Near-identical `useTheme()` hook duplicated across **6 web apps** — only the storage key and class-application strategy differ.
| Repo | File | Storage Key | Lines |
| ----------- | ------------------------------------ | ----------- | ----- |
| LocalMemGPT | `web/src/lib/use-theme.ts` | `lmg-theme` | 49 |
| LLM Lab | `dashboard/src/app/lib/use-theme.ts` | `llm-theme` | 48 |
| ChronoMind | `web/src/lib/use-theme.ts` | `cm-theme` | ~50 |
| FlowMonk | `web/src/lib/use-theme.ts` | `fm-theme` | ~50 |
| NomGap | `web/src/lib/use-theme.ts` | `ng-theme` | ~50 |
| ActionTrail | `web/src/lib/use-theme.ts` [R5] | `at-theme` | ~50 |
**Proposed package:** `packages/use-theme/`
```ts
export function useTheme(options?: { storageKey?: string; attribute?: 'class' | 'data-theme' }) {
// Configurable storage key (default: `${PRODUCT_ID}-theme`)
// Configurable DOM application (classList vs data-attribute)
// Returns { theme, setTheme, toggleTheme }
}
```
**Effort:** ~2 hours
**Priority:** P2 — saves ~300 total lines across 6 repos
---
### 4.4 `@bytelyst/use-keyboard-shortcuts` — Keyboard Shortcuts Hook (NEW PACKAGE)
**Problem:** Near-identical `useKeyboardShortcuts()` hook duplicated across **5+ web apps**, each with slight variations in handler names.
| Repo | File |
| ----------- | ------------------------------------------------------------ |
| LocalMemGPT | `web/src/lib/use-keyboard-shortcuts.ts` (53 lines) |
| LLM Lab | `dashboard/src/app/lib/use-keyboard-shortcuts.ts` (57 lines) |
| ChronoMind | `web/src/lib/use-keyboard-shortcuts.ts` |
| ActionTrail | `web/src/lib/use-keyboard-shortcuts.ts` |
| NoteLett | `web/src/lib/use-keyboard-shortcuts.ts` |
**Proposed package:** `packages/use-keyboard-shortcuts/`
```ts
export function useKeyboardShortcuts(shortcuts: ShortcutDef[]) {
// Each ShortcutDef: { key, meta?, shift?, handler, allowInInput? }
// Data-driven instead of hardcoded if/else chains
}
```
**Effort:** ~3 hours
**Priority:** P2 — saves ~250 lines, improves consistency
---
### 4.5 Ollama Model Types — Deduplicate into `@bytelyst/ollama-client` types
**Problem:** `OllamaModel` interface is defined independently in both repos with slightly different shapes.
| Repo | Definition |
| ----------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| LocalMemGPT `ollama.ts` | `{ name, size, digest, modified_at, details?: { parameter_size, quantization_level, family } }` |
| LLM Lab `types.ts` | `{ name, model, modified_at, size, digest, details: { families, family, format, parameter_size, quantization_level } }` |
**Resolution:** Canonical `OllamaModel` type in `@bytelyst/ollama-client/types.ts` — matches Ollama's actual API response. Both repos import from there.
**Effort:** included in 4.1
**Priority:** P0 (part of 4.1)
---
### 4.6 LLM Lab `format.ts` Utilities — Promote Reusable Functions
**Problem:** `format.ts` (96 lines) contains both LLM Lab-specific functions AND general-purpose utilities:
| Function | Scope | Extraction Target |
| ------------------------- | ------------------------------------------------------ | --------------------------------------------------- |
| `formatBytes()` | **Reusable** — any repo needs byte formatting | `@bytelyst/ollama-client` or new `@bytelyst/format` |
| `estimateRam()` | LLM-infrastructure-specific | Keep in LLM Lab |
| `checkMemoryFit()` | LLM-infrastructure-specific | Keep in LLM Lab |
| `getModelBadges()` | LLM-infrastructure-specific | Keep in LLM Lab |
| `estimateTokens()` | **Reusable** — any LLM app needs token estimation | `@bytelyst/ollama-client` |
| `getModelContextWindow()` | **Reusable** — any LLM app needs context window lookup | `@bytelyst/ollama-client` |
| `formatUptime()` | **Reusable** — monitoring/ops dashboards | `@bytelyst/ollama-client` |
**Effort:** ~1 hour (as part of 4.1)
**Priority:** P1
---
### 4.7 Client-Side SSE/NDJSON Stream Parser (Web)
**Problem:** The client-side SSE parsing pattern is copy-pasted 3 times in LocalMemGPT's `web/src/lib/api.ts` (`streamChat`, `streamCompare`, plus a `parseSSEBody` in tests). LLM Lab has 3 more copies in `ConversationView.tsx`, `compare/page.tsx`, and `mission-control/page.tsx`. **[R6] Note:** After Phase 1, `ConversationView.tsx` and `compare/page.tsx` are deleted — LLM Lab reduces to 1 copy (Mission Control). Total post-Phase-1 copies: 4 (3 LocalMemGPT + 1 LLM Lab).
**Proposed:** Export from `@bytelyst/ollama-client`:
```ts
// Client-side SSE stream consumer
export function consumeSSEStream(
response: Response,
onData: (data: Record<string, unknown>) => void,
onDone: () => void
): Promise<void>;
// Client-side NDJSON stream consumer
export function consumeNdjsonStream<T>(
response: Response,
onChunk: (chunk: T) => void
): Promise<void>;
```
**Effort:** ~2 hours (as part of 4.1)
**Priority:** P1 — eliminates 6 copy-pasted stream parsers
---
### Phase 4 Summary
| # | Package | Action | Lines Saved | Repos Affected | Priority |
| --- | ---------------------------------- | ---------- | ----------- | ----------------------- | -------- |
| 4.1 | `@bytelyst/ollama-client` | **NEW** | ~350 | 2 local AI + llm-router | P0 |
| 4.2 | `@bytelyst/fastify-sse` | **EXTEND** | ~26 | LocalMemGPT + future | P1 |
| 4.3 | `@bytelyst/use-theme` | **NEW** | ~300 | 6 web apps | P2 |
| 4.4 | `@bytelyst/use-keyboard-shortcuts` | **NEW** | ~250 | 5 web apps | P2 |
| 4.5 | (types in 4.1) | — | included | 2 local AI | P0 |
| 4.6 | (utils in 4.1) | — | ~40 | 2 local AI | P1 |
| 4.7 | (client parsers in 4.1) | — | ~180 | 2 local AI | P1 |
**Total estimated savings:** ~1,150 lines of duplicated code across the ecosystem.
**Recommended execution order:**
1. **4.1** `@bytelyst/ollama-client` (biggest win, unblocks both repos)
2. **4.2** Extend `@bytelyst/fastify-sse` (small, quick)
3. **4.3 + 4.4** React hooks (lower priority, wider ecosystem impact — could batch with next DRY sweep)
---
## Open Questions
1. ~~**TTS/Whisper in Mission Control:**~~ **[F13] RESOLVED.** Mission Control uses `api/whisper` (status), `api/whisper/transcribe` (test), `api/tts` (engine status), `api/ollama/chat` (model testing), `api/ollama/stream` (benchmarks/compare), `api/ollama/logs` (log viewer). All 6 routes are kept.
2. **IndexedDB export:** Should we write a migration script that exports LLM Lab conversations into LocalMemGPT's import format? Only matters if the user has valuable conversations in LLM Lab.
3. **Phase 3 prioritization:** Which features (if any) from the optional port list are actually wanted? Recommend trying Phase 1+2 first, then deciding based on what's missed.