diff --git a/docs/BACKEND_LLM_ROUTER_DECISION.md b/docs/BACKEND_LLM_ROUTER_DECISION.md new file mode 100644 index 0000000..6b313df --- /dev/null +++ b/docs/BACKEND_LLM_ROUTER_DECISION.md @@ -0,0 +1,64 @@ +# Backend LLM Router Decision + +Status: active decision +Date: May 5, 2026 +Decision: keep `@bytelyst/llm` as the NoteLett release-1 backend LLM abstraction; defer `@bytelyst/llm-router` adoption. + +## Current NoteLett Usage + +NoteLett backend uses `@bytelyst/llm` through `backend/src/lib/llm.ts` for: + +- copilot transforms and title suggestions +- Smart Action prompt runs, including text and image inputs +- prompt schedules and webhooks +- MCP tag suggestions and note operations +- Palace memory extraction +- embeddings through `provider.embed()` for duplicate/related note and Palace search behavior +- mock provider support for local development and tests +- OpenAI/Azure provider selection through existing `LLM_PROVIDER`, `LLM_DEFAULT_MODEL`, `LLM_VISION_MODEL`, and `LLM_EMBEDDING_MODEL` environment settings + +## Router Evaluation + +`@bytelyst/llm-router` provides useful provider/model governance: + +- deterministic prompt classification +- provider/model selection +- round-robin routing +- health tracking +- fallback across configured providers +- telemetry hooks + +However, the current router API is centered on OpenAI-compatible chat completions with string-only messages and API-key filtered providers. It does not yet replace the NoteLett release-1 needs listed above: + +- no embedding API equivalent to `provider.embed()` +- no first-class mock provider for test/local parity +- no Azure provider compatibility path matching current production env +- no multipart image-message compatibility with `buildVisionMessage()` from `@bytelyst/llm` +- no streaming or provider interface parity with the existing `LLMProvider` contract +- default provider set targets free-tier routing, while NoteLett release-1 config is already expressed through `@bytelyst/llm` + +## Decision + +Do not adopt `@bytelyst/llm-router` in the production-readiness handoff. + +Continue using `@bytelyst/llm` for release 1, while keeping the router as a future governance layer once it can sit behind or implement the same `LLMProvider` capabilities NoteLett already depends on. + +## Required Adoption Criteria + +Adopt the router later when it supports or cleanly bridges: + +- `chatCompletion()` parity with the current `LLMProvider` response shape +- `embed()` for NoteLett embeddings and Palace semantic search +- mock provider behavior for deterministic tests +- Azure/OpenAI production settings without changing release env names +- multipart text+image prompts or an adapter for `buildVisionMessage()` +- telemetry mapping into NoteLett/platform telemetry +- fallback behavior that preserves current timeout/retry semantics +- tests for copilot, Smart Actions, prompt schedules, MCP suggestions, Palace extraction, and embeddings + +## Current Guardrails + +- Keep `backend/src/lib/llm.ts` as the single backend LLM entrypoint. +- Do not call both `@bytelyst/llm` and `@bytelyst/llm-router` from product routes. +- Treat `@bytelyst/llm-router` as a future provider-governance improvement after release-1 LLM and embedding behavior is stable. +- Continue tracking current `@bytelyst/llm`/package build drift through the production-readiness baseline and backend hardening phases. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 9d8e3b2..39f4fed 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -511,6 +511,7 @@ Detailed implementation plans live under `docs/roadmaps/`. - ~~**Blocker** — Product identity not finalized~~ **Resolved:** locked as NoteLett (`e1fde25`) - **Deferred** — `@bytelyst/react-native-platform-sdk` adoption is intentionally deferred for release handoff; mobile uses direct shared `@bytelyst/*` clients instead. See `docs/MOBILE_PLATFORM_SDK_DECISION.md`. - **Deferred** — `@bytelyst/sync` adoption is intentionally deferred for release handoff; mobile keeps the narrower `@bytelyst/offline-queue` retry model until backend sync/conflict contracts are explicit. See `docs/MOBILE_SYNC_DECISION.md`. +- **Deferred** — `@bytelyst/llm-router` adoption is intentionally deferred for release handoff; backend keeps `@bytelyst/llm` for chat, embeddings, mock provider, Azure/OpenAI settings, and vision prompt compatibility. See `docs/BACKEND_LLM_ROUTER_DECISION.md`. - ~~**Deferred** — Blob-backed artifacts and upload/download flows~~ **Resolved:** SAS-based blob client (`196b210`) - ~~**Deferred** — Fully integrated approval/review/audit UX~~ **Resolved:** batch review, review notes, agent timeline across web + mobile (`ca3cdba`, `f8a4c18`) - ~~**Deferred** — Persisted saved views~~ **Resolved:** backend CRUD + web client + UI (`bdbf387`, `12d9009`)