# Backend LLM Router Decision

Status: active decision
Date: May 5, 2026
Decision: keep `@bytelyst/llm` as the NoteLett release-1 backend LLM abstraction; defer `@bytelyst/llm-router` adoption.

## Current NoteLett Usage

NoteLett backend uses `@bytelyst/llm` through `backend/src/lib/llm.ts` for:

- copilot transforms and title suggestions
- Smart Action prompt runs, including text and image inputs
- prompt schedules and webhooks
- MCP tag suggestions and note operations
- Palace memory extraction
- embeddings through `provider.embed()` for duplicate/related note and Palace search behavior
- mock provider support for local development and tests
- OpenAI/Azure provider selection through existing `LLM_PROVIDER`, `LLM_DEFAULT_MODEL`, `LLM_VISION_MODEL`, and `LLM_EMBEDDING_MODEL` environment settings

## Router Evaluation

`@bytelyst/llm-router` provides useful provider/model governance:

- deterministic prompt classification
- provider/model selection
- round-robin routing
- health tracking
- fallback across configured providers
- telemetry hooks

However, the current router API is centered on OpenAI-compatible chat completions with string-only messages and API-key filtered providers. It does not yet replace the NoteLett release-1 needs listed above:

- no embedding API equivalent to `provider.embed()`
- no first-class mock provider for test/local parity
- no Azure provider compatibility path matching current production env
- no multipart image-message compatibility with `buildVisionMessage()` from `@bytelyst/llm`
- no streaming or provider interface parity with the existing `LLMProvider` contract
- default provider set targets free-tier routing, while NoteLett release-1 config is already expressed through `@bytelyst/llm`

## Decision

Do not adopt `@bytelyst/llm-router` in the production-readiness handoff.

Continue using `@bytelyst/llm` for release 1, while keeping the router as a future governance layer once it can sit behind or implement the same `LLMProvider` capabilities NoteLett already depends on.

## Required Adoption Criteria

Adopt the router later when it supports or cleanly bridges:

- `chatCompletion()` parity with the current `LLMProvider` response shape
- `embed()` for NoteLett embeddings and Palace semantic search
- mock provider behavior for deterministic tests
- Azure/OpenAI production settings without changing release env names
- multipart text+image prompts or an adapter for `buildVisionMessage()`
- telemetry mapping into NoteLett/platform telemetry
- fallback behavior that preserves current timeout/retry semantics
- tests for copilot, Smart Actions, prompt schedules, MCP suggestions, Palace extraction, and embeddings

## Current Guardrails

- Keep `backend/src/lib/llm.ts` as the single backend LLM entrypoint.
- Do not call both `@bytelyst/llm` and `@bytelyst/llm-router` from product routes.
- Treat `@bytelyst/llm-router` as a future provider-governance improvement after release-1 LLM and embedding behavior is stable.
- Continue tracking current `@bytelyst/llm`/package build drift through the production-readiness baseline and backend hardening phases.