diff --git a/docs/MEMPALACE_INTEGRATION_ROADMAP.md b/docs/MEMPALACE_INTEGRATION_ROADMAP.md new file mode 100644 index 0000000..0763768 --- /dev/null +++ b/docs/MEMPALACE_INTEGRATION_ROADMAP.md @@ -0,0 +1,966 @@ +# MemPalace Integration Roadmap — NoteLett + +> **Goal:** Add persistent, structured semantic memory to NoteLett so users build a personal knowledge palace — notes auto-organized into wings (workspaces), rooms (topics), and halls (memory types) — with cross-note knowledge graph, semantic search, and AI-assisted recall. All with **strict user isolation** at every layer. +> +> **Ref:** [MemPalace](https://github.com/milla-jovovich/mempalace) — highest-scoring AI memory system (96.6% LongMemEval). Adapted for a multi-user Fastify/Cosmos backend. + +--- + +## Design Decisions + +| # | Decision | Choice | Rationale | +|---|----------|--------|-----------| +| 0 | **Shared package** | New `@bytelyst/palace` in `learning_ai_common_plat/packages/palace/` | Core palace types, cosine similarity, dedup logic, relevance decay, KG helpers, extraction prompts, and wake-up builder are product-agnostic — shared across NoteLett, MindLyst, and future products | +| 1 | **Storage** | Cosmos DB only (via `@bytelyst/datastore`) | NoteLett is already Cosmos-native; no SQLite or ChromaDB — keeps infra simple, user isolation inherits from Cosmos partition keys | +| 2 | **Vector search** | Cosmos DB vector index (DiskANN) or in-memory cosine over stored embeddings | Avoids introducing ChromaDB; Cosmos vector search is GA; falls back to `cosineSimilarity()` from existing `embeddings.ts` | +| 3 | **Embedding model** | `@bytelyst/llm` embed() — `text-embedding-3-small` (1536-dim) | Already configured in `config.ts` (`LLM_EMBEDDING_MODEL`), existing `embedText()` in `lib/embeddings.ts` | +| 4 | **Extraction** | `@bytelyst/llm` chat() — `gpt-4o-mini` with structured output + regex fallback | Same LLM infrastructure as Smart Actions; mock provider for tests | +| 5 | **User isolation** | `userId` required on every document and every query filter, enforced at repository layer | Same pattern as all existing NoteLett modules (notes, workspaces, etc.) | +| 6 | **Palace scope** | Per-user palace, workspace = wing | Each user has their own palace; their NoteLett workspaces map to wings | +| 7 | **Hall types** | 6 halls: `decisions`, `events`, `discoveries`, `preferences`, `advice`, `insights` | `insights` replaces `errors` (coding-focused); captures AI-generated observations from note analysis | +| 8 | **Wake-up budget** | ~600 tokens (L0:50 + L1:150 + L2:400) | Enough for note-writing context; smaller than Claw-Cowork's 800 since notes have narrower scope | +| 9 | **MCP integration** | Extend existing MCP tools in `backend/src/mcp/` | NoteLett already has 8 MCP tools; add `mempalace_*` tools to the same system | +| 10 | **Save triggers** | Note create/update, agent action completion, prompt execution, explicit store | Event-driven based on NoteLett's existing lifecycle hooks | +| 11 | **Field encryption** | Sensitive memory content encrypted via `@bytelyst/field-encrypt` | Mirrors existing note body encryption pattern in `notes/repository.ts` | +| 12 | **productId scoping** | Every palace document includes `productId: "notelett"` | Standard ByteLyst ecosystem convention | + +--- + +## @bytelyst/* Package Dependency Map + +NoteLett's palace module leverages these existing shared packages: + +| Package | Usage in Palace | +|---------|----------------| +| `@bytelyst/datastore` | All palace Cosmos CRUD via `getCollection()` | +| `@bytelyst/field-encrypt` | Encrypt memory content at rest (AES-256-GCM) | +| `@bytelyst/llm` | `embed()` for embeddings, `chat()` for extraction | +| `@bytelyst/backend-config` | Extend `baseBackendConfigSchema` with palace env vars | +| `@bytelyst/fastify-auth` | `getUserId(req)` for JWT-based user isolation | +| `@bytelyst/fastify-core` | Route registration via `createServiceApp()` | +| `@bytelyst/errors` | Typed errors: `NotFoundError`, `BadRequestError` | +| `@bytelyst/cosmos` | `registerContainers()` for palace containers | +| `@bytelyst/extraction` | `createExtractionClient()` for entity extraction via extraction-service | +| `@bytelyst/testing` | Test helpers for `buildTestApp()` | +| **`@bytelyst/palace`** | **NEW** — shared palace types, cosine, dedup, decay, KG helpers, extraction prompts, wake-up builder | + +--- + +## Current State (What Exists) + +| Component | File | What It Does | +|-----------|------|-------------| +| **Notes CRUD** | `modules/notes/repository.ts` | Note storage with field encryption, userId filtering | +| **Workspaces** | `modules/workspaces/` | Workspace CRUD — maps to MemPalace "wings" | +| **Note Relationships** | `modules/note-relationships/` | Manual note linking — related to MemPalace "tunnels" | +| **Embeddings** | `lib/embeddings.ts` | `embedText()`, `cosineSimilarity()`, `stripHtmlForEmbedding()` — already exists | +| **LLM** | `lib/llm.ts` | `@bytelyst/llm` singleton — embed + chat capability | +| **Field Encrypt** | `lib/field-encrypt.ts` | `@bytelyst/field-encrypt` for sensitive data at rest | +| **Request Context** | `lib/request-context.ts` | `getUserId(req)`, `getRequestProductId(req)` — JWT-extracted | +| **MCP Tools** | `mcp/note-tools.ts` | 8 MCP tools with `requireUserId()` + `requireProductScope()` guards | +| **Smart Actions** | `modules/note-prompts/` | LLM-powered transforms — extraction patterns reusable for memory mining | + +### Key Gap + +Notes exist in isolation. No cross-note semantic memory, no knowledge graph connecting entities, no AI-assisted recall of past decisions. Users can search by text but not by meaning. Relationships are manual, not auto-discovered. + +--- + +## Architecture + +``` +┌──────────────────────────────────────────────────────────┐ +│ NoteLett Backend │ +│ │ +│ ┌─────────────┐ ┌──────────────┐ ┌────────────────┐ │ +│ │ Notes Module │ │ Palace Module│ │ MCP Tools │ │ +│ │ (existing) │ │ (NEW) │ │ (extended) │ │ +│ └──────┬───────┘ └──────┬───────┘ └───────┬────────┘ │ +│ │ │ │ │ +│ ┌──────┴─────────────────┴───────────────────┴────────┐ │ +│ │ @bytelyst/datastore │ │ +│ │ getCollection() — userId in every filter │ │ +│ └──────────────────────┬───────────────────────────────┘ │ +│ │ │ +│ ┌──────────────────────┴───────────────────────────────┐ │ +│ │ Cosmos DB (partition: /userId) │ │ +│ │ │ │ +│ │ ┌────────────┐ ┌───────────┐ ┌──────────────────┐ │ │ +│ │ │palace_wings│ │palace_rooms│ │palace_memories │ │ │ +│ │ │/userId │ │/userId │ │/userId (+ vector) │ │ │ +│ │ └────────────┘ └───────────┘ └──────────────────┘ │ │ +│ │ ┌────────────┐ ┌───────────┐ ┌──────────────────┐ │ │ +│ │ │palace_kg │ │palace_ │ │palace_tunnels │ │ │ +│ │ │/userId │ │diaries │ │/userId │ │ │ +│ │ │ │ │/userId │ │ │ │ │ +│ │ └────────────┘ └───────────┘ └──────────────────┘ │ │ +│ └───────────────────────────────────────────────────────┘ │ +└──────────────────────────────────────────────────────────┘ +``` + +### User Isolation Model + +**Every palace operation is scoped to the authenticated user.** Isolation is enforced at 3 layers: + +1. **JWT Auth Layer** — `getUserId(req)` extracts userId from the verified JWT; no user input accepted for identity +2. **Repository Layer** — Every query includes `userId` in its filter. No query can omit it. +3. **Cosmos Partition Layer** — All palace containers use `/userId` as partition key. Cross-partition queries are physically impossible without the partition key. + +```typescript +// EVERY repository function signature includes userId — no exceptions: +export async function searchMemories(userId: string, productId: string, query: SearchQuery): Promise { + const filter: FilterMap = { userId, productId }; + // ... userId is ALWAYS in the filter +} + +// Route handlers extract userId from JWT — never from request body: +fastify.get('/api/palace/search', async (req, reply) => { + const userId = getUserId(req); // from JWT + const productId = getRequestProductId(req); // from JWT + const query = SearchQuerySchema.parse(req.query); + const results = await searchMemories(userId, productId, query); + // ... +}); +``` + +### Workspace-to-Wing Mapping + +NoteLett workspaces map 1:1 to palace wings: + +| NoteLett Concept | Palace Concept | Relationship | +|---|---|---| +| Workspace | Wing | `wing.sourceWorkspaceId = workspace.id` | +| Note tags | Rooms | Auto-created from note tags or AI-suggested topics | +| Note relationships | Tunnels | Existing links become tunnel seeds | +| Note body | Memory content | Extracted decisions, facts, insights from notes | +| Smart Action output | Memory content | Agent-generated memories from prompt executions | + +--- + +## Phase N0 — Shared Palace Package (`@bytelyst/palace`) + +**Goal:** Create a reusable shared package in `learning_ai_common_plat/packages/palace/` with product-agnostic palace primitives. + +**Estimated: 3-4 days | New files: 8 | Tests: ~25** + +> **This phase is done in the `learning_ai_common_plat` repo, not in NoteLett.** + +### N0.1 Package Scaffold + +New package: `packages/palace/` (`@bytelyst/palace`) + +``` +packages/palace/ +├── src/ +│ ├── index.ts # Public exports +│ ├── types.ts # PalaceWingDoc, PalaceRoomDoc, PalaceMemoryDoc, PalaceTunnelDoc, PalaceKGTripleDoc, PalaceDiaryDoc +│ ├── halls.ts # HallType union, HALL_PRESETS (notelett, mindlyst, coding), hallFromLabel() +│ ├── cosine.ts # cosineSimilarity(), topKByCosine(), normalizeVector() +│ ├── dedup.ts # isNearDuplicate() — exact match + cosine threshold logic +│ ├── decay.ts # decayRelevance(), computeDecayedRelevance() — exponential half-life +│ ├── extraction.ts # buildExtractionPrompt(), parseExtractionResponse(), regexFallbackExtraction() +│ ├── kg.ts # KG helper types, contradictionCheck(), mergeTriples() +│ ├── wakeup.ts # WakeUpContext type, buildWakeUpLayers(), truncateToTokenBudget() +│ ├── config.ts # palaceConfigSchema (Zod) — PALACE_ENABLED, budgets, thresholds +│ └── __tests__/ +│ ├── cosine.test.ts +│ ├── dedup.test.ts +│ ├── decay.test.ts +│ ├── extraction.test.ts +│ ├── kg.test.ts +│ └── wakeup.test.ts +├── package.json # @bytelyst/palace, peerDeps: @bytelyst/llm, zod +└── tsconfig.json +``` + +### N0.2 Shared Types (`types.ts`) + +Product-agnostic base types — products extend with their own fields: + +```typescript +import type { BaseDocument } from '@bytelyst/datastore'; + +// Base types — every product extends these +export interface BasePalaceWingDoc extends BaseDocument { + userId: string; + productId: string; + name: string; + memoryCount: number; + l1Cache?: string; + l1CacheUpdatedAt?: string; + createdAt: string; + updatedAt: string; +} + +export interface BasePalaceMemoryDoc extends BaseDocument { + userId: string; + productId: string; + wingId: string; + roomId: string; + hall: string; + content: string; + relevance: number; + embedding?: number[]; + createdAt: string; + updatedAt: string; +} + +// ... similar for Room, Tunnel, KGTriple, Diary +``` + +### N0.3 Cosine Similarity (`cosine.ts`) + +Extracted from NoteLett's `embeddings.ts` — shared by all products: + +```typescript +export function cosineSimilarity(a: number[], b: number[]): number; +export function topKByCosine(query: number[], items: T[], getEmbedding: (t: T) => number[] | undefined, k: number): Array<{ item: T; score: number }>; +export function normalizeVector(v: number[]): number[]; +``` + +### N0.4 Dedup, Decay, Extraction, KG, Wake-Up + +All product-agnostic algorithms extracted into pure functions: + +- **dedup.ts** — `isContentDuplicate(content, candidates, threshold)` (exact + cosine) +- **decay.ts** — `computeDecayedRelevance(originalRelevance, createdAt, halfLifeDays)` (exponential) +- **extraction.ts** — `buildExtractionPrompt(content, context, hallTypes)`, `parseExtractionResponse(llmOutput)`, `regexFallbackExtraction(content)` +- **kg.ts** — `findContradictions(triples)`, `mergeTriples(existing, incoming)`, `isTripleCurrent(triple, asOf?)` +- **wakeup.ts** — `buildWakeUpLayers(l0, l1, l2, budget)`, `truncateToTokenBudget(text, maxTokens)` +- **config.ts** — `palaceConfigSchema` (Zod) extending `baseBackendConfigSchema` with `PALACE_ENABLED`, `PALACE_WAKE_UP_BUDGET`, `PALACE_DEDUP_THRESHOLD`, `PALACE_RELEVANCE_HALF_LIFE_DAYS` + +### N0.5 Tests + +- [x] d1c6cf4 Cosine similarity correctness (identical=1.0, orthogonal=0.0) +- [x] d1c6cf4 topKByCosine returns sorted results, handles missing embeddings +- [x] d1c6cf4 Dedup detects exact matches and near-duplicates above threshold +- [x] d1c6cf4 Decay reduces relevance exponentially by half-life +- [x] d1c6cf4 Extraction prompt builder produces valid LLM prompts for all hall presets +- [x] d1c6cf4 Extraction response parser handles valid JSON, malformed JSON, empty +- [x] d1c6cf4 Regex fallback extracts decisions, events from markdown +- [x] d1c6cf4 KG contradiction detection finds conflicting triples +- [x] d1c6cf4 Wake-up layer builder respects token budget +- [x] d1c6cf4 Palace config schema validates all env vars with defaults + +--- + +## Phase N1 — Palace Core Module + +**Goal:** New `palace` module with Cosmos-backed storage, user-isolated CRUD, and semantic search — consuming `@bytelyst/palace` for shared logic. + +**Estimated: 4-5 days | New files: 4 | Tests: ~30** + +### N1.1 Palace Cosmos Containers + +New containers in `cosmos-init.ts`: + +```typescript +// Add to CONTAINER_DEFS: +palace_wings: { partitionKeyPath: '/userId' }, +palace_rooms: { partitionKeyPath: '/userId' }, +palace_memories: { partitionKeyPath: '/userId' }, +palace_tunnels: { partitionKeyPath: '/userId' }, +palace_kg: { partitionKeyPath: '/userId' }, +palace_diaries: { partitionKeyPath: '/userId' }, +``` + +### N1.2 Palace Types (`modules/palace/types.ts`) + +```typescript +export interface PalaceWingDoc { + id: string; + productId: string; + userId: string; + name: string; // e.g. "Work Notes" + sourceWorkspaceId: string; // links to NoteLett workspace + techStack?: string; // auto-detected from note content + memoryCount: number; // denormalized for fast display + createdAt: string; + updatedAt: string; +} + +export interface PalaceRoomDoc { + id: string; + productId: string; + userId: string; + wingId: string; + name: string; // e.g. "auth-migration" + description?: string; + memoryCount: number; + createdAt: string; +} + +export const HALL_TYPES = ['decisions', 'events', 'discoveries', 'preferences', 'advice', 'insights'] as const; +export type HallType = (typeof HALL_TYPES)[number]; + +export interface PalaceMemoryDoc { + id: string; + productId: string; + userId: string; + wingId: string; + roomId: string; + hall: HallType; + content: string; // the memory text + sourceNoteId?: string; // which note produced this memory + sourceTaskId?: string; // which agent action produced this + relevance: number; // 0.0–1.0, decays over time + embedding?: number[]; // 1536-dim from text-embedding-3-small + createdAt: string; + updatedAt: string; +} + +export interface PalaceTunnelDoc { + id: string; + productId: string; + userId: string; + roomA: string; // normalized: roomA < roomB + roomB: string; + reason: string; + strength: number; // 0.0–1.0 + createdAt: string; +} + +export interface PalaceKGTripleDoc { + id: string; + productId: string; + userId: string; + wingId: string; + subject: string; + predicate: string; + object: string; + validFrom: string; + validTo?: string; // null = still current + sourceTaskId?: string; + confidence: number; + createdAt: string; +} + +export interface PalaceDiaryDoc { + id: string; + productId: string; + userId: string; + roleId: string; // e.g. "note-copilot" + wingId?: string; + entry: string; + createdAt: string; +} +``` + +### N1.3 Palace Repository (`modules/palace/repository.ts`) + +Imports shared logic from `@bytelyst/palace`: + +```typescript +import { cosineSimilarity, topKByCosine } from '@bytelyst/palace'; +import { isContentDuplicate } from '@bytelyst/palace'; +import { computeDecayedRelevance } from '@bytelyst/palace'; +import type { BasePalaceMemoryDoc } from '@bytelyst/palace'; +``` + +All methods require `userId` + `productId` as first two params: + +- [ ] `ensureWing(userId, productId, workspaceId, name)` — upsert wing from workspace +- [ ] `getWing(userId, productId, wingId)` → `PalaceWingDoc | null` +- [ ] `listWings(userId, productId)` → `PalaceWingDoc[]` +- [ ] `deleteWing(userId, productId, wingId)` — cascade delete rooms, memories, tunnels, KG +- [ ] `ensureRoom(userId, productId, wingId, name, description?)` — upsert room +- [ ] `listRooms(userId, productId, wingId)` → `PalaceRoomDoc[]` +- [ ] `storeMemory(userId, productId, wingId, roomId, hall, content, sourceNoteId?, embedding?)` — create + dedup +- [ ] `searchSemantic(userId, productId, query, embedding, wingId?, limit)` → ranked memories +- [ ] `searchText(userId, productId, query, wingId?, limit)` → text-matched memories +- [ ] `searchHybrid(userId, productId, query, embedding, wingId?, limit)` → text candidates re-ranked by cosine +- [ ] `getWingSummary(userId, productId, wingId)` → rooms + memory counts +- [ ] `isNearDuplicate(userId, productId, roomId, hall, content, embedding, threshold)` → boolean +- [ ] `pruneOldMemories(userId, productId, wingId, olderThanDays, minRelevance)` → count deleted +- [ ] `decayRelevance(userId, productId, halfLifeDays)` → count updated +- [ ] `backfillEmbeddings(userId, productId)` → count embedded +- [ ] `healthCheck()` → `{ cosmos: boolean, llm: boolean }` + +**User isolation enforcement:** +```typescript +// PATTERN: Every function starts with userId + productId in filter +export async function searchSemantic( + userId: string, + productId: string, + query: string, + embedding: number[], + wingId?: string, + limit = 10, +): Promise { + const filter: FilterMap = { userId, productId }; + if (wingId) filter.wingId = wingId; + + // Fetch candidates, compute cosine similarity client-side + const candidates = await memoriesCollection().findMany({ + filter, + sort: { updatedAt: -1 }, + limit: limit * 5, // over-fetch for re-ranking + }); + + return candidates + .filter(m => m.embedding && m.embedding.length > 0) + .map(m => ({ doc: m, score: cosineSimilarity(embedding, m.embedding!) })) + .sort((a, b) => b.score - a.score) + .slice(0, limit) + .map(r => r.doc); +} +``` + +### N1.4 Memory Extraction (`modules/palace/extractor.ts`) + +Leverages `@bytelyst/palace` extraction helpers + `@bytelyst/llm` for LLM calls: + +```typescript +import { buildExtractionPrompt, parseExtractionResponse, regexFallbackExtraction } from '@bytelyst/palace'; +import { llm } from '../../lib/llm.js'; +import { config } from '../../lib/config.js'; + +export interface ExtractedMemory { + hall: HallType; + content: string; + roomSlug: string; + entities: string[]; +} + +export async function extractMemories( + noteBody: string, + noteTitle: string, + workspaceName: string, +): Promise { + const provider = llm(); + + // Use shared prompt builder from @bytelyst/palace + const prompt = buildExtractionPrompt(noteBody.slice(0, 6000), { + title: noteTitle, + context: workspaceName, + hallTypes: HALL_TYPES, + }); + + try { + const result = await provider.chat({ + model: config.LLM_DEFAULT_MODEL, + messages: [{ role: 'user', content: prompt }], + temperature: 0.1, + }); + // Use shared response parser from @bytelyst/palace + return parseExtractionResponse(result.content); + } catch { + // Shared regex fallback from @bytelyst/palace + return regexFallbackExtraction(noteBody); + } +} +``` + +**Regex fallback** extracts: +- Lines starting with "Decision:", "TODO:", "Note:" → appropriate halls +- Markdown headers → room slugs +- `@mentions` and `#tags` → entities + +### N1.5 Deduplication + +Uses `@bytelyst/palace` dedup logic with Cosmos-backed candidate fetching: + +```typescript +import { isContentDuplicate } from '@bytelyst/palace'; + +export async function isNearDuplicate( + userId: string, productId: string, + roomId: string, hall: HallType, + content: string, embedding: number[] | null, + threshold = 0.90, +): Promise { + // 1. Exact match via Cosmos count (cheap) + const exactCount = await memoriesCollection().count({ userId, productId, roomId, hall, content }); + if (exactCount > 0) return true; + + // 2. Semantic dedup via shared @bytelyst/palace helper + if (embedding) { + const candidates = await memoriesCollection().findMany({ + filter: { userId, productId, roomId, hall }, + sort: { createdAt: -1 }, + limit: 20, + }); + // isContentDuplicate is pure — from @bytelyst/palace + return isContentDuplicate(embedding, candidates.map(c => c.embedding).filter(Boolean) as number[][], threshold); + } + return false; +} +``` + +### N1.6 Field Encryption for Memories + +Sensitive memory content encrypted at rest, matching the existing pattern: + +```typescript +const ENCRYPT_CONTEXT = 'palace-memory'; + +async function encryptMemoryContent(doc: PalaceMemoryDoc): Promise { + const enc = getEncryptor(); + const ctx = { userId: doc.userId, context: ENCRYPT_CONTEXT }; + const encrypted = await enc.encrypt(doc.content, ctx); + return { ...doc, content: encrypted as unknown as string }; +} +``` + +### N1.7 Tests + +- [ ] Wing CRUD with user isolation (user A can't see user B's wings) +- [ ] Room CRUD scoped to wing + user +- [ ] Memory store with deduplication +- [ ] Semantic search returns relevant results ranked by cosine similarity +- [ ] Text search returns text-matched results +- [ ] Hybrid search combines text + semantic +- [ ] Near-duplicate detection (exact + cosine) +- [ ] Field encryption round-trip for memory content +- [ ] Prune removes old low-relevance memories only for the requesting user +- [ ] Relevance decay applies exponential reduction +- [ ] Wing deletion cascades to rooms, memories, tunnels, KG +- [ ] Cross-user isolation: user A's search never returns user B's memories +- [ ] productId scoping: wrong productId returns empty results + +--- + +## Phase N2 — Wake-Up Context + +**Goal:** When a user starts working in a workspace, provide AI with relevant memory context from the palace. + +**Estimated: 3 days | New files: 1 | Tests: ~12** + +### N2.1 Wake-Up Context Builder (`modules/palace/wakeup.ts`) + +```typescript +export interface WakeUpContext { + l0Identity: string; // ~50 tokens: workspace name, purpose + l1Facts: string; // ~150 tokens: recent decisions, preferences + l2Context: string; // ~400 tokens: task-relevant memories + totalTokens: number; +} + +export async function buildWakeUpContext( + userId: string, + productId: string, + wingId: string, + taskDescription?: string, +): Promise { /* ... */ } +``` + +**L0 — Identity** (~50 tokens): +- Workspace name and description (from wing → workspace) +- Detected topic domains from room names + +**L1 — Critical Facts** (~150 tokens): +- Recent decisions and preferences from palace (last 30 days) +- Current KG triples for this wing (if M3 built) +- Cached summary, regenerated after each note save + +**L2 — Task Context** (~400 tokens): +- If `taskDescription` provided: semantic search against palace memories +- Falls back to most recent memories across all halls + +### N2.2 Integration Points + +Wake-up context is injected into: +1. **Smart Actions** — `note-prompts/runner.ts` prepends palace context to system prompt +2. **MCP tool calls** — `mempalace_wake_up` returns context for external agents +3. **Note Copilot** — (future) AI writing assistant uses palace context + +### N2.3 L1 Cache Regeneration + +After each note create/update that triggers memory extraction: +```typescript +// Rebuild L1 facts cache for the wing +await regenerateCriticalFacts(userId, productId, wingId); +// 1. Pull recent memories from decisions + preferences + insights halls +// 2. Pull current KG triples (if N3 built, otherwise skip) +// 3. Produce compact ~150 token summary +// 4. Cache in wing doc (PalaceWingDoc.l1Cache field) for fast wake-up +``` + +### N2.4 Tests + +- [ ] Wake-up context includes workspace identity (L0) +- [ ] L1 facts reflect recent decisions and preferences +- [ ] L2 context is semantically relevant to task description +- [ ] L2 falls back to recent memories when no task description +- [ ] Total tokens stay within ~600 budget +- [ ] Wake-up for user A never includes user B's memories +- [ ] Empty palace returns minimal context gracefully +- [ ] L1 cache regeneration updates wing doc + +--- + +## Phase N3 — Knowledge Graph + +**Goal:** Extract entity-relationship triples from notes, building a temporal knowledge graph per user. + +**Estimated: 4 days | New files: 1 | Tests: ~16** + +### N3.1 KG Triple Extraction + +During memory extraction (N1.4), also extract entity triples: + +```typescript +export interface ExtractedTriple { + subject: string; // "React Router" + predicate: string; // "replaced_by" + object: string; // "TanStack Router" + confidence: number; +} + +// LLM prompt addition: +// "Also extract entity relationships as (subject, predicate, object) triples." +``` + +### N3.2 KG Repository Extensions + +Added to `modules/palace/repository.ts`: + +- [ ] `addTriple(userId, productId, wingId, subject, predicate, object, confidence)` — assert a fact +- [ ] `invalidateTriple(userId, productId, subject, predicate, object)` — mark fact as ended +- [ ] `queryEntity(userId, productId, entity, asOf?)` → all current triples about entity +- [ ] `queryRelation(userId, productId, subject, predicate)` → matching objects +- [ ] `timeline(userId, productId, entity)` → chronological story +- [ ] `contradictions(userId, productId, wingId?)` → conflicting current triples + +### N3.3 Auto-Link Notes via KG + +When two notes reference the same entity, suggest a note-relationship: +```typescript +// After KG extraction, check if entities appear in other notes' KG triples +// If so, suggest creating a note-relationship (existing module) +``` + +### N3.4 Tests + +- [ ] Triple CRUD with user isolation +- [ ] Temporal queries (point-in-time vs current) +- [ ] Entity timeline returns chronological order +- [ ] Contradiction detection finds conflicting facts +- [ ] Invalidated triples excluded from current queries +- [ ] KG auto-link suggests note relationships +- [ ] Cross-user: user A's KG never leaks to user B + +--- + +## Phase N4 — Auto-Save Hooks (Event-Driven Memory Capture) + +**Goal:** Automatically extract and store memories when notes change. + +**Estimated: 3 days | Modified files: 3 | Tests: ~10** + +### N4.1 Save Triggers + +| Trigger | Where | What Happens | +|---------|-------|-------------| +| **Note created** | `notes/routes.ts` POST | Extract memories + KG from note body | +| **Note updated** | `notes/routes.ts` PATCH | Incremental extraction from changed content | +| **Agent action completed** | `note-agent-actions/routes.ts` | Extract memories from agent output | +| **Prompt executed** | `note-prompts/routes.ts` run | Extract memories from prompt result | +| **Explicit store** | MCP `mempalace_store` tool | User/agent-directed memory storage | + +### N4.2 Post-Save Hook + +```typescript +// In notes/routes.ts, after successful create/update: +async function onNoteSaved(userId: string, productId: string, note: NoteDoc): Promise { + try { + const wing = await ensureWing(userId, productId, note.workspaceId, workspaceName); + const memories = await extractMemories(note.body, note.title, workspaceName); + + for (const mem of memories) { + const room = await ensureRoom(userId, productId, wing.id, mem.roomSlug); + const embedding = await embedText(mem.content); + const isDup = await isNearDuplicate(userId, productId, room.id, mem.hall, mem.content, embedding); + if (!isDup) { + await storeMemory(userId, productId, wing.id, room.id, mem.hall, mem.content, note.id, embedding); + } + } + + // Regenerate L1 cache for this wing + await regenerateCriticalFacts(userId, productId, wing.id); + } catch (err) { + // Best-effort — don't fail the note save + req.log.warn({ err }, 'palace: memory extraction failed'); + } +} +``` + +### N4.3 Tests + +- [ ] Note create triggers memory extraction +- [ ] Note update triggers incremental extraction +- [ ] Duplicate memories are skipped +- [ ] Extraction failure doesn't fail the note save +- [ ] Wing auto-created from workspace on first save +- [ ] Room auto-created from extracted topic +- [ ] Memories include sourceNoteId back-reference + +--- + +## Phase N5 — Palace API Routes + +**Goal:** REST endpoints for palace operations, secured with JWT auth. + +**Estimated: 3 days | New files: 1 | Tests: ~14** + +### N5.1 Routes (`modules/palace/routes.ts`) + +All routes extract `userId` from JWT via `getUserId(req)`: + +```typescript +// Palace search +GET /api/palace/search?q=...&wingId=...&limit=10 + +// Wings +GET /api/palace/wings +GET /api/palace/wings/:wingId +DELETE /api/palace/wings/:wingId + +// Rooms +GET /api/palace/wings/:wingId/rooms + +// Memories +POST /api/palace/memories // explicit store +GET /api/palace/memories?wingId=...&roomId=...&hall=... +DELETE /api/palace/memories/:id + +// Knowledge Graph +GET /api/palace/kg/entity/:entity +GET /api/palace/kg/timeline/:entity +GET /api/palace/kg/contradictions + +// Wake-up context +GET /api/palace/wake-up/:wingId?task=... + +// Maintenance +POST /api/palace/backfill-embeddings +POST /api/palace/prune +GET /api/palace/health +GET /api/palace/stats +``` + +### N5.2 Zod Schemas + +```typescript +export const PalaceSearchQuerySchema = z.object({ + q: z.string().min(1).max(500), + wingId: z.string().max(128).optional(), + roomId: z.string().max(128).optional(), + hall: z.enum(HALL_TYPES).optional(), + limit: z.coerce.number().int().min(1).max(50).default(10), +}); + +export const StoreMemorySchema = z.object({ + wingId: z.string().min(1).max(128), + roomId: z.string().min(1).max(128), + hall: z.enum(HALL_TYPES), + content: z.string().min(1).max(5000), + sourceNoteId: z.string().max(128).optional(), +}); +``` + +### N5.3 Tests + +- [ ] Search returns user-scoped results +- [ ] Wings list only returns requesting user's wings +- [ ] Explicit memory store persists and is immediately searchable +- [ ] Delete wing cascades correctly +- [ ] Wake-up context returns structured L0+L1+L2 +- [ ] KG entity query returns user-scoped triples +- [ ] Stats endpoint returns accurate counts per user +- [ ] Unauthenticated requests return 401 +- [ ] Wrong productId returns empty/403 + +--- + +## Phase N6 — MCP Memory Tools + +**Goal:** Extend NoteLett's existing MCP system with palace tools. + +**Estimated: 2 days | Modified files: 2 | New files: 1 | Tests: ~10** + +### N6.1 New MCP Tools + +Added to `mcp/` alongside existing note tools: + +| Tool | Description | +|------|-------------| +| `mempalace_search` | Semantic + text hybrid search across user's palace | +| `mempalace_store` | Explicitly store a memory (agent-directed) | +| `mempalace_wake_up` | Get wake-up context for a workspace/wing | +| `mempalace_query_entity` | Query KG triples about an entity | +| `mempalace_timeline` | Chronological story of an entity | +| `mempalace_list_wings` | List user's palace wings with stats | + +### N6.2 User Isolation in MCP + +Same pattern as existing MCP tools: + +```typescript +async function executeMempalaceSearch(args: MempalaceSearchInput, req: NotesMcpRequest) { + const userId = requireUserId(req); // JWT-enforced + const productId = PRODUCT_ID; + return searchHybrid(userId, productId, args.query, embedding, args.wingId, args.limit); +} +``` + +### N6.3 Tests + +- [ ] Each MCP tool returns valid responses +- [ ] `mempalace_search` returns ranked results +- [ ] `mempalace_store` persists and is searchable +- [ ] `mempalace_wake_up` returns structured context +- [ ] MCP tools enforce userId from JWT (user A can't query user B) +- [ ] Graceful error when palace is empty + +--- + +## Phase N7 — Web UI Integration + +**Goal:** Surface palace data in the NoteLett web dashboard. + +**Estimated: 4 days | New files: 4 React | Tests: ~8** + +### N7.1 Web API Client + +New file: `web/src/lib/palace-client.ts` + +```typescript +export async function searchPalace(query: string, wingId?: string): Promise; +export async function listWings(): Promise; +export async function getEntityTimeline(entity: string): Promise; +export async function getWakeUpContext(wingId: string): Promise; +export async function getPalaceStats(): Promise; +``` + +### N7.2 React Components + +- **PalacePanel** — sidebar tab showing palace search + recent memories +- **KnowledgeGraphView** — entity-relationship visualization for a workspace +- **MemoryTimeline** — chronological view of extracted memories +- **PalaceStats** — dashboard card showing wing counts, memory counts, KG size + +### N7.3 Integration with Existing UI + +- **Note detail page** — show "Related Memories" panel from palace search +- **Workspace page** — show wing stats + recent memories +- **Search page** — add "Search Palace" tab alongside note search + +### N7.4 Tests + +- [ ] Palace search renders results +- [ ] Wings list shows user's workspaces as wings +- [ ] Knowledge graph renders entity nodes +- [ ] Empty state renders gracefully + +--- + +## Summary + +| Phase | Description | Days | Tests | Depends On | +|-------|-------------|------|-------|------------| +| **N0** | `@bytelyst/palace` shared package in common-plat (types, cosine, dedup, decay, extraction, KG, wakeup) — d1c6cf4 | 3-4 | 91 ✅ | — | +| **N1** | Palace Core (Cosmos containers, repository, extraction, dedup, encryption) — consumes `@bytelyst/palace` | 4-5 | ~30 | N0 | +| **N2** | Wake-Up Context (L0/L1/L2 at ~600 tokens) | 3 | ~12 | N1 | +| **N3** | Knowledge Graph (temporal entity triples) | 4 | ~16 | N1 | +| **N4** | Auto-Save Hooks (note create/update/agent action triggers) | 3 | ~10 | N1 | +| **N5** | Palace API Routes (REST endpoints + Zod validation) | 3 | ~14 | N1, N2, N3 | +| **N6** | MCP Memory Tools (6 `mempalace_*` tools) | 2 | ~10 | N1, N5 | +| **N7** | Web UI (palace panel, KG view, timeline, stats) | 4 | ~8 | N5 | +| **Total** | | **27-32 days** | **~125** | | + +### New Dependencies (in NoteLett) + +```json +"@bytelyst/palace": "workspace:*" +``` + +(Added to `backend/package.json` — consumed alongside existing `@bytelyst/datastore`, `@bytelyst/llm`, `@bytelyst/field-encrypt`, etc.) + +### New Cosmos Containers + +| Container | Partition Key | Purpose | +|-----------|--------------|---------| +| `palace_wings` | `/userId` | Per-user workspace→wing mapping | +| `palace_rooms` | `/userId` | Topic rooms within wings | +| `palace_memories` | `/userId` | Core memory storage with embeddings | +| `palace_tunnels` | `/userId` | Cross-room links | +| `palace_kg` | `/userId` | Knowledge graph triples | +| `palace_diaries` | `/userId` | Role-scoped diary entries | + +### New Files + +``` +# ── In learning_ai_common_plat (Phase N0) ────────────────────── +packages/palace/ +├── src/ +│ ├── index.ts, types.ts, halls.ts, cosine.ts, dedup.ts, +│ ├── decay.ts, extraction.ts, kg.ts, wakeup.ts, config.ts +│ └── __tests__/ (6 test files) +├── package.json # @bytelyst/palace +└── tsconfig.json + +# ── In learning_ai_notes (Phases N1-N7) ──────────────────────── +backend/src/modules/palace/ +├── types.ts # NoteLett-specific palace types (extends @bytelyst/palace base types) +├── repository.ts # User-scoped CRUD + search (uses @bytelyst/palace cosine, dedup, decay) +├── extractor.ts # LLM extraction (uses @bytelyst/palace prompts + @bytelyst/llm) +├── wakeup.ts # Wake-up context (uses @bytelyst/palace wakeup builder) +├── routes.ts # Fastify REST routes (uses @bytelyst/fastify-auth for user isolation) +└── palace.test.ts # Vitest tests (uses @bytelyst/testing helpers) + +backend/src/mcp/ +├── palace-tool-contracts.ts # Zod schemas for MCP tools +└── palace-tools.ts # 6 MCP tool implementations + +web/src/lib/ +└── palace-client.ts # Typed API client (uses @bytelyst/api-client) + +web/src/components/ +├── PalacePanel.tsx # Sidebar palace search + memories (uses @bytelyst/ui) +├── KnowledgeGraphView.tsx +├── MemoryTimeline.tsx +└── PalaceStats.tsx +``` + +### New Config Variables + +| Variable | Default | Description | +|----------|---------|-------------| +| `PALACE_ENABLED` | `true` | Enable/disable palace feature | +| `PALACE_EXTRACTION_ENABLED` | `true` | Enable auto-extraction on note save | +| `PALACE_WAKE_UP_BUDGET` | `600` | Max tokens for wake-up context | +| `PALACE_RELEVANCE_HALF_LIFE_DAYS` | `90` | Days for relevance to halve | +| `PALACE_DEDUP_THRESHOLD` | `0.90` | Cosine similarity threshold for dedup | + +### Critical Path + +``` +N1 (Palace Core + extraction + dedup) + ├──→ N2 (Wake-Up) ──→ N5 (API Routes) ──→ N6 (MCP Tools) + ├──→ N3 (Knowledge Graph) ──→ N5 ↘ + ├──→ N4 (Auto-Save) — independent after N1 N7 (Web UI) + └──→ N6, N7 — last, needs N5 + +Note: N3 is optional for N2 (L1 works without KG; triples added once N3 is done). +N4 can be built in parallel with N2/N3. +``` + +**Recommended execution order:** N1 → N4 → N2 → N3 → N5 → N6 → N7 + +This gives auto-saving memories on note create/update after N1+N4 (~9 days), with semantic wake-up and KG following. + +--- + +## User Isolation Checklist + +Every phase must satisfy these checks before merging: + +- [ ] **Every new Cosmos container** uses `/userId` as partition key +- [ ] **Every repository function** takes `userId` as first parameter +- [ ] **Every query filter** includes `userId` — no exceptions +- [ ] **Every route handler** extracts userId via `getUserId(req)` — never from request body/params +- [ ] **Every test file** includes at least one cross-user isolation test +- [ ] **No endpoint** accepts userId as a URL parameter or request body field +- [ ] **Field encryption** uses `userId` in encryption context +- [ ] **MCP tools** use `requireUserId(req)` guard (from existing pattern)