learning_ai_notes/docs/MEMPALACE_INTEGRATION_ROADMAP.md

# MemPalace Integration Roadmap — NoteLett

> **Goal:** Add persistent, structured semantic memory to NoteLett so users build a personal knowledge palace — notes auto-organized into wings (workspaces), rooms (topics), and halls (memory types) — with cross-note knowledge graph, semantic search, and AI-assisted recall. All with **strict user isolation** at every layer.
>
> **Ref:** [MemPalace](https://github.com/milla-jovovich/mempalace) — highest-scoring AI memory system (96.6% LongMemEval). Adapted for a multi-user Fastify/Cosmos backend.

---

## Design Decisions

| # | Decision | Choice | Rationale |
|---|----------|--------|-----------|
| 0 | **Shared package** | New `@bytelyst/palace` in `learning_ai_common_plat/packages/palace/` | Core palace types, cosine similarity, dedup logic, relevance decay, KG helpers, extraction prompts, and wake-up builder are product-agnostic — shared across NoteLett, MindLyst, and future products |
| 1 | **Storage** | Cosmos DB only (via `@bytelyst/datastore`) | NoteLett is already Cosmos-native; no SQLite or ChromaDB — keeps infra simple, user isolation inherits from Cosmos partition keys |
| 2 | **Vector search** | Cosmos DB vector index (DiskANN) or in-memory cosine over stored embeddings | Avoids introducing ChromaDB; Cosmos vector search is GA; falls back to `cosineSimilarity()` from existing `embeddings.ts` |
| 3 | **Embedding model** | `@bytelyst/llm` embed() — `text-embedding-3-small` (1536-dim) | Already configured in `config.ts` (`LLM_EMBEDDING_MODEL`), existing `embedText()` in `lib/embeddings.ts` |
| 4 | **Extraction** | `@bytelyst/llm` chat() — `gpt-4o-mini` with structured output + regex fallback | Same LLM infrastructure as Smart Actions; mock provider for tests |
| 5 | **User isolation** | `userId` required on every document and every query filter, enforced at repository layer | Same pattern as all existing NoteLett modules (notes, workspaces, etc.) |
| 6 | **Palace scope** | Per-user palace, workspace = wing | Each user has their own palace; their NoteLett workspaces map to wings |
| 7 | **Hall types** | 6 halls: `decisions`, `events`, `discoveries`, `preferences`, `advice`, `insights` | `insights` replaces `errors` (coding-focused); captures AI-generated observations from note analysis |
| 8 | **Wake-up budget** | ~600 tokens (L0:50 + L1:150 + L2:400) | Enough for note-writing context; smaller than Claw-Cowork's 800 since notes have narrower scope |
| 9 | **MCP integration** | Extend existing MCP tools in `backend/src/mcp/` | NoteLett already has 8 MCP tools; add `mempalace_*` tools to the same system |
| 10 | **Save triggers** | Note create/update, agent action completion, prompt execution, explicit store | Event-driven based on NoteLett's existing lifecycle hooks |
| 11 | **Field encryption** | Sensitive memory content encrypted via `@bytelyst/field-encrypt` | Mirrors existing note body encryption pattern in `notes/repository.ts` |
| 12 | **productId scoping** | Every palace document includes `productId: "notelett"` | Standard ByteLyst ecosystem convention |

---

## @bytelyst/* Package Dependency Map

NoteLett's palace module leverages these existing shared packages:

| Package | Usage in Palace |
|---------|----------------|
| `@bytelyst/datastore` | All palace Cosmos CRUD via `getCollection()` |
| `@bytelyst/field-encrypt` | Encrypt memory content at rest (AES-256-GCM) |
| `@bytelyst/llm` | `embed()` for embeddings, `chat()` for extraction |
| `@bytelyst/backend-config` | Extend `baseBackendConfigSchema` with palace env vars |
| `@bytelyst/fastify-auth` | `getUserId(req)` for JWT-based user isolation |
| `@bytelyst/fastify-core` | Route registration via `createServiceApp()` |
| `@bytelyst/errors` | Typed errors: `NotFoundError`, `BadRequestError` |
| `@bytelyst/cosmos` | `registerContainers()` for palace containers |
| `@bytelyst/extraction` | `createExtractionClient()` for entity extraction via extraction-service |
| `@bytelyst/testing` | Test helpers for `buildTestApp()` |
| **`@bytelyst/palace`** | **NEW** — shared palace types, cosine, dedup, decay, KG helpers, extraction prompts, wake-up builder |

---

## Current State (What Exists)

| Component | File | What It Does |
|-----------|------|-------------|
| **Notes CRUD** | `modules/notes/repository.ts` | Note storage with field encryption, userId filtering |
| **Workspaces** | `modules/workspaces/` | Workspace CRUD — maps to MemPalace "wings" |
| **Note Relationships** | `modules/note-relationships/` | Manual note linking — related to MemPalace "tunnels" |
| **Embeddings** | `lib/embeddings.ts` | `embedText()`, `cosineSimilarity()`, `stripHtmlForEmbedding()` — already exists |
| **LLM** | `lib/llm.ts` | `@bytelyst/llm` singleton — embed + chat capability |
| **Field Encrypt** | `lib/field-encrypt.ts` | `@bytelyst/field-encrypt` for sensitive data at rest |
| **Request Context** | `lib/request-context.ts` | `getUserId(req)`, `getRequestProductId(req)` — JWT-extracted |
| **MCP Tools** | `mcp/note-tools.ts` | 8 MCP tools with `requireUserId()` + `requireProductScope()` guards |
| **Smart Actions** | `modules/note-prompts/` | LLM-powered transforms — extraction patterns reusable for memory mining |

### Key Gap

Notes exist in isolation. No cross-note semantic memory, no knowledge graph connecting entities, no AI-assisted recall of past decisions. Users can search by text but not by meaning. Relationships are manual, not auto-discovered.

---

## Architecture

```
┌──────────────────────────────────────────────────────────┐
│                    NoteLett Backend                        │
│                                                          │
│  ┌─────────────┐  ┌──────────────┐  ┌────────────────┐  │
│  │ Notes Module │  │ Palace Module│  │  MCP Tools     │  │
│  │ (existing)   │  │ (NEW)        │  │  (extended)    │  │
│  └──────┬───────┘  └──────┬───────┘  └───────┬────────┘  │
│         │                 │                   │           │
│  ┌──────┴─────────────────┴───────────────────┴────────┐ │
│  │              @bytelyst/datastore                      │ │
│  │   getCollection() — userId in every filter            │ │
│  └──────────────────────┬───────────────────────────────┘ │
│                         │                                  │
│  ┌──────────────────────┴───────────────────────────────┐ │
│  │              Cosmos DB (partition: /userId)            │ │
│  │                                                       │ │
│  │  ┌────────────┐ ┌───────────┐ ┌──────────────────┐   │ │
│  │  │palace_wings│ │palace_rooms│ │palace_memories    │   │ │
│  │  │/userId     │ │/userId    │ │/userId (+ vector) │   │ │
│  │  └────────────┘ └───────────┘ └──────────────────┘   │ │
│  │  ┌────────────┐ ┌───────────┐ ┌──────────────────┐   │ │
│  │  │palace_kg   │ │palace_    │ │palace_tunnels     │   │ │
│  │  │/userId     │ │diaries    │ │/userId            │   │ │
│  │  │            │ │/userId    │ │                    │   │ │
│  │  └────────────┘ └───────────┘ └──────────────────┘   │ │
│  └───────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────┘
```

### User Isolation Model

**Every palace operation is scoped to the authenticated user.** Isolation is enforced at 3 layers:

1. **JWT Auth Layer** — `getUserId(req)` extracts userId from the verified JWT; no user input accepted for identity
2. **Repository Layer** — Every query includes `userId` in its filter. No query can omit it.
3. **Cosmos Partition Layer** — All palace containers use `/userId` as partition key. Cross-partition queries are physically impossible without the partition key.

```typescript
// EVERY repository function signature includes userId — no exceptions:
export async function searchMemories(userId: string, productId: string, query: SearchQuery): Promise<PalaceMemoryDoc[]> {
  const filter: FilterMap = { userId, productId };
  // ... userId is ALWAYS in the filter
}

// Route handlers extract userId from JWT — never from request body:
fastify.get('/api/palace/search', async (req, reply) => {
  const userId = getUserId(req);           // from JWT
  const productId = getRequestProductId(req); // from JWT
  const query = SearchQuerySchema.parse(req.query);
  const results = await searchMemories(userId, productId, query);
  // ...
});
```

### Workspace-to-Wing Mapping

NoteLett workspaces map 1:1 to palace wings:

| NoteLett Concept | Palace Concept | Relationship |
|---|---|---|
| Workspace | Wing | `wing.sourceWorkspaceId = workspace.id` |
| Note tags | Rooms | Auto-created from note tags or AI-suggested topics |
| Note relationships | Tunnels | Existing links become tunnel seeds |
| Note body | Memory content | Extracted decisions, facts, insights from notes |
| Smart Action output | Memory content | Agent-generated memories from prompt executions |

---

## Phase N0 — Shared Palace Package (`@bytelyst/palace`)

**Goal:** Create a reusable shared package in `learning_ai_common_plat/packages/palace/` with product-agnostic palace primitives.

**Estimated: 3-4 days | New files: 8 | Tests: ~25**

> **This phase is done in the `learning_ai_common_plat` repo, not in NoteLett.**

### N0.1 Package Scaffold

New package: `packages/palace/` (`@bytelyst/palace`)

```
packages/palace/
├── src/
│   ├── index.ts            # Public exports
│   ├── types.ts            # PalaceWingDoc, PalaceRoomDoc, PalaceMemoryDoc, PalaceTunnelDoc, PalaceKGTripleDoc, PalaceDiaryDoc
│   ├── halls.ts            # HallType union, HALL_PRESETS (notelett, mindlyst, coding), hallFromLabel()
│   ├── cosine.ts           # cosineSimilarity(), topKByCosine(), normalizeVector()
│   ├── dedup.ts            # isNearDuplicate() — exact match + cosine threshold logic
│   ├── decay.ts            # decayRelevance(), computeDecayedRelevance() — exponential half-life
│   ├── extraction.ts       # buildExtractionPrompt(), parseExtractionResponse(), regexFallbackExtraction()
│   ├── kg.ts               # KG helper types, contradictionCheck(), mergeTriples()
│   ├── wakeup.ts           # WakeUpContext type, buildWakeUpLayers(), truncateToTokenBudget()
│   ├── config.ts           # palaceConfigSchema (Zod) — PALACE_ENABLED, budgets, thresholds
│   └── __tests__/
│       ├── cosine.test.ts
│       ├── dedup.test.ts
│       ├── decay.test.ts
│       ├── extraction.test.ts
│       ├── kg.test.ts
│       └── wakeup.test.ts
├── package.json            # @bytelyst/palace, peerDeps: @bytelyst/llm, zod
└── tsconfig.json
```

### N0.2 Shared Types (`types.ts`)

Product-agnostic base types — products extend with their own fields:

```typescript
import type { BaseDocument } from '@bytelyst/datastore';

// Base types — every product extends these
export interface BasePalaceWingDoc extends BaseDocument {
  userId: string;
  productId: string;
  name: string;
  memoryCount: number;
  l1Cache?: string;
  l1CacheUpdatedAt?: string;
  createdAt: string;
  updatedAt: string;
}

export interface BasePalaceMemoryDoc extends BaseDocument {
  userId: string;
  productId: string;
  wingId: string;
  roomId: string;
  hall: string;
  content: string;
  relevance: number;
  embedding?: number[];
  createdAt: string;
  updatedAt: string;
}

// ... similar for Room, Tunnel, KGTriple, Diary
```

### N0.3 Cosine Similarity (`cosine.ts`)

Extracted from NoteLett's `embeddings.ts` — shared by all products:

```typescript
export function cosineSimilarity(a: number[], b: number[]): number;
export function topKByCosine<T>(query: number[], items: T[], getEmbedding: (t: T) => number[] | undefined, k: number): Array<{ item: T; score: number }>;
export function normalizeVector(v: number[]): number[];
```

### N0.4 Dedup, Decay, Extraction, KG, Wake-Up

All product-agnostic algorithms extracted into pure functions:

- **dedup.ts** — `isContentDuplicate(content, candidates, threshold)` (exact + cosine)
- **decay.ts** — `computeDecayedRelevance(originalRelevance, createdAt, halfLifeDays)` (exponential)
- **extraction.ts** — `buildExtractionPrompt(content, context, hallTypes)`, `parseExtractionResponse(llmOutput)`, `regexFallbackExtraction(content)`
- **kg.ts** — `findContradictions(triples)`, `mergeTriples(existing, incoming)`, `isTripleCurrent(triple, asOf?)`
- **wakeup.ts** — `buildWakeUpLayers(l0, l1, l2, budget)`, `truncateToTokenBudget(text, maxTokens)`
- **config.ts** — `palaceConfigSchema` (Zod) extending `baseBackendConfigSchema` with `PALACE_ENABLED`, `PALACE_WAKE_UP_BUDGET`, `PALACE_DEDUP_THRESHOLD`, `PALACE_RELEVANCE_HALF_LIFE_DAYS`

### N0.5 Tests

- [x] d1c6cf4 Cosine similarity correctness (identical=1.0, orthogonal=0.0)
- [x] d1c6cf4 topKByCosine returns sorted results, handles missing embeddings
- [x] d1c6cf4 Dedup detects exact matches and near-duplicates above threshold
- [x] d1c6cf4 Decay reduces relevance exponentially by half-life
- [x] d1c6cf4 Extraction prompt builder produces valid LLM prompts for all hall presets
- [x] d1c6cf4 Extraction response parser handles valid JSON, malformed JSON, empty
- [x] d1c6cf4 Regex fallback extracts decisions, events from markdown
- [x] d1c6cf4 KG contradiction detection finds conflicting triples
- [x] d1c6cf4 Wake-up layer builder respects token budget
- [x] d1c6cf4 Palace config schema validates all env vars with defaults

---

## Phase N1 — Palace Core Module

**Goal:** New `palace` module with Cosmos-backed storage, user-isolated CRUD, and semantic search — consuming `@bytelyst/palace` for shared logic.

**Estimated: 4-5 days | New files: 4 | Tests: ~30**

### N1.1 Palace Cosmos Containers

New containers in `cosmos-init.ts`:

```typescript
// Add to CONTAINER_DEFS:
palace_wings:    { partitionKeyPath: '/userId' },
palace_rooms:    { partitionKeyPath: '/userId' },
palace_memories: { partitionKeyPath: '/userId' },
palace_tunnels:  { partitionKeyPath: '/userId' },
palace_kg:       { partitionKeyPath: '/userId' },
palace_diaries:  { partitionKeyPath: '/userId' },
```

### N1.2 Palace Types (`modules/palace/types.ts`)

```typescript
export interface PalaceWingDoc {
  id: string;
  productId: string;
  userId: string;
  name: string;                    // e.g. "Work Notes"
  sourceWorkspaceId: string;       // links to NoteLett workspace
  techStack?: string;              // auto-detected from note content
  memoryCount: number;             // denormalized for fast display
  createdAt: string;
  updatedAt: string;
}

export interface PalaceRoomDoc {
  id: string;
  productId: string;
  userId: string;
  wingId: string;
  name: string;                    // e.g. "auth-migration"
  description?: string;
  memoryCount: number;
  createdAt: string;
}

export const HALL_TYPES = ['decisions', 'events', 'discoveries', 'preferences', 'advice', 'insights'] as const;
export type HallType = (typeof HALL_TYPES)[number];

export interface PalaceMemoryDoc {
  id: string;
  productId: string;
  userId: string;
  wingId: string;
  roomId: string;
  hall: HallType;
  content: string;                 // the memory text
  sourceNoteId?: string;           // which note produced this memory
  sourceTaskId?: string;           // which agent action produced this
  relevance: number;               // 0.0–1.0, decays over time
  embedding?: number[];            // 1536-dim from text-embedding-3-small
  createdAt: string;
  updatedAt: string;
}

export interface PalaceTunnelDoc {
  id: string;
  productId: string;
  userId: string;
  roomA: string;                   // normalized: roomA < roomB
  roomB: string;
  reason: string;
  strength: number;                // 0.0–1.0
  createdAt: string;
}

export interface PalaceKGTripleDoc {
  id: string;
  productId: string;
  userId: string;
  wingId: string;
  subject: string;
  predicate: string;
  object: string;
  validFrom: string;
  validTo?: string;                // null = still current
  sourceTaskId?: string;
  confidence: number;
  createdAt: string;
}

export interface PalaceDiaryDoc {
  id: string;
  productId: string;
  userId: string;
  roleId: string;                  // e.g. "note-copilot"
  wingId?: string;
  entry: string;
  createdAt: string;
}
```

### N1.3 Palace Repository (`modules/palace/repository.ts`)

Imports shared logic from `@bytelyst/palace`:

```typescript
import { cosineSimilarity, topKByCosine } from '@bytelyst/palace';
import { isContentDuplicate } from '@bytelyst/palace';
import { computeDecayedRelevance } from '@bytelyst/palace';
import type { BasePalaceMemoryDoc } from '@bytelyst/palace';
```

All methods require `userId` + `productId` as first two params:

- [x] 44d8867 `ensureWing(userId, productId, workspaceId, name)` — upsert wing from workspace
- [x] 44d8867 `getWing(userId, productId, wingId)` → `PalaceWingDoc | null`
- [x] 44d8867 `listWings(userId, productId)` → `PalaceWingDoc[]`
- [x] 44d8867 `deleteWing(userId, productId, wingId)` — cascade delete rooms, memories, tunnels, KG
- [x] 44d8867 `ensureRoom(userId, productId, wingId, name, description?)` — upsert room
- [x] 44d8867 `listRooms(userId, productId, wingId)` → `PalaceRoomDoc[]`
- [x] 44d8867 `storeMemory(userId, productId, wingId, roomId, hall, content, sourceNoteId?, embedding?)` — create + dedup
- [x] 44d8867 `searchSemantic(userId, productId, query, embedding, wingId?, limit)` → ranked memories
- [x] 44d8867 `searchText(userId, productId, query, wingId?, limit)` → text-matched memories
- [x] 44d8867 `searchHybrid(userId, productId, query, embedding, wingId?, limit)` → text candidates re-ranked by cosine
- [x] 44d8867 `getWingSummary(userId, productId, wingId)` → rooms + memory counts
- [x] 44d8867 `isNearDuplicate(userId, productId, roomId, hall, content, embedding, threshold)` → boolean
- [x] 44d8867 `pruneOldMemories(userId, productId, wingId, olderThanDays, minRelevance)` → count deleted
- [x] 44d8867 `decayRelevance(userId, productId, halfLifeDays)` → count updated
- [x] 44d8867 `backfillEmbeddings(userId, productId)` → count embedded
- [x] 44d8867 `healthCheck()` → `{ cosmos: boolean, llm: boolean }`

**User isolation enforcement:**
```typescript
// PATTERN: Every function starts with userId + productId in filter
export async function searchSemantic(
  userId: string,
  productId: string,
  query: string,
  embedding: number[],
  wingId?: string,
  limit = 10,
): Promise<PalaceMemoryDoc[]> {
  const filter: FilterMap = { userId, productId };
  if (wingId) filter.wingId = wingId;

  // Fetch candidates, compute cosine similarity client-side
  const candidates = await memoriesCollection().findMany({
    filter,
    sort: { updatedAt: -1 },
    limit: limit * 5, // over-fetch for re-ranking
  });

  return candidates
    .filter(m => m.embedding && m.embedding.length > 0)
    .map(m => ({ doc: m, score: cosineSimilarity(embedding, m.embedding!) }))
    .sort((a, b) => b.score - a.score)
    .slice(0, limit)
    .map(r => r.doc);
}
```

### N1.4 Memory Extraction (`modules/palace/extractor.ts`)

Leverages `@bytelyst/palace` extraction helpers + `@bytelyst/llm` for LLM calls:

```typescript
import { buildExtractionPrompt, parseExtractionResponse, regexFallbackExtraction } from '@bytelyst/palace';
import { llm } from '../../lib/llm.js';
import { config } from '../../lib/config.js';

export interface ExtractedMemory {
  hall: HallType;
  content: string;
  roomSlug: string;
  entities: string[];
}

export async function extractMemories(
  noteBody: string,
  noteTitle: string,
  workspaceName: string,
): Promise<ExtractedMemory[]> {
  const provider = llm();

  // Use shared prompt builder from @bytelyst/palace
  const prompt = buildExtractionPrompt(noteBody.slice(0, 6000), {
    title: noteTitle,
    context: workspaceName,
    hallTypes: HALL_TYPES,
  });

  try {
    const result = await provider.chat({
      model: config.LLM_DEFAULT_MODEL,
      messages: [{ role: 'user', content: prompt }],
      temperature: 0.1,
    });
    // Use shared response parser from @bytelyst/palace
    return parseExtractionResponse(result.content);
  } catch {
    // Shared regex fallback from @bytelyst/palace
    return regexFallbackExtraction(noteBody);
  }
}
```

**Regex fallback** extracts:
- Lines starting with "Decision:", "TODO:", "Note:" → appropriate halls
- Markdown headers → room slugs
- `@mentions` and `#tags` → entities

### N1.5 Deduplication

Uses `@bytelyst/palace` dedup logic with Cosmos-backed candidate fetching:

```typescript
import { isContentDuplicate } from '@bytelyst/palace';

export async function isNearDuplicate(
  userId: string, productId: string,
  roomId: string, hall: HallType,
  content: string, embedding: number[] | null,
  threshold = 0.90,
): Promise<boolean> {
  // 1. Exact match via Cosmos count (cheap)
  const exactCount = await memoriesCollection().count({ userId, productId, roomId, hall, content });
  if (exactCount > 0) return true;

  // 2. Semantic dedup via shared @bytelyst/palace helper
  if (embedding) {
    const candidates = await memoriesCollection().findMany({
      filter: { userId, productId, roomId, hall },
      sort: { createdAt: -1 },
      limit: 20,
    });
    // isContentDuplicate is pure — from @bytelyst/palace
    return isContentDuplicate(embedding, candidates.map(c => c.embedding).filter(Boolean) as number[][], threshold);
  }
  return false;
}
```

### N1.6 Field Encryption for Memories

Sensitive memory content encrypted at rest, matching the existing pattern:

```typescript
const ENCRYPT_CONTEXT = 'palace-memory';

async function encryptMemoryContent(doc: PalaceMemoryDoc): Promise<PalaceMemoryDoc> {
  const enc = getEncryptor();
  const ctx = { userId: doc.userId, context: ENCRYPT_CONTEXT };
  const encrypted = await enc.encrypt(doc.content, ctx);
  return { ...doc, content: encrypted as unknown as string };
}
```

### N1.7 Tests

- [x] 632b5df Wing CRUD with user isolation (user A can't see user B's wings)
- [x] 632b5df Room CRUD scoped to wing + user
- [x] 632b5df Memory store with deduplication
- [x] 632b5df Semantic search returns relevant results ranked by cosine similarity
- [x] 632b5df Text search returns text-matched results
- [x] 632b5df Hybrid search combines text + semantic
- [x] 632b5df Near-duplicate detection (exact + cosine)
- [x] 632b5df Field encryption round-trip for memory content
- [x] 632b5df Prune removes old low-relevance memories only for the requesting user
- [x] 632b5df Relevance decay applies exponential reduction
- [x] 632b5df Wing deletion cascades to rooms, memories, tunnels, KG
- [x] 632b5df Cross-user isolation: user A's search never returns user B's memories
- [x] 632b5df productId scoping: wrong productId returns empty results

---

## Phase N2 — Wake-Up Context

**Goal:** When a user starts working in a workspace, provide AI with relevant memory context from the palace.

**Estimated: 3 days | New files: 1 | Tests: ~12**

### N2.1 Wake-Up Context Builder (`modules/palace/wakeup.ts`)

```typescript
export interface WakeUpContext {
  l0Identity: string;    // ~50 tokens: workspace name, purpose
  l1Facts: string;       // ~150 tokens: recent decisions, preferences
  l2Context: string;     // ~400 tokens: task-relevant memories
  totalTokens: number;
}

export async function buildWakeUpContext(
  userId: string,
  productId: string,
  wingId: string,
  taskDescription?: string,
): Promise<WakeUpContext> { /* ... */ }
```

**L0 — Identity** (~50 tokens):
- Workspace name and description (from wing → workspace)
- Detected topic domains from room names

**L1 — Critical Facts** (~150 tokens):
- Recent decisions and preferences from palace (last 30 days)
- Current KG triples for this wing (if M3 built)
- Cached summary, regenerated after each note save

**L2 — Task Context** (~400 tokens):
- If `taskDescription` provided: semantic search against palace memories
- Falls back to most recent memories across all halls

### N2.2 Integration Points

Wake-up context is injected into:
1. **Smart Actions** — `note-prompts/runner.ts` prepends palace context to system prompt
2. **MCP tool calls** — `mempalace_wake_up` returns context for external agents
3. **Note Copilot** — (future) AI writing assistant uses palace context

### N2.3 L1 Cache Regeneration

After each note create/update that triggers memory extraction:
```typescript
// Rebuild L1 facts cache for the wing
await regenerateCriticalFacts(userId, productId, wingId);
// 1. Pull recent memories from decisions + preferences + insights halls
// 2. Pull current KG triples (if N3 built, otherwise skip)
// 3. Produce compact ~150 token summary
// 4. Cache in wing doc (PalaceWingDoc.l1Cache field) for fast wake-up
```

### N2.4 Tests

- [x] a5dbeac Wake-up context includes workspace identity (L0)
- [x] a5dbeac L1 facts reflect recent decisions and preferences
- [x] a5dbeac L2 context is semantically relevant to task description
- [x] a5dbeac L2 falls back to recent memories when no task description
- [x] a5dbeac Total tokens stay within ~600 budget
- [x] a5dbeac Wake-up for user A never includes user B's memories
- [x] a5dbeac Empty palace returns minimal context gracefully
- [x] a5dbeac L1 cache regeneration updates wing doc

---

## Phase N3 — Knowledge Graph

**Goal:** Extract entity-relationship triples from notes, building a temporal knowledge graph per user.

**Estimated: 4 days | New files: 1 | Tests: ~16**

### N3.1 KG Triple Extraction

During memory extraction (N1.4), also extract entity triples:

```typescript
export interface ExtractedTriple {
  subject: string;    // "React Router"
  predicate: string;  // "replaced_by"
  object: string;     // "TanStack Router"
  confidence: number;
}

// LLM prompt addition:
// "Also extract entity relationships as (subject, predicate, object) triples."
```

### N3.2 KG Repository Extensions

Added to `modules/palace/repository.ts`:

- [x] 31bdb0a `addTriple(userId, productId, wingId, subject, predicate, object, confidence)` — assert a fact
- [x] 31bdb0a `invalidateTriple(userId, productId, subject, predicate, object)` — mark fact as ended
- [x] 31bdb0a `queryEntity(userId, productId, entity, asOf?)` → all current triples about entity
- [x] 31bdb0a `queryRelation(userId, productId, subject, predicate)` → matching objects
- [x] 31bdb0a `entityTimeline(userId, productId, entity)` → chronological story
- [x] 31bdb0a `findKGContradictions(userId, productId, wingId?)` → conflicting current triples

### N3.3 Auto-Link Notes via KG

When two notes reference the same entity, suggest a note-relationship:
```typescript
// After KG extraction, check if entities appear in other notes' KG triples
// If so, suggest creating a note-relationship (existing module)
```

### N3.4 Tests

- [x] 31bdb0a Triple CRUD with user isolation
- [x] 31bdb0a Temporal queries (point-in-time vs current)
- [x] 31bdb0a Entity timeline returns chronological order
- [x] 31bdb0a Contradiction detection finds conflicting facts
- [x] 31bdb0a Invalidated triples excluded from current queries
- [ ] KG auto-link suggests note relationships (deferred to N5 routes)
- [x] 31bdb0a Cross-user: user A's KG never leaks to user B

---

## Phase N4 — Auto-Save Hooks (Event-Driven Memory Capture)

**Goal:** Automatically extract and store memories when notes change.

**Estimated: 3 days | Modified files: 3 | Tests: ~10**

### N4.1 Save Triggers

| Trigger | Where | What Happens |
|---------|-------|-------------|
| **Note created** | `notes/routes.ts` POST | Extract memories + KG from note body |
| **Note updated** | `notes/routes.ts` PATCH | Incremental extraction from changed content |
| **Agent action completed** | `note-agent-actions/routes.ts` | Extract memories from agent output |
| **Prompt executed** | `note-prompts/routes.ts` run | Extract memories from prompt result |
| **Explicit store** | MCP `mempalace_store` tool | User/agent-directed memory storage |

### N4.2 Post-Save Hook

```typescript
// In notes/routes.ts, after successful create/update:
async function onNoteSaved(userId: string, productId: string, note: NoteDoc): Promise<void> {
  try {
    const wing = await ensureWing(userId, productId, note.workspaceId, workspaceName);
    const memories = await extractMemories(note.body, note.title, workspaceName);

    for (const mem of memories) {
      const room = await ensureRoom(userId, productId, wing.id, mem.roomSlug);
      const embedding = await embedText(mem.content);
      const isDup = await isNearDuplicate(userId, productId, room.id, mem.hall, mem.content, embedding);
      if (!isDup) {
        await storeMemory(userId, productId, wing.id, room.id, mem.hall, mem.content, note.id, embedding);
      }
    }

    // Regenerate L1 cache for this wing
    await regenerateCriticalFacts(userId, productId, wing.id);
  } catch (err) {
    // Best-effort — don't fail the note save
    req.log.warn({ err }, 'palace: memory extraction failed');
  }
}
```

### N4.3 Tests

- [x] 0af5f87 Note create triggers memory extraction
- [x] 0af5f87 Note update triggers incremental extraction (same hook, re-extracts)
- [x] 0af5f87 Duplicate memories are skipped
- [x] 0af5f87 Extraction failure doesn't fail the note save
- [x] 0af5f87 Wing auto-created from workspace on first save
- [x] 0af5f87 Room auto-created from extracted topic
- [x] 0af5f87 Memories include sourceNoteId back-reference
- [x] 0af5f87 Cross-user isolation: user A save never creates user B memories

---

## Phase N5 — Palace API Routes

**Goal:** REST endpoints for palace operations, secured with JWT auth.

**Estimated: 3 days | New files: 1 | Tests: ~14**

### N5.1 Routes (`modules/palace/routes.ts`)

All routes extract `userId` from JWT via `getUserId(req)`:

```typescript
// Palace search
GET    /api/palace/search?q=...&wingId=...&limit=10

// Wings
GET    /api/palace/wings
GET    /api/palace/wings/:wingId
DELETE /api/palace/wings/:wingId

// Rooms
GET    /api/palace/wings/:wingId/rooms

// Memories
POST   /api/palace/memories         // explicit store
GET    /api/palace/memories?wingId=...&roomId=...&hall=...
DELETE /api/palace/memories/:id

// Knowledge Graph
GET    /api/palace/kg/entity/:entity
GET    /api/palace/kg/timeline/:entity
GET    /api/palace/kg/contradictions

// Wake-up context
GET    /api/palace/wake-up/:wingId?task=...

// Maintenance
POST   /api/palace/backfill-embeddings
POST   /api/palace/prune
GET    /api/palace/health
GET    /api/palace/stats
```

### N5.2 Zod Schemas

```typescript
export const PalaceSearchQuerySchema = z.object({
  q: z.string().min(1).max(500),
  wingId: z.string().max(128).optional(),
  roomId: z.string().max(128).optional(),
  hall: z.enum(HALL_TYPES).optional(),
  limit: z.coerce.number().int().min(1).max(50).default(10),
});

export const StoreMemorySchema = z.object({
  wingId: z.string().min(1).max(128),
  roomId: z.string().min(1).max(128),
  hall: z.enum(HALL_TYPES),
  content: z.string().min(1).max(5000),
  sourceNoteId: z.string().max(128).optional(),
});
```

### N5.3 Tests

- [x] be2f4ff Search returns user-scoped results
- [x] be2f4ff Wings list only returns requesting user's wings
- [x] be2f4ff Explicit memory store persists and is immediately searchable
- [x] be2f4ff Delete wing cascades correctly
- [x] be2f4ff Wake-up context returns structured L0+L1+L2
- [x] be2f4ff KG entity query returns user-scoped triples
- [x] be2f4ff Stats endpoint returns accurate counts per user
- [x] be2f4ff Unauthenticated requests return 401
- [ ] Wrong productId returns empty/403 (enforced at repo layer, tested in N1)

---

## Phase N6 — MCP Memory Tools

**Goal:** Extend NoteLett's existing MCP system with palace tools.

**Estimated: 2 days | Modified files: 2 | New files: 1 | Tests: ~10**

### N6.1 New MCP Tools

Added to `mcp/` alongside existing note tools:

| Tool | Description |
|------|-------------|
| `mempalace_search` | Semantic + text hybrid search across user's palace |
| `mempalace_store` | Explicitly store a memory (agent-directed) |
| `mempalace_wake_up` | Get wake-up context for a workspace/wing |
| `mempalace_query_entity` | Query KG triples about an entity |
| `mempalace_timeline` | Chronological story of an entity |
| `mempalace_list_wings` | List user's palace wings with stats |

### N6.2 User Isolation in MCP

Same pattern as existing MCP tools:

```typescript
async function executeMempalaceSearch(args: MempalaceSearchInput, req: NotesMcpRequest) {
  const userId = requireUserId(req);  // JWT-enforced
  const productId = PRODUCT_ID;
  return searchHybrid(userId, productId, args.query, embedding, args.wingId, args.limit);
}
```

### N6.3 Tests

- [x] c7c1eba Each MCP tool returns valid responses (6 tools registered)
- [x] c7c1eba `mempalace_search` returns ranked results
- [x] c7c1eba `mempalace_store` persists and is searchable
- [x] c7c1eba `mempalace_wake_up` returns structured context
- [x] c7c1eba MCP tools enforce userId from JWT (user A can't query user B)
- [x] c7c1eba Graceful response when palace is empty

---

## Phase N7 — Web UI Integration

**Goal:** Surface palace data in the NoteLett web dashboard.

**Estimated: 4 days | New files: 4 React | Tests: ~8**

### N7.1 Web API Client

New file: `web/src/lib/palace-client.ts`

```typescript
export async function searchPalace(query: string, wingId?: string): Promise<PalaceMemory[]>;
export async function listWings(): Promise<PalaceWing[]>;
export async function getEntityTimeline(entity: string): Promise<TimelineEntry[]>;
export async function getWakeUpContext(wingId: string): Promise<WakeUpContext>;
export async function getPalaceStats(): Promise<PalaceStats>;
```

### N7.2 React Components

- **PalacePanel** — sidebar tab showing palace search + recent memories
- **KnowledgeGraphView** — entity-relationship visualization for a workspace
- **MemoryTimeline** — chronological view of extracted memories
- **PalaceStats** — dashboard card showing wing counts, memory counts, KG size

### N7.3 Integration with Existing UI

- **Note detail page** — show "Related Memories" panel from palace search
- **Workspace page** — show wing stats + recent memories
- **Search page** — add "Search Palace" tab alongside note search

### N7.4 Tests ✅ (e6dacbe)

- [x] Palace search renders results
- [x] Wings list shows user's workspaces as wings
- [x] Knowledge graph renders entity nodes
- [x] Empty state renders gracefully
- [x] Palace client API functions (15 tests)
- [x] PalaceStats component (3 tests)
- [x] PalacePanel component (3 tests)

---

## Summary

| Phase | Description | Days | Tests | Depends On |
|-------|-------------|------|-------|------------|
| **N0** | `@bytelyst/palace` shared package in common-plat (types, cosine, dedup, decay, extraction, KG, wakeup) — d1c6cf4 | 3-4 | 91 ✅ | — |
| **N1** | Palace Core (Cosmos containers, repository, extraction, dedup, encryption) — 38006af…632b5df | 4-5 | 31 ✅ | N0 |
| **N2** | Wake-Up Context (L0/L1/L2 at ~600 tokens) — a5dbeac | 3 | 9 ✅ | N1 |
| **N3** | Knowledge Graph (temporal entity triples) — 31bdb0a | 4 | 10 ✅ | N1 |
| **N4** | Auto-Save Hooks (note create/update/agent action triggers) — 0af5f87 | 3 | 7 ✅ | N1 |
| **N5** | Palace API Routes (REST endpoints + Zod validation) — be2f4ff | 3 | 9 ✅ | N1, N2, N3 |
| **N6** | MCP Memory Tools (6 `mempalace_*` tools) — c7c1eba | 2 | 7 ✅ | N1, N5 |
| **N7** | Web UI (palace panel, KG view, timeline, stats) — e6dacbe | 4 | 21 ✅ | N5 |
| **Total** | | **27-32 days** | **~125** | |

### New Dependencies (in NoteLett)

```json
"@bytelyst/palace": "workspace:*"
```

(Added to `backend/package.json` — consumed alongside existing `@bytelyst/datastore`, `@bytelyst/llm`, `@bytelyst/field-encrypt`, etc.)

### New Cosmos Containers

| Container | Partition Key | Purpose |
|-----------|--------------|---------|
| `palace_wings` | `/userId` | Per-user workspace→wing mapping |
| `palace_rooms` | `/userId` | Topic rooms within wings |
| `palace_memories` | `/userId` | Core memory storage with embeddings |
| `palace_tunnels` | `/userId` | Cross-room links |
| `palace_kg` | `/userId` | Knowledge graph triples |
| `palace_diaries` | `/userId` | Role-scoped diary entries |

### New Files

```
# ── In learning_ai_common_plat (Phase N0) ──────────────────────
packages/palace/
├── src/
│   ├── index.ts, types.ts, halls.ts, cosine.ts, dedup.ts,
│   ├── decay.ts, extraction.ts, kg.ts, wakeup.ts, config.ts
│   └── __tests__/ (6 test files)
├── package.json            # @bytelyst/palace
└── tsconfig.json

# ── In learning_ai_notes (Phases N1-N7) ────────────────────────
backend/src/modules/palace/
├── types.ts           # NoteLett-specific palace types (extends @bytelyst/palace base types)
├── repository.ts      # User-scoped CRUD + search (uses @bytelyst/palace cosine, dedup, decay)
├── extractor.ts       # LLM extraction (uses @bytelyst/palace prompts + @bytelyst/llm)
├── wakeup.ts          # Wake-up context (uses @bytelyst/palace wakeup builder)
├── routes.ts          # Fastify REST routes (uses @bytelyst/fastify-auth for user isolation)
└── palace.test.ts     # Vitest tests (uses @bytelyst/testing helpers)

backend/src/mcp/
├── palace-tool-contracts.ts   # Zod schemas for MCP tools
└── palace-tools.ts            # 6 MCP tool implementations

web/src/lib/
└── palace-client.ts   # Typed API client (uses @bytelyst/api-client)

web/src/components/
├── PalacePanel.tsx    # Sidebar palace search + memories (uses @bytelyst/ui)
├── KnowledgeGraphView.tsx
├── MemoryTimeline.tsx
└── PalaceStats.tsx
```

### New Config Variables

| Variable | Default | Description |
|----------|---------|-------------|
| `PALACE_ENABLED` | `true` | Enable/disable palace feature |
| `PALACE_EXTRACTION_ENABLED` | `true` | Enable auto-extraction on note save |
| `PALACE_WAKE_UP_BUDGET` | `600` | Max tokens for wake-up context |
| `PALACE_RELEVANCE_HALF_LIFE_DAYS` | `90` | Days for relevance to halve |
| `PALACE_DEDUP_THRESHOLD` | `0.90` | Cosine similarity threshold for dedup |

### Critical Path

```
N1 (Palace Core + extraction + dedup)
 ├──→ N2 (Wake-Up) ──→ N5 (API Routes) ──→ N6 (MCP Tools)
 ├──→ N3 (Knowledge Graph) ──→ N5                    ↘
 ├──→ N4 (Auto-Save) — independent after N1            N7 (Web UI)
 └──→ N6, N7 — last, needs N5

Note: N3 is optional for N2 (L1 works without KG; triples added once N3 is done).
N4 can be built in parallel with N2/N3.
```

**Recommended execution order:** N1 → N4 → N2 → N3 → N5 → N6 → N7

This gives auto-saving memories on note create/update after N1+N4 (~9 days), with semantic wake-up and KG following.

---

## User Isolation Checklist

Every phase must satisfy these checks before merging:

- [x] **Every new Cosmos container** uses `/userId` as partition key — all 6 containers
- [x] **Every repository function** takes `userId` as first parameter
- [x] **Every query filter** includes `userId` — no exceptions
- [x] **Every route handler** extracts userId via `getUserId(req)` — never from request body/params
- [x] **Every test file** includes at least one cross-user isolation test
- [x] **No endpoint** accepts userId as a URL parameter or request body field
- [x] **Field encryption** uses `userId` in encryption context
- [x] **MCP tools** use `requireUserId(req)` guard (from existing pattern)