diff --git a/docs/devops/END_TO_END_ENCRYPTION_DESIGN.md b/docs/devops/END_TO_END_ENCRYPTION_DESIGN.md new file mode 100644 index 00000000..f9a0bd01 --- /dev/null +++ b/docs/devops/END_TO_END_ENCRYPTION_DESIGN.md @@ -0,0 +1,612 @@ +# ByteLyst — End-to-End Encryption & Data Protection Design + +> **Purpose:** Comprehensive encryption strategy for all ByteLyst ecosystem repos and components. +> **Status:** Design document — not yet implemented +> **Author:** AI Architecture Review +> **Last updated:** 2026-03-21 +> **See also:** [`AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md`](AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md), [`END_TO_END_ENCRYPTION_ROADMAP.md`](END_TO_END_ENCRYPTION_ROADMAP.md) + +--- + +## 1. Executive Summary + +The ByteLyst ecosystem spans 10 products, 3 shared services, 15 repos, and 5 client surfaces (Web, iOS, Android, macOS, watchOS). Data flows through Cosmos DB, Azure Blob Storage, SQLite (local), and in-transit via HTTPS. + +This document defines a **tiered encryption strategy** that balances security, performance, and developer experience: + +| Tier | Scope | Threat Model | Effort | +| ---------- | --------------------------------------------- | ------------------------------------------------ | -------------- | +| **Tier 0** | Infrastructure encryption (already done) | DB breach at rest, network sniffing | N/A (complete) | +| **Tier 1** | Application-layer field encryption | DB admin access, backup leaks, insider threat | Medium | +| **Tier 2** | Client-side encryption with server decryption | Reduced server-side exposure window | High | +| **Tier 3** | True E2EE (zero-knowledge) | Server compromise, legal subpoena, full distrust | Very High | + +**Recommendation:** Implement Tier 1 ecosystem-wide via a new `@bytelyst/field-encrypt` package. Offer Tier 3 as opt-in "Private Vaults" for JarvisJr and NoteLett only. + +--- + +## 2. Current State Audit + +### 2.1 What We Already Have (Tier 0 — Infrastructure) + +| Layer | Protection | Status | +| --------------------------------- | -------------------------------- | ----------------------------------- | +| **Cosmos DB at rest** | AES-256 (Microsoft-managed keys) | ✅ Enabled by default | +| **Cosmos DB in transit** | TLS 1.2+ | ✅ Enforced | +| **Azure Blob Storage at rest** | AES-256 (Microsoft-managed keys) | ✅ Enabled by default | +| **Azure Blob Storage in transit** | TLS 1.2+ | ✅ Enforced | +| **SQLite (LocalMemGPT)** | No encryption (local disk) | ⚠️ Gap | +| **Azure Key Vault** | HSM-backed secret storage | ✅ 30 secrets managed | +| **JWT tokens** | HS256 signed (RS256 planned) | ✅ Active | +| **HTTPS** | TLS 1.2+ on all endpoints | ✅ Enforced | +| **iOS Keychain** | Hardware-backed (Secure Enclave) | ✅ via `BLKeychain` | +| **Android Keystore** | Hardware-backed (TEE/StrongBox) | ✅ via `BLSecureStore` (AES256-GCM) | +| **Secret scanning** | Husky pre-commit + pre-push | ✅ Active | + +### 2.2 Existing Application-Layer Encryption + +The only field-level encryption today is in SmartAuth MFA: + +``` +platform-service/src/modules/auth/mfa/repository.ts +├── encryptSecret() — AES-256-GCM, key from AUTH_TOTP_ENCRYPTION_KEY (hex in AKV) +├── decryptSecret() — AES-256-GCM with IV + authTag verification +├── hashRecoveryCode() — SHA-256 one-way hash +└── generateRecoveryCodes() — crypto.randomBytes +``` + +**Pattern:** `{ encrypted: string, iv: string, authTag: string }` stored alongside the document. This is the pattern we'll generalize into `@bytelyst/field-encrypt`. + +### 2.3 Client-Side Secure Storage + +| Platform | Component | Encryption | Key Storage | +| ----------- | --------------------------------- | --------------------------------------------------------------- | ------------------------ | +| **iOS** | `BLKeychain` | Keychain Services (AES-256) | Secure Enclave | +| **Android** | `BLSecureStore` | EncryptedSharedPreferences (AES256-SIV keys, AES256-GCM values) | Android Keystore | +| **Web** | `localStorage` / `sessionStorage` | None | ⚠️ Gap — no encryption | +| **macOS** | `BLKeychain` (shared with iOS) | Keychain Services | Secure Enclave / T2 chip | +| **watchOS** | `BLKeychain` (shared with iOS) | Keychain Services | Secure Enclave | + +--- + +## 3. Threat Model + +### 3.1 Threats by Severity + +| # | Threat | Likelihood | Impact | Current Mitigation | Gap | +| --- | -------------------------------------------------- | ---------- | -------- | -------------------------------------------------------- | ---------------------------------------------------------- | +| T1 | **Database breach** (Cosmos DB credentials leaked) | Medium | Critical | AKV, secret rotation runbooks, Cosmos encryption at rest | No app-layer encryption — leaked key exposes all plaintext | +| T2 | **Backup/export data leak** | Medium | High | N/A | Backups contain plaintext — no field-level encryption | +| T3 | **Insider threat** (DB admin reads data) | Low | High | AKV access policies, RBAC | Cosmos data readable with any valid key | +| T4 | **Man-in-the-middle** | Low | High | TLS 1.2+ everywhere | ✅ Covered | +| T5 | **Client device theft** | Medium | Medium | iOS Keychain, Android Keystore | Web localStorage is plaintext | +| T6 | **Server compromise** | Low | Critical | AKV, managed identities | Server can decrypt everything — no zero-knowledge | +| T7 | **Legal subpoena / government request** | Low | High | N/A | Server holds all keys — must comply | +| T8 | **Cross-product data leak** | Low | Medium | `productId` field isolation | No cryptographic isolation between products | +| T9 | **SQLite local DB theft** (LocalMemGPT) | Low | Medium | OS-level file permissions | SQLite has no encryption | + +### 3.2 What Each Tier Addresses + +| Threat | Tier 0 (Current) | Tier 1 (Field Encrypt) | Tier 2 (Server Decrypt) | Tier 3 (True E2EE) | +| --------------------- | :--------------: | :--------------------: | :---------------------: | :----------------: | +| T1 Database breach | Partial | ✅ | ✅ | ✅ | +| T2 Backup leak | ❌ | ✅ | ✅ | ✅ | +| T3 Insider threat | ❌ | Partial | ✅ | ✅ | +| T4 MITM | ✅ | ✅ | ✅ | ✅ | +| T5 Device theft | Partial | Partial | ✅ | ✅ | +| T6 Server compromise | ❌ | ❌ | ❌ | ✅ | +| T7 Legal subpoena | ❌ | ❌ | ❌ | ✅ | +| T8 Cross-product leak | ❌ | ✅ | ✅ | ✅ | +| T9 SQLite theft | ❌ | ❌ | ✅ | ✅ | + +--- + +## 4. Tier 1 — Application-Layer Field Encryption + +### 4.1 Architecture + +``` +┌─────────────┐ ┌──────────────────┐ ┌─────────────┐ +│ Client │──HTTPS──│ Backend Service │──TLS──→│ Cosmos DB │ +│ (Web/iOS/ │ │ (Fastify 5) │ │ │ +│ Android) │ │ │ │ Stores: │ +└─────────────┘ │ encrypt(field) │ │ { encrypted,│ + │ before write │ │ iv, │ + │ │ │ authTag } │ + │ decrypt(field) │ │ │ + │ after read │ │ │ + └──────────────────┘ └─────────────┘ + │ + │ DEK wrapped by MEK + ▼ + ┌──────────────────┐ + │ Azure Key Vault │ + │ (MEK storage) │ + └──────────────────┘ +``` + +### 4.2 Key Hierarchy + +``` +Master Encryption Key (MEK) +├── Stored in Azure Key Vault (HSM-backed) +├── Never leaves AKV — used only for wrapping/unwrapping DEKs +├── One MEK per product (e.g., lysnr-mek, mindlyst-mek, jarvisjr-mek) +└── Rotation: quarterly, with old MEK retained for decryption + + └── Data Encryption Keys (DEKs) + ├── Generated per-user or per-workspace + ├── AES-256-GCM symmetric key (32 bytes) + ├── Wrapped (encrypted) by MEK before storage + ├── Stored alongside data in Cosmos (as wrapped blob) + └── Cached in memory for session duration (max 15 min TTL) +``` + +**Why envelope encryption (MEK → DEK)?** + +- MEK rotation doesn't require re-encrypting all data — just re-wrap the DEKs +- Per-user DEKs provide cryptographic isolation between users +- DEK caching avoids an AKV round-trip on every read/write +- AKV rate limits (5,000 ops/10s) are not hit because DEK unwrap is infrequent + +### 4.3 `@bytelyst/field-encrypt` Package Design + +``` +packages/field-encrypt/ +├── src/ +│ ├── index.ts # Public API exports +│ ├── types.ts # EncryptedField, FieldEncryptOptions, KeyProvider +│ ├── aes-gcm.ts # AES-256-GCM encrypt/decrypt (Node.js crypto) +│ ├── envelope.ts # DEK generation, MEK wrapping/unwrapping +│ ├── key-provider-akv.ts # Azure Key Vault key provider (production) +│ ├── key-provider-env.ts # Env-var key provider (dev/test — like current MFA) +│ ├── key-provider-memory.ts # In-memory key provider (unit tests) +│ ├── key-cache.ts # TTL cache for unwrapped DEKs +│ ├── migration.ts # Helpers for encrypting existing plaintext fields +│ └── index.test.ts # Tests +├── package.json +└── tsconfig.json +``` + +### 4.4 API Design + +```typescript +// ── Setup (once per service) ────────────────────────────── +import { createFieldEncryptor } from '@bytelyst/field-encrypt'; + +const encryptor = createFieldEncryptor({ + keyProvider: 'akv', // 'akv' | 'env' | 'memory' + keyVaultUrl: process.env.AZURE_KEYVAULT_URL, + mekName: 'lysnr-mek', // product-specific MEK in AKV + dekCacheTtlMs: 15 * 60 * 1000, // 15 min DEK cache +}); + +// ── Encrypt a field before writing to Cosmos ────────────── +const encrypted = await encryptor.encrypt('Hello, this is a transcript', { + userId: 'user_123', // DEK scope + context: 'transcripts', // additional authenticated data (AAD) +}); +// Returns: { ciphertext: string, iv: string, authTag: string, dekId: string, version: 1 } + +// ── Decrypt a field after reading from Cosmos ───────────── +const plaintext = await encryptor.decrypt(encrypted, { + userId: 'user_123', + context: 'transcripts', +}); +// Returns: 'Hello, this is a transcript' + +// ── Batch encrypt/decrypt (optimized — single DEK unwrap) ─ +const results = await encryptor.encryptBatch(fields, { userId, context }); +const plaintexts = await encryptor.decryptBatch(encryptedFields, { userId, context }); + +// ── Re-wrap DEKs after MEK rotation ────────────────────── +await encryptor.rewrapDeks({ oldMekName: 'lysnr-mek-v1', newMekName: 'lysnr-mek-v2' }); + +// ── Migration: encrypt existing plaintext in-place ──────── +await encryptor.migrateField(cosmosContainer, { + fieldName: 'transcriptText', + partitionKey: '/userId', + batchSize: 100, + dryRun: false, +}); +``` + +### 4.5 Encrypted Document Schema + +Before encryption: + +```json +{ + "id": "transcript_abc123", + "userId": "user_123", + "productId": "lysnrai", + "transcriptText": "Hello, this is a very sensitive transcript...", + "createdAt": "2026-03-21T08:00:00Z" +} +``` + +After encryption: + +```json +{ + "id": "transcript_abc123", + "userId": "user_123", + "productId": "lysnrai", + "transcriptText": { + "__encrypted": true, + "v": 1, + "alg": "aes-256-gcm", + "ct": "a1b2c3d4e5f6...", + "iv": "0102030405060708090a0b0c", + "tag": "f1e2d3c4b5a6...", + "dekId": "dek_user_123_transcripts" + }, + "createdAt": "2026-03-21T08:00:00Z" +} +``` + +**Key design decisions:** + +- `__encrypted: true` sentinel allows code to detect encrypted vs plaintext fields during migration +- `v: 1` version field enables future algorithm changes without re-encryption +- `dekId` identifies which DEK to unwrap — enables per-user key isolation +- Non-sensitive fields (`userId`, `productId`, `createdAt`) remain in plaintext for querying +- Cosmos DB queries on encrypted fields are impossible by design — use indexed plaintext metadata fields for filtering + +### 4.6 Fields to Encrypt by Product + +| Product | Container | Field(s) to Encrypt | Sensitivity | Priority | +| --------------- | ----------------- | --------------------------------------- | ----------- | --------------- | +| **LysnrAI** | `transcripts` | `transcriptText`, `rawAudio` (blob ref) | Critical | P0 | +| **LysnrAI** | `sessions` | `notes` | High | P1 | +| **JarvisJr** | `jarvis_sessions` | `transcript`, `coachingNotes` | Critical | P0 | +| **JarvisJr** | `jarvis_memory` | `content` | Critical | P0 | +| **NoteLett** | `notes` | `body` | Critical | P0 | +| **NoteLett** | `note_artifacts` | `content` | High | P1 | +| **MindLyst** | `memory_items` | `content`, `voiceTranscriptText` | Critical | P0 | +| **MindLyst** | `brains` | `description` | Medium | P2 | +| **MindLyst** | `reflections` | `content` | High | P1 | +| **NomGap** | `meal_logs` | `notes` | Low | P3 | +| **ChronoMind** | `timers` | (none — low sensitivity) | Low | Defer | +| **PeakPulse** | `peak_sessions` | (none — GPS/stats not sensitive) | Low | Defer | +| **FlowMonk** | `tasks` | `notes` | Low | P3 | +| **ActionTrail** | `trail_actions` | `beforeSnapshot`, `afterSnapshot` | Medium | P2 | +| **LocalMemGPT** | SQLite `messages` | `content` | High | P1 | +| **SmartAuth** | `auth_mfa` | `encryptedSecret` | Critical | ✅ Already done | + +### 4.7 Query Impact Analysis + +**Fields that are encrypted cannot be queried via Cosmos SQL.** This impacts: + +| Product | Current Query Pattern | Mitigation | +| ----------- | -------------------------- | -------------------------------------------------------------------------------------------- | +| NoteLett | FTS on `body` | Maintain plaintext search index (separate collection) or use extraction-service | +| LocalMemGPT | FTS5 on `content` | FTS5 index operates on plaintext — encrypt at SQLite column level with `sqlcipher` | +| MindLyst | Search on `content` | Use extraction-service for entity extraction; store extracted entities as plaintext metadata | +| LysnrAI | Search on `transcriptText` | Same: extraction → plaintext keywords | + +**Pattern:** Extract searchable metadata before encryption, store as plaintext alongside encrypted content. + +--- + +## 5. Tier 2 — Client-Side Encryption with Server Decryption + +### 5.1 When to Use + +Tier 2 reduces the **exposure window** — data is encrypted on the client, transmitted encrypted, and only decrypted on the server when actively needed (e.g., AI processing). At rest, it's stored encrypted. + +This is relevant for: + +- **JarvisJr voice sessions** — encrypt audio/transcript on-device, decrypt only during PromptBuilder execution +- **LysnrAI dictation** — encrypt transcript on-device, decrypt only during cloud processing +- **LocalMemGPT RAG** — encrypt documents on upload, decrypt only during embedding generation + +### 5.2 Architecture + +``` +┌───────────────┐ ┌──────────────────┐ ┌───────────┐ +│ Client │ encrypted │ Backend │ encrypted │ Cosmos DB │ +│ │───payload───────→│ │─────────────→│ │ +│ 1. Generate │ │ 2. Store as-is │ │ (encrypted │ +│ session │ │ (no decrypt) │ │ at rest + │ +│ key (SK) │ │ │ │ at app │ +│ 3. Encrypt │ │ 4. When AI │ │ layer) │ +│ with SK │ │ needed: │ │ │ +│ 5. Send SK │ │ unwrap SK │ │ │ +│ wrapped │ │ from AKV, │ │ │ +│ by MEK │ │ decrypt, │ │ │ +│ │ │ process, │ │ │ +│ │ │ re-encrypt │ │ │ +└───────────────┘ └──────────────────┘ └───────────┘ +``` + +### 5.3 Client-Side Encryption Primitives + +| Platform | API | Algorithm | +| ----------- | ------------------------------- | ----------------- | +| **Web** | `SubtleCrypto` (Web Crypto API) | AES-256-GCM | +| **iOS** | `CryptoKit` (`AES.GCM`) | AES-256-GCM | +| **Android** | `javax.crypto.Cipher` | AES/GCM/NoPadding | +| **Node.js** | `node:crypto` | AES-256-GCM | + +All platforms support AES-256-GCM natively — no external dependencies needed. + +### 5.4 New SDK Components + +| SDK | Component | Purpose | +| ------------------------- | ------------------------- | ----------------------------------------- | +| `@bytelyst/field-encrypt` | `createClientEncryptor()` | Web + React Native client-side encryption | +| `swift-platform-sdk` | `BLFieldEncrypt` | iOS/macOS/watchOS client-side encryption | +| `kotlin-platform-sdk` | `BLFieldEncrypt` | Android/Wear OS client-side encryption | + +--- + +## 6. Tier 3 — True End-to-End Encryption (Zero-Knowledge) + +### 6.1 When to Use + +True E2EE means the **server never sees plaintext** and **cannot decrypt** the data. This is the Signal/ProtonMail model. + +**Applicable products:** JarvisJr (opt-in private agents), NoteLett (opt-in private workspaces). + +**NOT applicable to:** + +- Products requiring server-side AI (NomGap coaching, MindLyst triage, FlowMonk scheduling) +- Products requiring server-side search (LocalMemGPT FTS5) +- Products where the server IS the authority (ActionTrail oversight, PeakPulse leaderboards) + +### 6.2 Architecture + +``` +┌───────────────┐ ┌──────────────────┐ ┌───────────┐ +│ Client │ encrypted │ Backend │ encrypted │ Cosmos DB │ +│ │───payload───────→│ │─────────────→│ │ +│ 1. Generate │ │ 2. Store as-is │ │ Ciphertext │ +│ user │ │ (CANNOT │ │ only — │ +│ keypair │ │ decrypt) │ │ no keys │ +│ 2. Encrypt │ │ │ │ │ +│ with │ │ 3. Return │ │ │ +│ user's │◀──ciphertext────│ ciphertext │ │ │ +│ public key │ │ to client │ │ │ +│ 3. Decrypt │ │ │ │ │ +│ with │ │ NO MEK/DEK │ │ │ +│ private │ │ access at all │ │ │ +│ key │ │ │ │ │ +└───────────────┘ └──────────────────┘ └───────────┘ + │ + │ Private key stored in: + │ - iOS Keychain (Secure Enclave) + │ - Android Keystore (StrongBox) + │ - Web: derived from passphrase (PBKDF2) + │ + │ Multi-device sync via: + │ - iCloud Keychain (iOS/Mac) + │ - Google Backup (Android) + │ - QR code transfer (cross-platform) +``` + +### 6.3 Key Management Challenges + +| Challenge | Solution | Complexity | +| ------------------------ | ------------------------------------------------------------------- | ---------- | +| **Key loss = data loss** | Recovery phrase (BIP-39 mnemonic, 12 words) | Medium | +| **Multi-device sync** | Platform keychain sync (iCloud/Google) + QR fallback | High | +| **Cross-platform** | Export keypair via QR code / encrypted backup file | High | +| **Key rotation** | Re-encrypt all data with new key — expensive | High | +| **Shared workspaces** | Group key distribution (like Signal group protocol) | Very High | +| **Search** | Client-side search only (download + decrypt + search) | Medium | +| **AI processing** | Not possible — would need homomorphic encryption or secure enclaves | Blocker | + +### 6.4 Trade-offs + +| Feature | With E2EE | Without E2EE | +| --------------------------- | ------------------------------------ | ---------------------------- | +| Server-side AI | ❌ Impossible | ✅ Works | +| Server-side search | ❌ Impossible | ✅ Works | +| MCP tool access | ❌ Impossible | ✅ Works | +| Data recovery (lost device) | ⚠️ Recovery phrase only | ✅ Server can restore | +| Multi-device sync | ⚠️ Complex key distribution | ✅ Automatic | +| Regulatory compliance | ✅ Strong (GDPR Art. 32) | ⚠️ Depends on other controls | +| Marketing / trust | ✅ "Zero-knowledge encryption" claim | ⚠️ Standard | + +--- + +## 7. SQLite Encryption (LocalMemGPT) + +LocalMemGPT uses SQLite as its primary datastore. SQLite databases are unencrypted files on disk. + +### 7.1 Options + +| Option | Library | License | Effort | +| --------------------- | -------------------------------- | ------------------- | ------------------------- | +| **SQLCipher** | `better-sqlite3-sqlcipher` | BSD | Low — drop-in replacement | +| **SEE** | SQLite Encryption Extension | Commercial ($2,000) | Low | +| **Application-level** | Encrypt each field before INSERT | N/A | Medium | + +**Recommendation:** Use `better-sqlite3` with application-level field encryption via `@bytelyst/field-encrypt` (env key provider). This avoids a native dependency change and is consistent with the Cosmos DB approach. + +### 7.2 FTS5 Impact + +If message `content` is encrypted, FTS5 cannot index it. Options: + +1. **Plaintext FTS5 index** — Store plaintext in FTS5 virtual table, encrypted in main table. Risk: FTS5 table is also on disk unencrypted. +2. **Client-side search** — Download + decrypt + search in memory. Acceptable for local-first app. +3. **SQLCipher** — Encrypts entire database file. FTS5 works normally. Best option if willing to change native dep. + +--- + +## 8. Web Client Secure Storage + +Web `localStorage` stores tokens and app state in plaintext. Options: + +| Approach | Protection | Complexity | +| ------------------------- | ------------------------------------ | ---------------------- | +| **IndexedDB + CryptoKey** | AES-256-GCM with non-extractable key | Medium | +| **Session-only storage** | Data lost on tab close | Low | +| **Cookie httpOnly** | Server-managed, XSS-proof | Low (auth tokens only) | + +**Recommendation:** + +- Auth tokens → httpOnly cookies (already planned in SmartAuth RS256 migration) +- Sensitive cached data → IndexedDB with `SubtleCrypto` non-extractable key +- Non-sensitive state → `localStorage` as-is + +--- + +## 9. Blob Storage Encryption + +Azure Blob Storage (`bytelystblobs`) stores audio files, meal photos, GPX exports, document uploads. Currently encrypted at rest by Azure (Microsoft-managed keys). + +### 9.1 Upgrade Path + +| Level | Method | Impact | +| ----------- | ------------------------------------ | -------------------------------------------- | +| **Current** | Azure-managed encryption at rest | Server-side transparent | +| **Level 1** | Customer-managed key (CMK) in AKV | Control over key lifecycle; no code changes | +| **Level 2** | Client-side encryption before upload | Blobs encrypted before leaving client device | + +**Recommendation:** Enable CMK (Level 1) for the `bytelystblobs` storage account. This is a portal-only change with zero code impact and gives us key rotation control. + +For Level 2, the existing `BLBlobClient` (Swift/Kotlin) and `@bytelyst/blob-client` (TS) would need an encrypt-before-upload wrapper — this naturally integrates with Tier 2. + +--- + +## 10. Cross-Cutting Concerns + +### 10.1 Performance Impact + +| Operation | Overhead | Mitigation | +| ---------------------------------- | -------------------- | -------------------------------------- | +| AES-256-GCM encrypt (1 KB field) | ~0.01 ms (Node.js) | Negligible | +| AES-256-GCM encrypt (100 KB field) | ~0.1 ms | Negligible | +| DEK unwrap via AKV | ~50-200 ms | DEK cache (15 min TTL) | +| MEK rotation (re-wrap DEKs) | ~50 ms/DEK × N users | Background job, non-blocking | +| Migration (encrypt existing data) | ~100 docs/sec | Batch migration with progress tracking | + +### 10.2 Compliance Mapping + +| Regulation | Requirement | Tier 0 | Tier 1 | Tier 3 | +| ------------------------- | ------------------------------------------ | :-----: | :-----: | :-----: | +| **GDPR Art. 32** | Encryption of personal data | Partial | ✅ | ✅ | +| **GDPR Art. 34** | Breach notification exemption if encrypted | ❌ | ✅ | ✅ | +| **HIPAA** (if applicable) | PHI encryption at rest + in transit | Partial | ✅ | ✅ | +| **SOC 2 Type II** | Encryption controls | Partial | ✅ | ✅ | +| **Apple App Store** | Encryption disclosure (ERN) | N/A | Declare | Declare | +| **Google Play** | Encryption disclosure | N/A | Declare | Declare | + +**GDPR Art. 34 exemption is the key business driver** — if data is encrypted and keys are not compromised, breach notification to individual users is not required. This significantly reduces breach liability. + +### 10.3 Key Rotation Strategy + +| Key Type | Rotation Frequency | Process | Downtime | +| ---------------- | -------------------- | -------------------------------------------------------------- | ----------------------------------- | +| MEK (in AKV) | Quarterly | Create new MEK version → re-wrap DEKs → deactivate old version | Zero (old MEK retained for decrypt) | +| DEK (per-user) | On demand / annually | Generate new DEK → re-encrypt fields → delete old DEK | Zero (background migration) | +| SQLite key (env) | On app update | Re-encrypt database with new key | Brief (migration on startup) | + +### 10.4 Monitoring & Alerting + +| Event | Source | Alert | +| ------------------------------ | ------------------- | -------- | +| MEK access from unknown IP | AKV diagnostic logs | Critical | +| Decryption failure spike | App telemetry | High | +| DEK cache miss rate > 50% | Backend metrics | Medium | +| Migration progress stall | Background job | Medium | +| Encrypted field count mismatch | Audit job | Low | + +--- + +## 11. Decision Matrix — What to Encrypt Where + +| Product | Server-Side AI? | Server-Side Search? | Recommended Tier | Justification | +| --------------- | :-----------------: | :-----------------: | :----------------------: | ------------------------------------------------------------ | +| **LysnrAI** | Yes (extraction) | Yes (transcripts) | Tier 1 | Server needs plaintext for extraction; field-encrypt at rest | +| **JarvisJr** | Yes (PromptBuilder) | No | Tier 1 + Tier 3 opt-in | Default field-encrypt; offer E2EE private agents | +| **NoteLett** | Yes (MCP tools) | Yes (note search) | Tier 1 + Tier 3 opt-in | Default field-encrypt; offer E2EE private workspaces | +| **MindLyst** | Yes (triage) | Yes (memory search) | Tier 1 | Server needs plaintext for AI triage pipeline | +| **NomGap** | Yes (AI coach) | No | Tier 1 (meal notes only) | Low sensitivity overall; encrypt meal notes | +| **LocalMemGPT** | Yes (Ollama) | Yes (FTS5) | Tier 1 + SQLCipher | Local-first; SQLCipher for full-DB encryption | +| **ChronoMind** | No | No | Tier 0 (current) | Timer configs are not sensitive | +| **PeakPulse** | No | No | Tier 0 (current) | GPS/stats are not sensitive | +| **FlowMonk** | Yes (scheduler) | No | Tier 1 (task notes only) | Backend is authoritative; encrypt task notes | +| **ActionTrail** | No | Yes (action search) | Tier 1 (snapshots only) | Oversight requires readable actions; encrypt snapshots | + +--- + +## 12. Dependencies & Prerequisites + +| Prerequisite | Status | Blocks | +| --------------------------------------------------------- | -------------------------------------- | ---------------------------------- | +| Azure Key Vault RBAC mode | ⚠️ Planned (currently access policies) | Tier 1 MEK management | +| AKV `encrypt`/`decrypt`/`wrapKey`/`unwrapKey` permissions | Not configured | Tier 1 envelope encryption | +| `@azure/keyvault-keys` npm package | Available (not installed) | Tier 1 AKV key provider | +| RS256 JWT migration (SmartAuth Phase 4) | Planned | Tier 2 key distribution | +| iCloud Keychain sharing entitlements | Not configured | Tier 3 iOS key sync | +| QR code library (iOS/Android) | Not installed | Tier 3 cross-platform key transfer | +| `better-sqlite3-sqlcipher` or app-level encrypt | Not installed | LocalMemGPT SQLite encryption | +| CMK configuration on `bytelystblobs` | Not configured | Blob storage Level 1 | + +--- + +## 13. Related Documents + +| Document | Purpose | +| -------------------------------------------------------------------------------------------- | ----------------------------------------- | +| [`END_TO_END_ENCRYPTION_ROADMAP.md`](END_TO_END_ENCRYPTION_ROADMAP.md) | Phased implementation plan with timelines | +| [`AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md`](AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md) | Current AKV setup and rotation runbooks | +| [`AZURE_RESOURCE_INVENTORY.md`](AZURE_RESOURCE_INVENTORY.md) | Azure resource inventory | +| [`ENVIRONMENT_VARIABLES_AND_KEYVAULT_AUDIT.md`](ENVIRONMENT_VARIABLES_AND_KEYVAULT_AUDIT.md) | Env var and AKV audit | +| `../architecture/ECOSYSTEM_ARCHITECTURE.md` | ByteLyst ecosystem architecture | +| `../../services/platform-service/src/modules/auth/mfa/repository.ts` | Existing AES-256-GCM pattern | +| `../../packages/kotlin-platform-sdk/src/main/kotlin/.../BLSecureStore.kt` | Android EncryptedSharedPreferences | +| `../../packages/swift-platform-sdk/Sources/BLKeychain.swift` | iOS Keychain wrapper | + +--- + +## Appendix A: Algorithm Selection Rationale + +| Algorithm | Use Case | Why | +| ----------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------- | +| **AES-256-GCM** | Field encryption (all tiers) | AEAD — provides confidentiality + integrity + authentication. Native support on all platforms. NIST approved. | +| **RSA-OAEP-256** | MEK wrapping in AKV | Azure Key Vault natively supports RSA key wrapping. 4096-bit RSA for quantum-resistant margin. | +| **PBKDF2-SHA256** | Web passphrase → key derivation | Web Crypto API native. 600,000 iterations (OWASP 2024 recommendation). | +| **X25519** | Tier 3 key exchange | Modern elliptic-curve DH. Used by Signal, WireGuard. Compact (32-byte keys). | +| **HKDF-SHA256** | Tier 3 key derivation from shared secret | Standard KDF for deriving symmetric keys from DH output. | + +## Appendix B: Encrypted Field Detection Pattern + +Repositories can use a type guard to handle both encrypted and plaintext fields during migration: + +```typescript +import type { EncryptedField } from '@bytelyst/field-encrypt'; + +function isEncrypted(value: unknown): value is EncryptedField { + return ( + typeof value === 'object' && + value !== null && + '__encrypted' in value && + (value as Record).__encrypted === true + ); +} + +// Usage in repository +async function getNote(id: string): Promise { + const doc = await collection.findById(id, partitionKey); + if (isEncrypted(doc.body)) { + doc.body = await encryptor.decrypt(doc.body, { userId: doc.userId, context: 'notes' }); + } + return doc; +} +``` + +## Appendix C: Cost Impact + +| Resource | Current | After Tier 1 | Notes | +| -------------- | ----------------------- | ------------------------------- | ------------------------------------------------------------------- | +| AKV operations | ~100/day (secret reads) | ~500/day (+DEK unwraps) | Well within free tier (10K ops/month free) | +| AKV keys | 0 RSA keys | 10 RSA keys (1 MEK per product) | $1/key/month for software-protected; $5 for HSM | +| Cosmos RU | Baseline | +5-10% (larger encrypted docs) | Encrypted fields are ~30% larger than plaintext (base64 + metadata) | +| Compute | Baseline | +1-2% CPU | AES-256-GCM is hardware-accelerated (AES-NI) on all modern CPUs | +| Storage | Baseline | +15-20% for encrypted fields | Base64 encoding + IV + authTag + dekId overhead | + +**Estimated monthly cost increase: < $5** (dominated by AKV key charges). diff --git a/docs/devops/END_TO_END_ENCRYPTION_ROADMAP.md b/docs/devops/END_TO_END_ENCRYPTION_ROADMAP.md new file mode 100644 index 00000000..2d7bfb05 --- /dev/null +++ b/docs/devops/END_TO_END_ENCRYPTION_ROADMAP.md @@ -0,0 +1,689 @@ +# ByteLyst — End-to-End Encryption Implementation Roadmap + +> **Purpose:** Phased implementation plan for encryption across the ByteLyst ecosystem. +> **Status:** Roadmap — not yet started +> **Author:** AI Architecture Review +> **Last updated:** 2026-03-21 +> **Design doc:** [`END_TO_END_ENCRYPTION_DESIGN.md`](END_TO_END_ENCRYPTION_DESIGN.md) + +--- + +## Overview + +This roadmap implements the tiered encryption strategy defined in the design document. Work is organized into 6 sprints across 3 phases, estimated at **12-16 weeks** total elapsed time. + +### Phase Summary + +| Phase | Sprints | Duration | Deliverable | +| ------------------------------ | ---------- | --------- | ------------------------------------------------------------------------------------- | +| **Phase 1: Foundation** | Sprint 1-2 | 4-5 weeks | `@bytelyst/field-encrypt` package + AKV infrastructure + first 2 product integrations | +| **Phase 2: Ecosystem Rollout** | Sprint 3-4 | 4-5 weeks | All 10 product backends encrypted + native SDK components + web secure storage | +| **Phase 3: Advanced** | Sprint 5-6 | 4-6 weeks | True E2EE for JarvisJr/NoteLett + SQLite encryption + blob CMK + audit tooling | + +### Milestone Timeline + +``` +Week 1-2 Week 3-4 Week 5-6 Week 7-8 Week 9-10 Week 11-14 +┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────┐ +│ Sprint 1 │ │ Sprint 2 │ │ Sprint 3 │ │ Sprint 4 │ │ Sprint 5 │ │ Sprint 6 │ +│ Package + │ │ LysnrAI +│ │ 4 more │ │ Native │ │ E2EE │ │ Audit + │ +│ AKV infra │ │ JarvisJr │ │ products │ │ SDKs + │ │ private │ │ polish + │ +│ │ │ │ │ │ │ web │ │ vaults │ │ docs │ +└──────────┘ └──────────┘ └──────────┘ └──────────┘ └──────────┘ └──────────────┘ + Phase 1 Phase 1 Phase 2 Phase 2 Phase 3 Phase 3 +``` + +--- + +## Phase 1: Foundation (Weeks 1-5) + +### Sprint 1 — `@bytelyst/field-encrypt` Package + AKV Infrastructure (Week 1-2) + +**Goal:** Ship the shared encryption package and configure Azure Key Vault for key management. + +#### 1.1 Azure Key Vault Infrastructure + +- [ ] **1.1.1** Enable RBAC mode on `kv-mywisprai` vault (`enableRbacAuthorization=true`) + - Assign `Key Vault Crypto Officer` to deployer user + - Assign `Key Vault Crypto User` to service managed identities (when deployed) + - **File:** Azure Portal / `az keyvault update` + - **Risk:** Breaking change — all existing access policies stop working. Must reassign as RBAC roles simultaneously. + - **Rollback:** `az keyvault update --enable-rbac-authorization false` + re-create access policies + +- [ ] **1.1.2** Create 10 Master Encryption Keys (MEKs) in AKV + + ``` + lysnr-mek, mindlyst-mek, jarvisjr-mek, chronomind-mek, + nomgap-mek, peakpulse-mek, flowmonk-mek, actiontrail-mek, + notelett-mek, localmemgpt-mek + ``` + + - RSA 4096-bit, `wrapKey` + `unwrapKey` operations enabled + - **Script:** `scripts/create-encryption-keys.sh` + +- [ ] **1.1.3** Add new env vars to `.env.example` and AKV secrets doc + ``` + AZURE_KEYVAULT_URL=https://kv-mywisprai.vault.azure.net + FIELD_ENCRYPT_KEY_PROVIDER=akv # 'akv' | 'env' | 'memory' + FIELD_ENCRYPT_MEK_NAME=lysnr-mek # product-specific + ``` + + - **Files:** `.env.example`, `docs/devops/ENVIRONMENT_VARIABLES_AND_KEYVAULT_AUDIT.md` + +#### 1.2 `@bytelyst/field-encrypt` Package + +- [ ] **1.2.1** Create package scaffold + + ``` + packages/field-encrypt/ + ├── src/ + │ ├── index.ts + │ ├── types.ts + │ ├── aes-gcm.ts + │ ├── envelope.ts + │ ├── key-provider-akv.ts + │ ├── key-provider-env.ts + │ ├── key-provider-memory.ts + │ ├── key-cache.ts + │ ├── migration.ts + │ └── index.test.ts + ├── package.json + └── tsconfig.json + ``` + + - **Dependencies:** `@azure/keyvault-keys`, `@azure/identity` (peer), `zod` (peer) + - **Dev dependencies:** `vitest` + +- [ ] **1.2.2** Implement core AES-256-GCM encrypt/decrypt + - `aes-gcm.ts` — `encryptField()`, `decryptField()` using `node:crypto` + - Input: plaintext string + key buffer + optional AAD + - Output: `EncryptedField` object `{ __encrypted, v, alg, ct, iv, tag, dekId }` + - **Tests:** encrypt → decrypt roundtrip, invalid authTag rejection, empty string, unicode, large payload (1 MB) + +- [ ] **1.2.3** Implement envelope encryption (DEK management) + - `envelope.ts` — `generateDek()`, `wrapDek()`, `unwrapDek()` + - DEK: 32-byte random AES key, identified by `dek_{userId}_{context}` + - Wrap: RSA-OAEP with MEK in AKV + - Stored: `{ dekId, wrappedKey, mekVersion, createdAt }` in a `_encryption_keys` Cosmos container + - **Tests:** generate → wrap → unwrap roundtrip, MEK version tracking + +- [ ] **1.2.4** Implement key providers + - `key-provider-akv.ts` — Production: `@azure/keyvault-keys` `CryptographyClient` + - `key-provider-env.ts` — Dev/staging: raw hex key from `FIELD_ENCRYPT_KEY` env var (like current MFA pattern) + - `key-provider-memory.ts` — Unit tests: random in-memory key, no external deps + - **Tests:** provider interface contract tests for each + +- [ ] **1.2.5** Implement DEK cache + - `key-cache.ts` — In-memory LRU cache with configurable TTL (default 15 min) + - Cache key: `dekId`, value: unwrapped DEK buffer + - Auto-evict on TTL expiry, manual `invalidate(dekId)` for rotation + - **Tests:** cache hit/miss, TTL expiry, manual invalidation, max size eviction + +- [ ] **1.2.6** Implement `createFieldEncryptor()` factory + - Wires key provider + cache + AES-GCM + - Public API: `encrypt()`, `decrypt()`, `encryptBatch()`, `decryptBatch()`, `isEncrypted()` + - **Tests:** full integration test with memory provider + +- [ ] **1.2.7** Implement migration helpers + - `migration.ts` — `migrateField()`: read plaintext, encrypt, write back (idempotent) + - Supports batch size, dry run, progress callback + - Detects already-encrypted fields via `__encrypted` sentinel + - **Tests:** migrate plaintext → encrypted, skip already-encrypted, dry run mode + +- [ ] **1.2.8** Add `isEncrypted()` type guard + - Exported utility for repositories to detect encrypted vs plaintext during migration period + - **Tests:** true for EncryptedField, false for string, false for null/undefined + +**Sprint 1 deliverable:** `@bytelyst/field-encrypt` package with all 3 key providers, ~30 tests, `pnpm build && pnpm test` passing. + +**Commit:** `feat(field-encrypt): create @bytelyst/field-encrypt package with AES-256-GCM envelope encryption` + +--- + +### Sprint 2 — First Product Integrations: LysnrAI + JarvisJr (Week 3-4) + +**Goal:** Encrypt the highest-sensitivity fields in 2 products. Validate the pattern end-to-end. + +#### 2.1 LysnrAI Backend (port 4015) + +- [ ] **2.1.1** Add `@bytelyst/field-encrypt` dependency to `backend/package.json` + + ```json + "@bytelyst/field-encrypt": "file:../../learning_ai_common_plat/packages/field-encrypt" + ``` + +- [ ] **2.1.2** Create `backend/src/lib/field-encrypt.ts` — service-level encryptor singleton + + ```typescript + import { createFieldEncryptor } from '@bytelyst/field-encrypt'; + import { config } from './config.js'; + + export const encryptor = createFieldEncryptor({ + keyProvider: config.FIELD_ENCRYPT_KEY_PROVIDER ?? 'memory', + mekName: 'lysnr-mek', + keyVaultUrl: config.AZURE_KEYVAULT_URL, + }); + ``` + +- [ ] **2.1.3** Encrypt `transcriptText` in transcripts module + - `repository.ts`: encrypt on `create()`, decrypt on `findById()` and `findByUserId()` + - Keep `transcriptText` field name but value changes from string to EncryptedField + - **Migration:** Add `POST /api/admin/migrate-encryption` endpoint (admin-only) that batch-encrypts existing plaintext + +- [ ] **2.1.4** Update config schema with encryption env vars + + ```typescript + FIELD_ENCRYPT_KEY_PROVIDER: z.enum(['akv', 'env', 'memory']).default('memory'), + FIELD_ENCRYPT_KEY: z.string().optional(), // only for 'env' provider + AZURE_KEYVAULT_URL: z.string().url().optional(), + ``` + +- [ ] **2.1.5** Update tests — ensure existing tests pass with memory provider (no env vars needed) + +**Commit:** `feat(lysnrai): encrypt transcriptText field with @bytelyst/field-encrypt` + +#### 2.2 JarvisJr Backend (port 4012) + +- [ ] **2.2.1** Add `@bytelyst/field-encrypt` dependency + +- [ ] **2.2.2** Create `backend/src/lib/field-encrypt.ts` encryptor singleton + +- [ ] **2.2.3** Encrypt fields in jarvis-sessions module + - `transcript` field — session voice transcripts + - `coachingNotes` field — extracted coaching insights + +- [ ] **2.2.4** Encrypt fields in jarvis-memory module + - `content` field — per-agent persistent memory entries + +- [ ] **2.2.5** Update config + tests + +**Commit:** `feat(jarvisjr): encrypt session transcripts, coaching notes, and agent memory` + +#### 2.3 Refactor MFA to Use Shared Package + +- [ ] **2.3.1** Migrate `platform-service/src/modules/auth/mfa/repository.ts` from inline AES-256-GCM to `@bytelyst/field-encrypt` + - Replace `encryptSecret()` / `decryptSecret()` with `encryptor.encrypt()` / `encryptor.decrypt()` + - Remove inline `createCipheriv` / `createDecipheriv` code + - Keep `AUTH_TOTP_ENCRYPTION_KEY` as `env` key provider for backward compatibility + - **Tests:** All existing MFA tests must still pass + +**Commit:** `refactor(auth): migrate MFA encryption to @bytelyst/field-encrypt` + +**Sprint 2 deliverable:** 2 product backends fully encrypted, MFA migrated, all tests green. + +--- + +## Phase 2: Ecosystem Rollout (Weeks 5-10) + +### Sprint 3 — Four More Product Backends (Week 5-6) + +**Goal:** Encrypt sensitive fields in NoteLett, MindLyst, NomGap, and ActionTrail. + +#### 3.1 NoteLett Backend (port 4016) + +- [ ] **3.1.1** Add dependency + create encryptor singleton +- [ ] **3.1.2** Encrypt `body` field in notes module +- [ ] **3.1.3** Encrypt `content` field in note-artifacts module +- [ ] **3.1.4** Update MCP tools to decrypt before processing +- [ ] **3.1.5** Update tests + +**Commit:** `feat(notelett): encrypt note body and artifact content` + +#### 3.2 MindLyst Backend (port 4014) + +- [ ] **3.2.1** Add dependency + create encryptor singleton +- [ ] **3.2.2** Encrypt `content` and `voiceTranscriptText` in memory_items module +- [ ] **3.2.3** Encrypt `content` in reflections module +- [ ] **3.2.4** Update tests + +**Commit:** `feat(mindlyst): encrypt memory content, voice transcripts, and reflections` + +#### 3.3 NomGap Backend (port 4013) + +- [ ] **3.3.1** Add dependency + create encryptor singleton +- [ ] **3.3.2** Encrypt `notes` field in meal-logs module (low sensitivity, but consistent) +- [ ] **3.3.3** Update tests + +**Commit:** `feat(nomgap): encrypt meal log notes` + +#### 3.4 ActionTrail Backend (port 4018) + +- [ ] **3.4.1** Add dependency + create encryptor singleton +- [ ] **3.4.2** Encrypt `beforeSnapshot` and `afterSnapshot` in actions module +- [ ] **3.4.3** Encrypt `beforeSnapshot` and `afterSnapshot` in reverts module +- [ ] **3.4.4** Update tests + +**Commit:** `feat(actiontrail): encrypt action and revert snapshots` + +#### 3.5 Remaining Backends (Low Priority — Defer or Skip) + +| Product | Backend Port | Encrypted Fields | Decision | +| ---------------------- | ------------------ | ----------------------------------- | -------- | +| **FlowMonk** (4017) | `tasks.notes` | Defer to Sprint 4 — low sensitivity | +| **ChronoMind** (4011) | None | Skip — timer configs not sensitive | +| **PeakPulse** (4010) | None | Skip — GPS/stats not sensitive | +| **LocalMemGPT** (4019) | `messages.content` | Sprint 5 (SQLite-specific approach) | + +**Sprint 3 deliverable:** 6 product backends encrypted (LysnrAI, JarvisJr, NoteLett, MindLyst, NomGap, ActionTrail). + +--- + +### Sprint 4 — Native SDKs + Web Secure Storage + FlowMonk (Week 7-8) + +**Goal:** Add client-side encryption primitives to all SDKs. Secure web localStorage. + +#### 4.1 Swift Platform SDK — `BLFieldEncrypt` + +- [ ] **4.1.1** Create `Sources/BLFieldEncrypt.swift` in `packages/swift-platform-sdk/` + + ```swift + import CryptoKit + + public struct BLFieldEncrypt { + /// Encrypt a string field with AES-256-GCM + public static func encrypt(_ plaintext: String, key: SymmetricKey) -> EncryptedField + /// Decrypt an encrypted field + public static func decrypt(_ field: EncryptedField, key: SymmetricKey) -> String? + /// Check if a JSON value is an encrypted field + public static func isEncrypted(_ value: Any) -> Bool + } + + public struct EncryptedField: Codable { + public let __encrypted: Bool // always true + public let v: Int // version + public let alg: String // "aes-256-gcm" + public let ct: String // ciphertext (base64) + public let iv: String // IV (hex) + public let tag: String // auth tag (hex) + public let dekId: String // DEK identifier + } + ``` + +- [ ] **4.1.2** Key derivation from Keychain-stored secret +- [ ] **4.1.3** Unit tests (XCTest) + +**Commit:** `feat(swift-sdk): add BLFieldEncrypt for client-side AES-256-GCM encryption` + +#### 4.2 Kotlin Platform SDK — `BLFieldEncrypt` + +- [ ] **4.2.1** Create `src/main/.../BLFieldEncrypt.kt` in `packages/kotlin-platform-sdk/` + ```kotlin + object BLFieldEncrypt { + fun encrypt(plaintext: String, key: SecretKeySpec): EncryptedField + fun decrypt(field: EncryptedField, key: SecretKeySpec): String? + fun isEncrypted(value: Any?): Boolean + } + ``` +- [ ] **4.2.2** Key derivation from BLSecureStore +- [ ] **4.2.3** Unit tests (JUnit5) + +**Commit:** `feat(kotlin-sdk): add BLFieldEncrypt for client-side AES-256-GCM encryption` + +#### 4.3 TypeScript Client Package — `@bytelyst/client-encrypt` + +- [ ] **4.3.1** Create `packages/client-encrypt/` for browser + React Native + ```typescript + // Uses Web Crypto API (SubtleCrypto) — works in browsers and React Native + export function encryptField(plaintext: string, key: CryptoKey): Promise; + export function decryptField(field: EncryptedField, key: CryptoKey): Promise; + export function deriveKey(passphrase: string, salt: Uint8Array): Promise; + ``` +- [ ] **4.3.2** PBKDF2 key derivation for web (600,000 iterations) +- [ ] **4.3.3** Tests with vitest + happy-dom + +**Commit:** `feat(client-encrypt): create @bytelyst/client-encrypt for browser/RN encryption` + +#### 4.4 Web Secure Storage + +- [ ] **4.4.1** Create `@bytelyst/secure-storage-web` package + - IndexedDB-backed storage with `SubtleCrypto` non-extractable AES key + - API: `secureGet(key)`, `secureSet(key, value)`, `secureDelete(key)`, `secureClear()` + - Key is generated once, stored as non-extractable CryptoKey in IndexedDB + - Falls back to `localStorage` if `SubtleCrypto` unavailable + +- [ ] **4.4.2** Migrate auth tokens from `localStorage` to secure storage in all web apps + - Update `@bytelyst/auth-client` to use secure storage + - **Products affected:** All 7 web dashboards/apps + +**Commit:** `feat(secure-storage-web): IndexedDB + SubtleCrypto encrypted storage for web` + +#### 4.5 FlowMonk Backend (port 4017) + +- [ ] **4.5.1** Add dependency + encrypt `tasks.notes` field +- [ ] **4.5.2** Update tests + +**Commit:** `feat(flowmonk): encrypt task notes` + +**Sprint 4 deliverable:** All native SDKs have encryption primitives. Web secure storage. 7 backends encrypted. + +--- + +## Phase 3: Advanced Encryption (Weeks 9-14) + +### Sprint 5 — True E2EE Private Vaults + SQLite Encryption (Week 9-10) + +**Goal:** Implement zero-knowledge encryption for opt-in private spaces in JarvisJr and NoteLett. Encrypt LocalMemGPT SQLite. + +#### 5.1 E2EE Key Management Infrastructure + +- [ ] **5.1.1** Design E2EE key exchange protocol + - User generates X25519 keypair on device + - Public key uploaded to platform-service `user_public_keys` container + - Private key stored in Keychain (iOS/Mac) / Keystore (Android) / derived from passphrase (Web) + - Recovery: 12-word BIP-39 mnemonic → seed → keypair + +- [ ] **5.1.2** Create `@bytelyst/e2ee-core` package + + ```typescript + export function generateKeypair(): { publicKey: Uint8Array; privateKey: Uint8Array }; + export function generateRecoveryPhrase(): string; // 12 BIP-39 words + export function keypairFromRecovery(phrase: string): { publicKey; privateKey }; + export function encryptForUser( + plaintext: string, + recipientPublicKey: Uint8Array + ): EncryptedPayload; + export function decryptForUser(payload: EncryptedPayload, privateKey: Uint8Array): string; + ``` + + - Uses X25519 for key agreement, HKDF-SHA256 for derivation, AES-256-GCM for encryption + - **Dependencies:** `@noble/curves` (X25519), `@noble/hashes` (HKDF, SHA256) + - **Tests:** ~20 tests + +- [ ] **5.1.3** Add `user_public_keys` container to platform-service + - Schema: `{ id, userId, productId, publicKey (base64), deviceId, createdAt }` + - Partition key: `/userId` + - Routes: `POST /auth/e2ee/register-key`, `GET /auth/e2ee/public-key/:userId` + +- [ ] **5.1.4** Multi-device key sync + - iOS: Share private key via iCloud Keychain (requires entitlement) + - Android: Share via Google Backup (EncryptedSharedPreferences) + - Cross-platform: QR code containing encrypted private key (encrypted with passphrase) + +**Commit:** `feat(e2ee): create @bytelyst/e2ee-core + platform key registration` + +#### 5.2 JarvisJr E2EE Private Agents + +- [ ] **5.2.1** Add `isPrivate` boolean field to agent config + - When `true`, all session transcripts and memory for this agent are E2EE + - Server stores only ciphertext — cannot decrypt + - AI features (PromptBuilder, memory retrieval) disabled for private agents + - UI shows "Private Agent" badge + "AI features unavailable" warning + +- [ ] **5.2.2** Client-side encrypt before API calls (iOS + Web) + - Session transcript → `e2eeCore.encryptForUser(transcript, userPublicKey)` + - Memory content → `e2eeCore.encryptForUser(content, userPublicKey)` + - Client-side decrypt after reads + +- [ ] **5.2.3** Update JarvisJr web + iOS to handle E2EE toggle + +**Commit:** `feat(jarvisjr): E2EE private agents with zero-knowledge encryption` + +#### 5.3 NoteLett E2EE Private Workspaces + +- [ ] **5.3.1** Add `isEncrypted` boolean field to workspace config + - When `true`, all notes in workspace are E2EE + - MCP tools return encrypted content (agent cannot read) + - Search disabled for encrypted workspaces + +- [ ] **5.3.2** Client-side encrypt before API calls (Web + Mobile) +- [ ] **5.3.3** Update UI with "Encrypted Workspace" badge + trade-off warnings + +**Commit:** `feat(notelett): E2EE private workspaces with zero-knowledge encryption` + +#### 5.4 LocalMemGPT SQLite Encryption + +- [ ] **5.4.1** Evaluate `better-sqlite3-sqlcipher` vs application-level encryption + - If SQLCipher: swap `better-sqlite3` → `better-sqlite3-sqlcipher`, add `PRAGMA key` + - If app-level: encrypt `content` column via `@bytelyst/field-encrypt` (env provider) + +- [ ] **5.4.2** Implement chosen approach +- [ ] **5.4.3** Handle FTS5 index (see design doc §7.2) +- [ ] **5.4.4** Migration: encrypt existing unencrypted databases on first startup + +**Commit:** `feat(localmemgpt): encrypt SQLite message content` + +**Sprint 5 deliverable:** E2EE private vaults for JarvisJr + NoteLett. SQLite encryption for LocalMemGPT. + +--- + +### Sprint 6 — Blob CMK + Audit Tooling + Documentation (Week 11-14) + +**Goal:** Complete the encryption story with blob storage, monitoring, migration tooling, and docs. + +#### 6.1 Azure Blob Storage Customer-Managed Key (CMK) + +- [ ] **6.1.1** Create `bytelyst-blob-mek` RSA key in AKV +- [ ] **6.1.2** Configure `bytelystblobs` storage account to use CMK from AKV + - Azure Portal → Storage account → Encryption → Customer-managed keys + - Select `bytelyst-blob-mek` from `kv-mywisprai` +- [ ] **6.1.3** Verify all blob operations still work (upload, download, SAS tokens) +- [ ] **6.1.4** Update `AZURE_RESOURCE_INVENTORY.md` + +**Commit:** `feat(infra): enable customer-managed encryption key for Azure Blob Storage` + +#### 6.2 Encryption Migration CLI + +- [ ] **6.2.1** Create `scripts/encrypt-migrate.ts` — CLI tool for batch encrypting existing plaintext + + ```bash + npx tsx scripts/encrypt-migrate.ts \ + --product lysnrai \ + --container transcripts \ + --field transcriptText \ + --partition-key /userId \ + --batch-size 100 \ + --dry-run + ``` + + - Uses `@bytelyst/field-encrypt` migration helpers + - Progress bar, resumable (tracks last processed ID) + - Dry run mode (report count without modifying) + +- [ ] **6.2.2** Create per-product migration scripts + ``` + scripts/migrations/ + ├── encrypt-lysnrai.sh + ├── encrypt-jarvisjr.sh + ├── encrypt-notelett.sh + ├── encrypt-mindlyst.sh + ├── encrypt-nomgap.sh + ├── encrypt-actiontrail.sh + ├── encrypt-flowmonk.sh + └── encrypt-localmemgpt.sh + ``` + +**Commit:** `feat(devops): encryption migration CLI and per-product migration scripts` + +#### 6.3 Monitoring & Audit + +- [ ] **6.3.1** Add encryption metrics to telemetry + - `field_encrypt_count` — encryptions performed + - `field_decrypt_count` — decryptions performed + - `dek_cache_hit_rate` — DEK cache efficiency + - `dek_unwrap_latency_ms` — AKV round-trip time + - `migration_progress` — % of documents encrypted per container + +- [ ] **6.3.2** Add AKV diagnostic logging + - Enable AKV diagnostics → Log Analytics workspace + - Alert on: unexpected IP access, high error rate, key deletion attempts + +- [ ] **6.3.3** Add encryption audit endpoint to platform-service + - `GET /api/admin/encryption-status` — returns per-product encryption coverage + + ```json + { + "products": [ + { + "productId": "lysnrai", + "containers": [ + { + "name": "transcripts", + "field": "transcriptText", + "total": 10000, + "encrypted": 9500, + "coverage": "95%" + } + ] + } + ] + } + ``` + +- [ ] **6.3.4** Add encryption status to admin dashboard Ops page + - Table showing encryption coverage per product/container + - Migration trigger button (admin-only) + +**Commit:** `feat(monitoring): encryption telemetry, AKV diagnostics, audit endpoint` + +#### 6.4 Key Rotation Runbooks + +- [ ] **6.4.1** Write MEK rotation runbook + + ``` + 1. Create new MEK version in AKV (az keyvault key rotate) + 2. Run DEK re-wrap script: scripts/rotate-mek.ts --product lysnrai + 3. Verify decryption still works (run tests) + 4. Update AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md + ``` + +- [ ] **6.4.2** Write DEK rotation runbook (per-user key refresh) +- [ ] **6.4.3** Write emergency key revocation runbook +- [ ] **6.4.4** Add rotation schedule to `AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md` + +**Commit:** `docs(devops): encryption key rotation runbooks` + +#### 6.5 Documentation Updates + +- [ ] **6.5.1** Update all product `AGENTS.md` files with encryption conventions + + ```markdown + ### Encryption + + - Sensitive fields encrypted via `@bytelyst/field-encrypt` + - Encryption singleton in `backend/src/lib/field-encrypt.ts` + - Tests use `memory` key provider (no env vars needed) + - Production uses `akv` key provider with product-specific MEK + ``` + +- [ ] **6.5.2** Update `AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md` with MEK section +- [ ] **6.5.3** Update `ENVIRONMENT_VARIABLES_AND_KEYVAULT_AUDIT.md` with new env vars +- [ ] **6.5.4** Update `AZURE_RESOURCE_INVENTORY.md` with new AKV keys + containers +- [ ] **6.5.5** Write `docs/BEST_PRACTICES/ENCRYPTION.md` — developer guide for adding encryption to new fields + +**Commit:** `docs: comprehensive encryption documentation across all repos` + +**Sprint 6 deliverable:** Complete encryption story — blob CMK, migration tooling, monitoring, key rotation runbooks, full documentation. + +--- + +## Test Plan + +### Package Tests + +| Package | Tests (estimated) | Coverage | +| ------------------------------ | :---------------: | ----------------------------------------------------------- | +| `@bytelyst/field-encrypt` | 30 | AES-GCM, envelope, providers, cache, migration, type guard | +| `@bytelyst/client-encrypt` | 15 | Web Crypto encrypt/decrypt, PBKDF2, roundtrip | +| `@bytelyst/e2ee-core` | 20 | Keypair gen, recovery phrase, encrypt/decrypt, key exchange | +| `@bytelyst/secure-storage-web` | 10 | IndexedDB CRUD, fallback, CryptoKey persistence | + +### Integration Tests (per product) + +Each product backend gets 3-5 new tests: + +- Encrypt → store → read → decrypt roundtrip +- Migration: plaintext → encrypted (idempotent) +- Encrypted field detection (`isEncrypted()`) +- Batch encrypt/decrypt +- Config: memory provider works without env vars + +**Total estimated new tests: ~120** + +### E2E Verification + +- [ ] Manual: Create encrypted note in NoteLett, verify ciphertext in Cosmos Data Explorer +- [ ] Manual: Create private agent in JarvisJr, verify server cannot read transcript +- [ ] Manual: Rotate MEK, verify all reads still work +- [ ] Manual: Run migration script on test container, verify 100% coverage + +--- + +## Risk Register + +| # | Risk | Likelihood | Impact | Mitigation | +| --- | ----------------------------------------------------- | ---------- | -------- | ---------------------------------------------------------------------------------------------- | +| R1 | AKV RBAC migration breaks existing services | Medium | High | Test in staging first. Keep access policy backup. Immediate rollback plan. | +| R2 | DEK cache memory pressure on high-user-count services | Low | Medium | Configurable max cache size. LRU eviction. Monitor memory usage. | +| R3 | Encryption migration corrupts data | Low | Critical | Dry-run mode first. Backup before migration. Idempotent (skip already-encrypted). | +| R4 | AKV outage blocks all reads/writes | Low | Critical | DEK cache (15 min) provides buffer. Env key provider as emergency fallback. | +| R5 | Performance regression on high-throughput endpoints | Low | Medium | AES-256-GCM is hardware-accelerated. DEK cache avoids AKV round-trips. Benchmark before/after. | +| R6 | E2EE key loss = permanent data loss | Medium | Critical | Recovery phrase (12 words). Prominent UI warnings. Confirmation flow before enabling. | +| R7 | Cosmos RU increase from larger encrypted documents | Low | Low | ~15-20% doc size increase. Monitor RU consumption. | +| R8 | FTS5/search breaks on encrypted fields | Medium | Medium | Maintain plaintext search index or use extraction-service for keywords. | + +--- + +## Success Metrics + +| Metric | Target | Measured By | +| ------------------------ | -------------------------------------------------------- | --------------------------- | +| **Encryption coverage** | 100% of P0 fields encrypted within 8 weeks | Audit endpoint | +| **Zero plaintext leaks** | 0 unencrypted sensitive fields in Cosmos after migration | Audit scan | +| **Performance impact** | < 5% latency increase on encrypted endpoints | Backend metrics | +| **Test coverage** | 120+ new encryption tests | Vitest/XCTest/JUnit reports | +| **DEK cache hit rate** | > 95% | Telemetry | +| **Key rotation time** | < 30 min for full MEK rotation | Runbook timing | +| **Developer experience** | < 10 LOC to encrypt a new field | Code review | + +--- + +## Cost Estimate + +| Item | Monthly Cost | Notes | +| -------------------------- | :-------------: | ------------------------------------------ | +| AKV RSA keys (10 MEKs) | $10-50 | $1/key (software) or $5/key (HSM) | +| AKV operations (DEK wraps) | $0 | Within free tier (10K/month) | +| Cosmos RU increase | ~$2-5 | 15-20% larger docs on encrypted containers | +| Compute (AES-GCM) | ~$0 | Hardware-accelerated, negligible | +| **Total** | **< $60/month** | | + +--- + +## Decision Log + +| Date | Decision | Rationale | +| ---------- | ---------------------------------------- | -------------------------------------------------------------------------- | +| 2026-03-21 | Use AES-256-GCM for all field encryption | AEAD, native on all platforms, NIST approved, matches existing MFA pattern | +| 2026-03-21 | Envelope encryption (MEK → DEK) | Enables key rotation without re-encrypting all data; per-user isolation | +| 2026-03-21 | Tier 3 E2EE only for JarvisJr + NoteLett | Server-side AI in other products requires plaintext access | +| 2026-03-21 | Skip ChronoMind + PeakPulse encryption | Timer configs and GPS stats are not sensitive data | +| 2026-03-21 | RSA-4096 for MEKs | Quantum-resistant margin; AKV native support | +| 2026-03-21 | 15-min DEK cache TTL | Balances AKV rate limits vs key exposure window | +| 2026-03-21 | `__encrypted` sentinel in documents | Enables gradual migration without schema changes | + +--- + +## Appendix: Sprint Checkpoint Verification + +After each sprint, run: + +```bash +# ── Package tests ────────────────────────────────── +cd learning_ai_common_plat && pnpm build && pnpm test + +# ── All product backends ─────────────────────────── +cd learning_voice_ai_agent/backend && npm test # LysnrAI +cd learning_ai_jarvis_jr/backend && npm test # JarvisJr +cd learning_ai_notes/backend && npm test # NoteLett +cd learning_multimodal_memory_agents/backend && npm test # MindLyst +cd learning_ai_fastgap/backend && npm test # NomGap +cd learning_ai_trails/backend && npm test # ActionTrail +cd learning_ai_flowmonk/backend && npm test # FlowMonk +cd learning_ai_local_memory_gpt/backend && npm test # LocalMemGPT + +# ── Platform service ────────────────────────────── +cd learning_ai_common_plat && pnpm --filter @lysnrai/platform-service test + +# ── Type check everything ────────────────────────── +cd learning_ai_common_plat && pnpm typecheck +```