docs(devops): add E2EE design document and implementation roadmap
This commit is contained in:
saravanakumardb1
parent
6856d23a2e
commit
b6a1d637fb
612
docs/devops/END_TO_END_ENCRYPTION_DESIGN.md
Normal file
612
docs/devops/END_TO_END_ENCRYPTION_DESIGN.md
Normal file
@ -0,0 +1,612 @@
|
||||
# ByteLyst — End-to-End Encryption & Data Protection Design
|
||||
|
||||
> **Purpose:** Comprehensive encryption strategy for all ByteLyst ecosystem repos and components.
|
||||
> **Status:** Design document — not yet implemented
|
||||
> **Author:** AI Architecture Review
|
||||
> **Last updated:** 2026-03-21
|
||||
> **See also:** [`AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md`](AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md), [`END_TO_END_ENCRYPTION_ROADMAP.md`](END_TO_END_ENCRYPTION_ROADMAP.md)
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive Summary
|
||||
|
||||
The ByteLyst ecosystem spans 10 products, 3 shared services, 15 repos, and 5 client surfaces (Web, iOS, Android, macOS, watchOS). Data flows through Cosmos DB, Azure Blob Storage, SQLite (local), and in-transit via HTTPS.
|
||||
|
||||
This document defines a **tiered encryption strategy** that balances security, performance, and developer experience:
|
||||
|
||||
| Tier | Scope | Threat Model | Effort |
|
||||
| ---------- | --------------------------------------------- | ------------------------------------------------ | -------------- |
|
||||
| **Tier 0** | Infrastructure encryption (already done) | DB breach at rest, network sniffing | N/A (complete) |
|
||||
| **Tier 1** | Application-layer field encryption | DB admin access, backup leaks, insider threat | Medium |
|
||||
| **Tier 2** | Client-side encryption with server decryption | Reduced server-side exposure window | High |
|
||||
| **Tier 3** | True E2EE (zero-knowledge) | Server compromise, legal subpoena, full distrust | Very High |
|
||||
|
||||
**Recommendation:** Implement Tier 1 ecosystem-wide via a new `@bytelyst/field-encrypt` package. Offer Tier 3 as opt-in "Private Vaults" for JarvisJr and NoteLett only.
|
||||
|
||||
---
|
||||
|
||||
## 2. Current State Audit
|
||||
|
||||
### 2.1 What We Already Have (Tier 0 — Infrastructure)
|
||||
|
||||
| Layer | Protection | Status |
|
||||
| --------------------------------- | -------------------------------- | ----------------------------------- |
|
||||
| **Cosmos DB at rest** | AES-256 (Microsoft-managed keys) | ✅ Enabled by default |
|
||||
| **Cosmos DB in transit** | TLS 1.2+ | ✅ Enforced |
|
||||
| **Azure Blob Storage at rest** | AES-256 (Microsoft-managed keys) | ✅ Enabled by default |
|
||||
| **Azure Blob Storage in transit** | TLS 1.2+ | ✅ Enforced |
|
||||
| **SQLite (LocalMemGPT)** | No encryption (local disk) | ⚠️ Gap |
|
||||
| **Azure Key Vault** | HSM-backed secret storage | ✅ 30 secrets managed |
|
||||
| **JWT tokens** | HS256 signed (RS256 planned) | ✅ Active |
|
||||
| **HTTPS** | TLS 1.2+ on all endpoints | ✅ Enforced |
|
||||
| **iOS Keychain** | Hardware-backed (Secure Enclave) | ✅ via `BLKeychain` |
|
||||
| **Android Keystore** | Hardware-backed (TEE/StrongBox) | ✅ via `BLSecureStore` (AES256-GCM) |
|
||||
| **Secret scanning** | Husky pre-commit + pre-push | ✅ Active |
|
||||
|
||||
### 2.2 Existing Application-Layer Encryption
|
||||
|
||||
The only field-level encryption today is in SmartAuth MFA:
|
||||
|
||||
```
|
||||
platform-service/src/modules/auth/mfa/repository.ts
|
||||
├── encryptSecret() — AES-256-GCM, key from AUTH_TOTP_ENCRYPTION_KEY (hex in AKV)
|
||||
├── decryptSecret() — AES-256-GCM with IV + authTag verification
|
||||
├── hashRecoveryCode() — SHA-256 one-way hash
|
||||
└── generateRecoveryCodes() — crypto.randomBytes
|
||||
```
|
||||
|
||||
**Pattern:** `{ encrypted: string, iv: string, authTag: string }` stored alongside the document. This is the pattern we'll generalize into `@bytelyst/field-encrypt`.
|
||||
|
||||
### 2.3 Client-Side Secure Storage
|
||||
|
||||
| Platform | Component | Encryption | Key Storage |
|
||||
| ----------- | --------------------------------- | --------------------------------------------------------------- | ------------------------ |
|
||||
| **iOS** | `BLKeychain` | Keychain Services (AES-256) | Secure Enclave |
|
||||
| **Android** | `BLSecureStore` | EncryptedSharedPreferences (AES256-SIV keys, AES256-GCM values) | Android Keystore |
|
||||
| **Web** | `localStorage` / `sessionStorage` | None | ⚠️ Gap — no encryption |
|
||||
| **macOS** | `BLKeychain` (shared with iOS) | Keychain Services | Secure Enclave / T2 chip |
|
||||
| **watchOS** | `BLKeychain` (shared with iOS) | Keychain Services | Secure Enclave |
|
||||
|
||||
---
|
||||
|
||||
## 3. Threat Model
|
||||
|
||||
### 3.1 Threats by Severity
|
||||
|
||||
| # | Threat | Likelihood | Impact | Current Mitigation | Gap |
|
||||
| --- | -------------------------------------------------- | ---------- | -------- | -------------------------------------------------------- | ---------------------------------------------------------- |
|
||||
| T1 | **Database breach** (Cosmos DB credentials leaked) | Medium | Critical | AKV, secret rotation runbooks, Cosmos encryption at rest | No app-layer encryption — leaked key exposes all plaintext |
|
||||
| T2 | **Backup/export data leak** | Medium | High | N/A | Backups contain plaintext — no field-level encryption |
|
||||
| T3 | **Insider threat** (DB admin reads data) | Low | High | AKV access policies, RBAC | Cosmos data readable with any valid key |
|
||||
| T4 | **Man-in-the-middle** | Low | High | TLS 1.2+ everywhere | ✅ Covered |
|
||||
| T5 | **Client device theft** | Medium | Medium | iOS Keychain, Android Keystore | Web localStorage is plaintext |
|
||||
| T6 | **Server compromise** | Low | Critical | AKV, managed identities | Server can decrypt everything — no zero-knowledge |
|
||||
| T7 | **Legal subpoena / government request** | Low | High | N/A | Server holds all keys — must comply |
|
||||
| T8 | **Cross-product data leak** | Low | Medium | `productId` field isolation | No cryptographic isolation between products |
|
||||
| T9 | **SQLite local DB theft** (LocalMemGPT) | Low | Medium | OS-level file permissions | SQLite has no encryption |
|
||||
|
||||
### 3.2 What Each Tier Addresses
|
||||
|
||||
| Threat | Tier 0 (Current) | Tier 1 (Field Encrypt) | Tier 2 (Server Decrypt) | Tier 3 (True E2EE) |
|
||||
| --------------------- | :--------------: | :--------------------: | :---------------------: | :----------------: |
|
||||
| T1 Database breach | Partial | ✅ | ✅ | ✅ |
|
||||
| T2 Backup leak | ❌ | ✅ | ✅ | ✅ |
|
||||
| T3 Insider threat | ❌ | Partial | ✅ | ✅ |
|
||||
| T4 MITM | ✅ | ✅ | ✅ | ✅ |
|
||||
| T5 Device theft | Partial | Partial | ✅ | ✅ |
|
||||
| T6 Server compromise | ❌ | ❌ | ❌ | ✅ |
|
||||
| T7 Legal subpoena | ❌ | ❌ | ❌ | ✅ |
|
||||
| T8 Cross-product leak | ❌ | ✅ | ✅ | ✅ |
|
||||
| T9 SQLite theft | ❌ | ❌ | ✅ | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## 4. Tier 1 — Application-Layer Field Encryption
|
||||
|
||||
### 4.1 Architecture
|
||||
|
||||
```
|
||||
┌─────────────┐ ┌──────────────────┐ ┌─────────────┐
|
||||
│ Client │──HTTPS──│ Backend Service │──TLS──→│ Cosmos DB │
|
||||
│ (Web/iOS/ │ │ (Fastify 5) │ │ │
|
||||
│ Android) │ │ │ │ Stores: │
|
||||
└─────────────┘ │ encrypt(field) │ │ { encrypted,│
|
||||
│ before write │ │ iv, │
|
||||
│ │ │ authTag } │
|
||||
│ decrypt(field) │ │ │
|
||||
│ after read │ │ │
|
||||
└──────────────────┘ └─────────────┘
|
||||
│
|
||||
│ DEK wrapped by MEK
|
||||
▼
|
||||
┌──────────────────┐
|
||||
│ Azure Key Vault │
|
||||
│ (MEK storage) │
|
||||
└──────────────────┘
|
||||
```
|
||||
|
||||
### 4.2 Key Hierarchy
|
||||
|
||||
```
|
||||
Master Encryption Key (MEK)
|
||||
├── Stored in Azure Key Vault (HSM-backed)
|
||||
├── Never leaves AKV — used only for wrapping/unwrapping DEKs
|
||||
├── One MEK per product (e.g., lysnr-mek, mindlyst-mek, jarvisjr-mek)
|
||||
└── Rotation: quarterly, with old MEK retained for decryption
|
||||
|
||||
└── Data Encryption Keys (DEKs)
|
||||
├── Generated per-user or per-workspace
|
||||
├── AES-256-GCM symmetric key (32 bytes)
|
||||
├── Wrapped (encrypted) by MEK before storage
|
||||
├── Stored alongside data in Cosmos (as wrapped blob)
|
||||
└── Cached in memory for session duration (max 15 min TTL)
|
||||
```
|
||||
|
||||
**Why envelope encryption (MEK → DEK)?**
|
||||
|
||||
- MEK rotation doesn't require re-encrypting all data — just re-wrap the DEKs
|
||||
- Per-user DEKs provide cryptographic isolation between users
|
||||
- DEK caching avoids an AKV round-trip on every read/write
|
||||
- AKV rate limits (5,000 ops/10s) are not hit because DEK unwrap is infrequent
|
||||
|
||||
### 4.3 `@bytelyst/field-encrypt` Package Design
|
||||
|
||||
```
|
||||
packages/field-encrypt/
|
||||
├── src/
|
||||
│ ├── index.ts # Public API exports
|
||||
│ ├── types.ts # EncryptedField, FieldEncryptOptions, KeyProvider
|
||||
│ ├── aes-gcm.ts # AES-256-GCM encrypt/decrypt (Node.js crypto)
|
||||
│ ├── envelope.ts # DEK generation, MEK wrapping/unwrapping
|
||||
│ ├── key-provider-akv.ts # Azure Key Vault key provider (production)
|
||||
│ ├── key-provider-env.ts # Env-var key provider (dev/test — like current MFA)
|
||||
│ ├── key-provider-memory.ts # In-memory key provider (unit tests)
|
||||
│ ├── key-cache.ts # TTL cache for unwrapped DEKs
|
||||
│ ├── migration.ts # Helpers for encrypting existing plaintext fields
|
||||
│ └── index.test.ts # Tests
|
||||
├── package.json
|
||||
└── tsconfig.json
|
||||
```
|
||||
|
||||
### 4.4 API Design
|
||||
|
||||
```typescript
|
||||
// ── Setup (once per service) ──────────────────────────────
|
||||
import { createFieldEncryptor } from '@bytelyst/field-encrypt';
|
||||
|
||||
const encryptor = createFieldEncryptor({
|
||||
keyProvider: 'akv', // 'akv' | 'env' | 'memory'
|
||||
keyVaultUrl: process.env.AZURE_KEYVAULT_URL,
|
||||
mekName: 'lysnr-mek', // product-specific MEK in AKV
|
||||
dekCacheTtlMs: 15 * 60 * 1000, // 15 min DEK cache
|
||||
});
|
||||
|
||||
// ── Encrypt a field before writing to Cosmos ──────────────
|
||||
const encrypted = await encryptor.encrypt('Hello, this is a transcript', {
|
||||
userId: 'user_123', // DEK scope
|
||||
context: 'transcripts', // additional authenticated data (AAD)
|
||||
});
|
||||
// Returns: { ciphertext: string, iv: string, authTag: string, dekId: string, version: 1 }
|
||||
|
||||
// ── Decrypt a field after reading from Cosmos ─────────────
|
||||
const plaintext = await encryptor.decrypt(encrypted, {
|
||||
userId: 'user_123',
|
||||
context: 'transcripts',
|
||||
});
|
||||
// Returns: 'Hello, this is a transcript'
|
||||
|
||||
// ── Batch encrypt/decrypt (optimized — single DEK unwrap) ─
|
||||
const results = await encryptor.encryptBatch(fields, { userId, context });
|
||||
const plaintexts = await encryptor.decryptBatch(encryptedFields, { userId, context });
|
||||
|
||||
// ── Re-wrap DEKs after MEK rotation ──────────────────────
|
||||
await encryptor.rewrapDeks({ oldMekName: 'lysnr-mek-v1', newMekName: 'lysnr-mek-v2' });
|
||||
|
||||
// ── Migration: encrypt existing plaintext in-place ────────
|
||||
await encryptor.migrateField(cosmosContainer, {
|
||||
fieldName: 'transcriptText',
|
||||
partitionKey: '/userId',
|
||||
batchSize: 100,
|
||||
dryRun: false,
|
||||
});
|
||||
```
|
||||
|
||||
### 4.5 Encrypted Document Schema
|
||||
|
||||
Before encryption:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "transcript_abc123",
|
||||
"userId": "user_123",
|
||||
"productId": "lysnrai",
|
||||
"transcriptText": "Hello, this is a very sensitive transcript...",
|
||||
"createdAt": "2026-03-21T08:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
After encryption:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "transcript_abc123",
|
||||
"userId": "user_123",
|
||||
"productId": "lysnrai",
|
||||
"transcriptText": {
|
||||
"__encrypted": true,
|
||||
"v": 1,
|
||||
"alg": "aes-256-gcm",
|
||||
"ct": "a1b2c3d4e5f6...",
|
||||
"iv": "0102030405060708090a0b0c",
|
||||
"tag": "f1e2d3c4b5a6...",
|
||||
"dekId": "dek_user_123_transcripts"
|
||||
},
|
||||
"createdAt": "2026-03-21T08:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
**Key design decisions:**
|
||||
|
||||
- `__encrypted: true` sentinel allows code to detect encrypted vs plaintext fields during migration
|
||||
- `v: 1` version field enables future algorithm changes without re-encryption
|
||||
- `dekId` identifies which DEK to unwrap — enables per-user key isolation
|
||||
- Non-sensitive fields (`userId`, `productId`, `createdAt`) remain in plaintext for querying
|
||||
- Cosmos DB queries on encrypted fields are impossible by design — use indexed plaintext metadata fields for filtering
|
||||
|
||||
### 4.6 Fields to Encrypt by Product
|
||||
|
||||
| Product | Container | Field(s) to Encrypt | Sensitivity | Priority |
|
||||
| --------------- | ----------------- | --------------------------------------- | ----------- | --------------- |
|
||||
| **LysnrAI** | `transcripts` | `transcriptText`, `rawAudio` (blob ref) | Critical | P0 |
|
||||
| **LysnrAI** | `sessions` | `notes` | High | P1 |
|
||||
| **JarvisJr** | `jarvis_sessions` | `transcript`, `coachingNotes` | Critical | P0 |
|
||||
| **JarvisJr** | `jarvis_memory` | `content` | Critical | P0 |
|
||||
| **NoteLett** | `notes` | `body` | Critical | P0 |
|
||||
| **NoteLett** | `note_artifacts` | `content` | High | P1 |
|
||||
| **MindLyst** | `memory_items` | `content`, `voiceTranscriptText` | Critical | P0 |
|
||||
| **MindLyst** | `brains` | `description` | Medium | P2 |
|
||||
| **MindLyst** | `reflections` | `content` | High | P1 |
|
||||
| **NomGap** | `meal_logs` | `notes` | Low | P3 |
|
||||
| **ChronoMind** | `timers` | (none — low sensitivity) | Low | Defer |
|
||||
| **PeakPulse** | `peak_sessions` | (none — GPS/stats not sensitive) | Low | Defer |
|
||||
| **FlowMonk** | `tasks` | `notes` | Low | P3 |
|
||||
| **ActionTrail** | `trail_actions` | `beforeSnapshot`, `afterSnapshot` | Medium | P2 |
|
||||
| **LocalMemGPT** | SQLite `messages` | `content` | High | P1 |
|
||||
| **SmartAuth** | `auth_mfa` | `encryptedSecret` | Critical | ✅ Already done |
|
||||
|
||||
### 4.7 Query Impact Analysis
|
||||
|
||||
**Fields that are encrypted cannot be queried via Cosmos SQL.** This impacts:
|
||||
|
||||
| Product | Current Query Pattern | Mitigation |
|
||||
| ----------- | -------------------------- | -------------------------------------------------------------------------------------------- |
|
||||
| NoteLett | FTS on `body` | Maintain plaintext search index (separate collection) or use extraction-service |
|
||||
| LocalMemGPT | FTS5 on `content` | FTS5 index operates on plaintext — encrypt at SQLite column level with `sqlcipher` |
|
||||
| MindLyst | Search on `content` | Use extraction-service for entity extraction; store extracted entities as plaintext metadata |
|
||||
| LysnrAI | Search on `transcriptText` | Same: extraction → plaintext keywords |
|
||||
|
||||
**Pattern:** Extract searchable metadata before encryption, store as plaintext alongside encrypted content.
|
||||
|
||||
---
|
||||
|
||||
## 5. Tier 2 — Client-Side Encryption with Server Decryption
|
||||
|
||||
### 5.1 When to Use
|
||||
|
||||
Tier 2 reduces the **exposure window** — data is encrypted on the client, transmitted encrypted, and only decrypted on the server when actively needed (e.g., AI processing). At rest, it's stored encrypted.
|
||||
|
||||
This is relevant for:
|
||||
|
||||
- **JarvisJr voice sessions** — encrypt audio/transcript on-device, decrypt only during PromptBuilder execution
|
||||
- **LysnrAI dictation** — encrypt transcript on-device, decrypt only during cloud processing
|
||||
- **LocalMemGPT RAG** — encrypt documents on upload, decrypt only during embedding generation
|
||||
|
||||
### 5.2 Architecture
|
||||
|
||||
```
|
||||
┌───────────────┐ ┌──────────────────┐ ┌───────────┐
|
||||
│ Client │ encrypted │ Backend │ encrypted │ Cosmos DB │
|
||||
│ │───payload───────→│ │─────────────→│ │
|
||||
│ 1. Generate │ │ 2. Store as-is │ │ (encrypted │
|
||||
│ session │ │ (no decrypt) │ │ at rest + │
|
||||
│ key (SK) │ │ │ │ at app │
|
||||
│ 3. Encrypt │ │ 4. When AI │ │ layer) │
|
||||
│ with SK │ │ needed: │ │ │
|
||||
│ 5. Send SK │ │ unwrap SK │ │ │
|
||||
│ wrapped │ │ from AKV, │ │ │
|
||||
│ by MEK │ │ decrypt, │ │ │
|
||||
│ │ │ process, │ │ │
|
||||
│ │ │ re-encrypt │ │ │
|
||||
└───────────────┘ └──────────────────┘ └───────────┘
|
||||
```
|
||||
|
||||
### 5.3 Client-Side Encryption Primitives
|
||||
|
||||
| Platform | API | Algorithm |
|
||||
| ----------- | ------------------------------- | ----------------- |
|
||||
| **Web** | `SubtleCrypto` (Web Crypto API) | AES-256-GCM |
|
||||
| **iOS** | `CryptoKit` (`AES.GCM`) | AES-256-GCM |
|
||||
| **Android** | `javax.crypto.Cipher` | AES/GCM/NoPadding |
|
||||
| **Node.js** | `node:crypto` | AES-256-GCM |
|
||||
|
||||
All platforms support AES-256-GCM natively — no external dependencies needed.
|
||||
|
||||
### 5.4 New SDK Components
|
||||
|
||||
| SDK | Component | Purpose |
|
||||
| ------------------------- | ------------------------- | ----------------------------------------- |
|
||||
| `@bytelyst/field-encrypt` | `createClientEncryptor()` | Web + React Native client-side encryption |
|
||||
| `swift-platform-sdk` | `BLFieldEncrypt` | iOS/macOS/watchOS client-side encryption |
|
||||
| `kotlin-platform-sdk` | `BLFieldEncrypt` | Android/Wear OS client-side encryption |
|
||||
|
||||
---
|
||||
|
||||
## 6. Tier 3 — True End-to-End Encryption (Zero-Knowledge)
|
||||
|
||||
### 6.1 When to Use
|
||||
|
||||
True E2EE means the **server never sees plaintext** and **cannot decrypt** the data. This is the Signal/ProtonMail model.
|
||||
|
||||
**Applicable products:** JarvisJr (opt-in private agents), NoteLett (opt-in private workspaces).
|
||||
|
||||
**NOT applicable to:**
|
||||
|
||||
- Products requiring server-side AI (NomGap coaching, MindLyst triage, FlowMonk scheduling)
|
||||
- Products requiring server-side search (LocalMemGPT FTS5)
|
||||
- Products where the server IS the authority (ActionTrail oversight, PeakPulse leaderboards)
|
||||
|
||||
### 6.2 Architecture
|
||||
|
||||
```
|
||||
┌───────────────┐ ┌──────────────────┐ ┌───────────┐
|
||||
│ Client │ encrypted │ Backend │ encrypted │ Cosmos DB │
|
||||
│ │───payload───────→│ │─────────────→│ │
|
||||
│ 1. Generate │ │ 2. Store as-is │ │ Ciphertext │
|
||||
│ user │ │ (CANNOT │ │ only — │
|
||||
│ keypair │ │ decrypt) │ │ no keys │
|
||||
│ 2. Encrypt │ │ │ │ │
|
||||
│ with │ │ 3. Return │ │ │
|
||||
│ user's │◀──ciphertext────│ ciphertext │ │ │
|
||||
│ public key │ │ to client │ │ │
|
||||
│ 3. Decrypt │ │ │ │ │
|
||||
│ with │ │ NO MEK/DEK │ │ │
|
||||
│ private │ │ access at all │ │ │
|
||||
│ key │ │ │ │ │
|
||||
└───────────────┘ └──────────────────┘ └───────────┘
|
||||
│
|
||||
│ Private key stored in:
|
||||
│ - iOS Keychain (Secure Enclave)
|
||||
│ - Android Keystore (StrongBox)
|
||||
│ - Web: derived from passphrase (PBKDF2)
|
||||
│
|
||||
│ Multi-device sync via:
|
||||
│ - iCloud Keychain (iOS/Mac)
|
||||
│ - Google Backup (Android)
|
||||
│ - QR code transfer (cross-platform)
|
||||
```
|
||||
|
||||
### 6.3 Key Management Challenges
|
||||
|
||||
| Challenge | Solution | Complexity |
|
||||
| ------------------------ | ------------------------------------------------------------------- | ---------- |
|
||||
| **Key loss = data loss** | Recovery phrase (BIP-39 mnemonic, 12 words) | Medium |
|
||||
| **Multi-device sync** | Platform keychain sync (iCloud/Google) + QR fallback | High |
|
||||
| **Cross-platform** | Export keypair via QR code / encrypted backup file | High |
|
||||
| **Key rotation** | Re-encrypt all data with new key — expensive | High |
|
||||
| **Shared workspaces** | Group key distribution (like Signal group protocol) | Very High |
|
||||
| **Search** | Client-side search only (download + decrypt + search) | Medium |
|
||||
| **AI processing** | Not possible — would need homomorphic encryption or secure enclaves | Blocker |
|
||||
|
||||
### 6.4 Trade-offs
|
||||
|
||||
| Feature | With E2EE | Without E2EE |
|
||||
| --------------------------- | ------------------------------------ | ---------------------------- |
|
||||
| Server-side AI | ❌ Impossible | ✅ Works |
|
||||
| Server-side search | ❌ Impossible | ✅ Works |
|
||||
| MCP tool access | ❌ Impossible | ✅ Works |
|
||||
| Data recovery (lost device) | ⚠️ Recovery phrase only | ✅ Server can restore |
|
||||
| Multi-device sync | ⚠️ Complex key distribution | ✅ Automatic |
|
||||
| Regulatory compliance | ✅ Strong (GDPR Art. 32) | ⚠️ Depends on other controls |
|
||||
| Marketing / trust | ✅ "Zero-knowledge encryption" claim | ⚠️ Standard |
|
||||
|
||||
---
|
||||
|
||||
## 7. SQLite Encryption (LocalMemGPT)
|
||||
|
||||
LocalMemGPT uses SQLite as its primary datastore. SQLite databases are unencrypted files on disk.
|
||||
|
||||
### 7.1 Options
|
||||
|
||||
| Option | Library | License | Effort |
|
||||
| --------------------- | -------------------------------- | ------------------- | ------------------------- |
|
||||
| **SQLCipher** | `better-sqlite3-sqlcipher` | BSD | Low — drop-in replacement |
|
||||
| **SEE** | SQLite Encryption Extension | Commercial ($2,000) | Low |
|
||||
| **Application-level** | Encrypt each field before INSERT | N/A | Medium |
|
||||
|
||||
**Recommendation:** Use `better-sqlite3` with application-level field encryption via `@bytelyst/field-encrypt` (env key provider). This avoids a native dependency change and is consistent with the Cosmos DB approach.
|
||||
|
||||
### 7.2 FTS5 Impact
|
||||
|
||||
If message `content` is encrypted, FTS5 cannot index it. Options:
|
||||
|
||||
1. **Plaintext FTS5 index** — Store plaintext in FTS5 virtual table, encrypted in main table. Risk: FTS5 table is also on disk unencrypted.
|
||||
2. **Client-side search** — Download + decrypt + search in memory. Acceptable for local-first app.
|
||||
3. **SQLCipher** — Encrypts entire database file. FTS5 works normally. Best option if willing to change native dep.
|
||||
|
||||
---
|
||||
|
||||
## 8. Web Client Secure Storage
|
||||
|
||||
Web `localStorage` stores tokens and app state in plaintext. Options:
|
||||
|
||||
| Approach | Protection | Complexity |
|
||||
| ------------------------- | ------------------------------------ | ---------------------- |
|
||||
| **IndexedDB + CryptoKey** | AES-256-GCM with non-extractable key | Medium |
|
||||
| **Session-only storage** | Data lost on tab close | Low |
|
||||
| **Cookie httpOnly** | Server-managed, XSS-proof | Low (auth tokens only) |
|
||||
|
||||
**Recommendation:**
|
||||
|
||||
- Auth tokens → httpOnly cookies (already planned in SmartAuth RS256 migration)
|
||||
- Sensitive cached data → IndexedDB with `SubtleCrypto` non-extractable key
|
||||
- Non-sensitive state → `localStorage` as-is
|
||||
|
||||
---
|
||||
|
||||
## 9. Blob Storage Encryption
|
||||
|
||||
Azure Blob Storage (`bytelystblobs`) stores audio files, meal photos, GPX exports, document uploads. Currently encrypted at rest by Azure (Microsoft-managed keys).
|
||||
|
||||
### 9.1 Upgrade Path
|
||||
|
||||
| Level | Method | Impact |
|
||||
| ----------- | ------------------------------------ | -------------------------------------------- |
|
||||
| **Current** | Azure-managed encryption at rest | Server-side transparent |
|
||||
| **Level 1** | Customer-managed key (CMK) in AKV | Control over key lifecycle; no code changes |
|
||||
| **Level 2** | Client-side encryption before upload | Blobs encrypted before leaving client device |
|
||||
|
||||
**Recommendation:** Enable CMK (Level 1) for the `bytelystblobs` storage account. This is a portal-only change with zero code impact and gives us key rotation control.
|
||||
|
||||
For Level 2, the existing `BLBlobClient` (Swift/Kotlin) and `@bytelyst/blob-client` (TS) would need an encrypt-before-upload wrapper — this naturally integrates with Tier 2.
|
||||
|
||||
---
|
||||
|
||||
## 10. Cross-Cutting Concerns
|
||||
|
||||
### 10.1 Performance Impact
|
||||
|
||||
| Operation | Overhead | Mitigation |
|
||||
| ---------------------------------- | -------------------- | -------------------------------------- |
|
||||
| AES-256-GCM encrypt (1 KB field) | ~0.01 ms (Node.js) | Negligible |
|
||||
| AES-256-GCM encrypt (100 KB field) | ~0.1 ms | Negligible |
|
||||
| DEK unwrap via AKV | ~50-200 ms | DEK cache (15 min TTL) |
|
||||
| MEK rotation (re-wrap DEKs) | ~50 ms/DEK × N users | Background job, non-blocking |
|
||||
| Migration (encrypt existing data) | ~100 docs/sec | Batch migration with progress tracking |
|
||||
|
||||
### 10.2 Compliance Mapping
|
||||
|
||||
| Regulation | Requirement | Tier 0 | Tier 1 | Tier 3 |
|
||||
| ------------------------- | ------------------------------------------ | :-----: | :-----: | :-----: |
|
||||
| **GDPR Art. 32** | Encryption of personal data | Partial | ✅ | ✅ |
|
||||
| **GDPR Art. 34** | Breach notification exemption if encrypted | ❌ | ✅ | ✅ |
|
||||
| **HIPAA** (if applicable) | PHI encryption at rest + in transit | Partial | ✅ | ✅ |
|
||||
| **SOC 2 Type II** | Encryption controls | Partial | ✅ | ✅ |
|
||||
| **Apple App Store** | Encryption disclosure (ERN) | N/A | Declare | Declare |
|
||||
| **Google Play** | Encryption disclosure | N/A | Declare | Declare |
|
||||
|
||||
**GDPR Art. 34 exemption is the key business driver** — if data is encrypted and keys are not compromised, breach notification to individual users is not required. This significantly reduces breach liability.
|
||||
|
||||
### 10.3 Key Rotation Strategy
|
||||
|
||||
| Key Type | Rotation Frequency | Process | Downtime |
|
||||
| ---------------- | -------------------- | -------------------------------------------------------------- | ----------------------------------- |
|
||||
| MEK (in AKV) | Quarterly | Create new MEK version → re-wrap DEKs → deactivate old version | Zero (old MEK retained for decrypt) |
|
||||
| DEK (per-user) | On demand / annually | Generate new DEK → re-encrypt fields → delete old DEK | Zero (background migration) |
|
||||
| SQLite key (env) | On app update | Re-encrypt database with new key | Brief (migration on startup) |
|
||||
|
||||
### 10.4 Monitoring & Alerting
|
||||
|
||||
| Event | Source | Alert |
|
||||
| ------------------------------ | ------------------- | -------- |
|
||||
| MEK access from unknown IP | AKV diagnostic logs | Critical |
|
||||
| Decryption failure spike | App telemetry | High |
|
||||
| DEK cache miss rate > 50% | Backend metrics | Medium |
|
||||
| Migration progress stall | Background job | Medium |
|
||||
| Encrypted field count mismatch | Audit job | Low |
|
||||
|
||||
---
|
||||
|
||||
## 11. Decision Matrix — What to Encrypt Where
|
||||
|
||||
| Product | Server-Side AI? | Server-Side Search? | Recommended Tier | Justification |
|
||||
| --------------- | :-----------------: | :-----------------: | :----------------------: | ------------------------------------------------------------ |
|
||||
| **LysnrAI** | Yes (extraction) | Yes (transcripts) | Tier 1 | Server needs plaintext for extraction; field-encrypt at rest |
|
||||
| **JarvisJr** | Yes (PromptBuilder) | No | Tier 1 + Tier 3 opt-in | Default field-encrypt; offer E2EE private agents |
|
||||
| **NoteLett** | Yes (MCP tools) | Yes (note search) | Tier 1 + Tier 3 opt-in | Default field-encrypt; offer E2EE private workspaces |
|
||||
| **MindLyst** | Yes (triage) | Yes (memory search) | Tier 1 | Server needs plaintext for AI triage pipeline |
|
||||
| **NomGap** | Yes (AI coach) | No | Tier 1 (meal notes only) | Low sensitivity overall; encrypt meal notes |
|
||||
| **LocalMemGPT** | Yes (Ollama) | Yes (FTS5) | Tier 1 + SQLCipher | Local-first; SQLCipher for full-DB encryption |
|
||||
| **ChronoMind** | No | No | Tier 0 (current) | Timer configs are not sensitive |
|
||||
| **PeakPulse** | No | No | Tier 0 (current) | GPS/stats are not sensitive |
|
||||
| **FlowMonk** | Yes (scheduler) | No | Tier 1 (task notes only) | Backend is authoritative; encrypt task notes |
|
||||
| **ActionTrail** | No | Yes (action search) | Tier 1 (snapshots only) | Oversight requires readable actions; encrypt snapshots |
|
||||
|
||||
---
|
||||
|
||||
## 12. Dependencies & Prerequisites
|
||||
|
||||
| Prerequisite | Status | Blocks |
|
||||
| --------------------------------------------------------- | -------------------------------------- | ---------------------------------- |
|
||||
| Azure Key Vault RBAC mode | ⚠️ Planned (currently access policies) | Tier 1 MEK management |
|
||||
| AKV `encrypt`/`decrypt`/`wrapKey`/`unwrapKey` permissions | Not configured | Tier 1 envelope encryption |
|
||||
| `@azure/keyvault-keys` npm package | Available (not installed) | Tier 1 AKV key provider |
|
||||
| RS256 JWT migration (SmartAuth Phase 4) | Planned | Tier 2 key distribution |
|
||||
| iCloud Keychain sharing entitlements | Not configured | Tier 3 iOS key sync |
|
||||
| QR code library (iOS/Android) | Not installed | Tier 3 cross-platform key transfer |
|
||||
| `better-sqlite3-sqlcipher` or app-level encrypt | Not installed | LocalMemGPT SQLite encryption |
|
||||
| CMK configuration on `bytelystblobs` | Not configured | Blob storage Level 1 |
|
||||
|
||||
---
|
||||
|
||||
## 13. Related Documents
|
||||
|
||||
| Document | Purpose |
|
||||
| -------------------------------------------------------------------------------------------- | ----------------------------------------- |
|
||||
| [`END_TO_END_ENCRYPTION_ROADMAP.md`](END_TO_END_ENCRYPTION_ROADMAP.md) | Phased implementation plan with timelines |
|
||||
| [`AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md`](AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md) | Current AKV setup and rotation runbooks |
|
||||
| [`AZURE_RESOURCE_INVENTORY.md`](AZURE_RESOURCE_INVENTORY.md) | Azure resource inventory |
|
||||
| [`ENVIRONMENT_VARIABLES_AND_KEYVAULT_AUDIT.md`](ENVIRONMENT_VARIABLES_AND_KEYVAULT_AUDIT.md) | Env var and AKV audit |
|
||||
| `../architecture/ECOSYSTEM_ARCHITECTURE.md` | ByteLyst ecosystem architecture |
|
||||
| `../../services/platform-service/src/modules/auth/mfa/repository.ts` | Existing AES-256-GCM pattern |
|
||||
| `../../packages/kotlin-platform-sdk/src/main/kotlin/.../BLSecureStore.kt` | Android EncryptedSharedPreferences |
|
||||
| `../../packages/swift-platform-sdk/Sources/BLKeychain.swift` | iOS Keychain wrapper |
|
||||
|
||||
---
|
||||
|
||||
## Appendix A: Algorithm Selection Rationale
|
||||
|
||||
| Algorithm | Use Case | Why |
|
||||
| ----------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
|
||||
| **AES-256-GCM** | Field encryption (all tiers) | AEAD — provides confidentiality + integrity + authentication. Native support on all platforms. NIST approved. |
|
||||
| **RSA-OAEP-256** | MEK wrapping in AKV | Azure Key Vault natively supports RSA key wrapping. 4096-bit RSA for quantum-resistant margin. |
|
||||
| **PBKDF2-SHA256** | Web passphrase → key derivation | Web Crypto API native. 600,000 iterations (OWASP 2024 recommendation). |
|
||||
| **X25519** | Tier 3 key exchange | Modern elliptic-curve DH. Used by Signal, WireGuard. Compact (32-byte keys). |
|
||||
| **HKDF-SHA256** | Tier 3 key derivation from shared secret | Standard KDF for deriving symmetric keys from DH output. |
|
||||
|
||||
## Appendix B: Encrypted Field Detection Pattern
|
||||
|
||||
Repositories can use a type guard to handle both encrypted and plaintext fields during migration:
|
||||
|
||||
```typescript
|
||||
import type { EncryptedField } from '@bytelyst/field-encrypt';
|
||||
|
||||
function isEncrypted(value: unknown): value is EncryptedField {
|
||||
return (
|
||||
typeof value === 'object' &&
|
||||
value !== null &&
|
||||
'__encrypted' in value &&
|
||||
(value as Record<string, unknown>).__encrypted === true
|
||||
);
|
||||
}
|
||||
|
||||
// Usage in repository
|
||||
async function getNote(id: string): Promise<Note> {
|
||||
const doc = await collection.findById(id, partitionKey);
|
||||
if (isEncrypted(doc.body)) {
|
||||
doc.body = await encryptor.decrypt(doc.body, { userId: doc.userId, context: 'notes' });
|
||||
}
|
||||
return doc;
|
||||
}
|
||||
```
|
||||
|
||||
## Appendix C: Cost Impact
|
||||
|
||||
| Resource | Current | After Tier 1 | Notes |
|
||||
| -------------- | ----------------------- | ------------------------------- | ------------------------------------------------------------------- |
|
||||
| AKV operations | ~100/day (secret reads) | ~500/day (+DEK unwraps) | Well within free tier (10K ops/month free) |
|
||||
| AKV keys | 0 RSA keys | 10 RSA keys (1 MEK per product) | $1/key/month for software-protected; $5 for HSM |
|
||||
| Cosmos RU | Baseline | +5-10% (larger encrypted docs) | Encrypted fields are ~30% larger than plaintext (base64 + metadata) |
|
||||
| Compute | Baseline | +1-2% CPU | AES-256-GCM is hardware-accelerated (AES-NI) on all modern CPUs |
|
||||
| Storage | Baseline | +15-20% for encrypted fields | Base64 encoding + IV + authTag + dekId overhead |
|
||||
|
||||
**Estimated monthly cost increase: < $5** (dominated by AKV key charges).
|
||||
689
docs/devops/END_TO_END_ENCRYPTION_ROADMAP.md
Normal file
689
docs/devops/END_TO_END_ENCRYPTION_ROADMAP.md
Normal file
@ -0,0 +1,689 @@
|
||||
# ByteLyst — End-to-End Encryption Implementation Roadmap
|
||||
|
||||
> **Purpose:** Phased implementation plan for encryption across the ByteLyst ecosystem.
|
||||
> **Status:** Roadmap — not yet started
|
||||
> **Author:** AI Architecture Review
|
||||
> **Last updated:** 2026-03-21
|
||||
> **Design doc:** [`END_TO_END_ENCRYPTION_DESIGN.md`](END_TO_END_ENCRYPTION_DESIGN.md)
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
This roadmap implements the tiered encryption strategy defined in the design document. Work is organized into 6 sprints across 3 phases, estimated at **12-16 weeks** total elapsed time.
|
||||
|
||||
### Phase Summary
|
||||
|
||||
| Phase | Sprints | Duration | Deliverable |
|
||||
| ------------------------------ | ---------- | --------- | ------------------------------------------------------------------------------------- |
|
||||
| **Phase 1: Foundation** | Sprint 1-2 | 4-5 weeks | `@bytelyst/field-encrypt` package + AKV infrastructure + first 2 product integrations |
|
||||
| **Phase 2: Ecosystem Rollout** | Sprint 3-4 | 4-5 weeks | All 10 product backends encrypted + native SDK components + web secure storage |
|
||||
| **Phase 3: Advanced** | Sprint 5-6 | 4-6 weeks | True E2EE for JarvisJr/NoteLett + SQLite encryption + blob CMK + audit tooling |
|
||||
|
||||
### Milestone Timeline
|
||||
|
||||
```
|
||||
Week 1-2 Week 3-4 Week 5-6 Week 7-8 Week 9-10 Week 11-14
|
||||
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────┐
|
||||
│ Sprint 1 │ │ Sprint 2 │ │ Sprint 3 │ │ Sprint 4 │ │ Sprint 5 │ │ Sprint 6 │
|
||||
│ Package + │ │ LysnrAI +│ │ 4 more │ │ Native │ │ E2EE │ │ Audit + │
|
||||
│ AKV infra │ │ JarvisJr │ │ products │ │ SDKs + │ │ private │ │ polish + │
|
||||
│ │ │ │ │ │ │ web │ │ vaults │ │ docs │
|
||||
└──────────┘ └──────────┘ └──────────┘ └──────────┘ └──────────┘ └──────────────┘
|
||||
Phase 1 Phase 1 Phase 2 Phase 2 Phase 3 Phase 3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Foundation (Weeks 1-5)
|
||||
|
||||
### Sprint 1 — `@bytelyst/field-encrypt` Package + AKV Infrastructure (Week 1-2)
|
||||
|
||||
**Goal:** Ship the shared encryption package and configure Azure Key Vault for key management.
|
||||
|
||||
#### 1.1 Azure Key Vault Infrastructure
|
||||
|
||||
- [ ] **1.1.1** Enable RBAC mode on `kv-mywisprai` vault (`enableRbacAuthorization=true`)
|
||||
- Assign `Key Vault Crypto Officer` to deployer user
|
||||
- Assign `Key Vault Crypto User` to service managed identities (when deployed)
|
||||
- **File:** Azure Portal / `az keyvault update`
|
||||
- **Risk:** Breaking change — all existing access policies stop working. Must reassign as RBAC roles simultaneously.
|
||||
- **Rollback:** `az keyvault update --enable-rbac-authorization false` + re-create access policies
|
||||
|
||||
- [ ] **1.1.2** Create 10 Master Encryption Keys (MEKs) in AKV
|
||||
|
||||
```
|
||||
lysnr-mek, mindlyst-mek, jarvisjr-mek, chronomind-mek,
|
||||
nomgap-mek, peakpulse-mek, flowmonk-mek, actiontrail-mek,
|
||||
notelett-mek, localmemgpt-mek
|
||||
```
|
||||
|
||||
- RSA 4096-bit, `wrapKey` + `unwrapKey` operations enabled
|
||||
- **Script:** `scripts/create-encryption-keys.sh`
|
||||
|
||||
- [ ] **1.1.3** Add new env vars to `.env.example` and AKV secrets doc
|
||||
```
|
||||
AZURE_KEYVAULT_URL=https://kv-mywisprai.vault.azure.net
|
||||
FIELD_ENCRYPT_KEY_PROVIDER=akv # 'akv' | 'env' | 'memory'
|
||||
FIELD_ENCRYPT_MEK_NAME=lysnr-mek # product-specific
|
||||
```
|
||||
|
||||
- **Files:** `.env.example`, `docs/devops/ENVIRONMENT_VARIABLES_AND_KEYVAULT_AUDIT.md`
|
||||
|
||||
#### 1.2 `@bytelyst/field-encrypt` Package
|
||||
|
||||
- [ ] **1.2.1** Create package scaffold
|
||||
|
||||
```
|
||||
packages/field-encrypt/
|
||||
├── src/
|
||||
│ ├── index.ts
|
||||
│ ├── types.ts
|
||||
│ ├── aes-gcm.ts
|
||||
│ ├── envelope.ts
|
||||
│ ├── key-provider-akv.ts
|
||||
│ ├── key-provider-env.ts
|
||||
│ ├── key-provider-memory.ts
|
||||
│ ├── key-cache.ts
|
||||
│ ├── migration.ts
|
||||
│ └── index.test.ts
|
||||
├── package.json
|
||||
└── tsconfig.json
|
||||
```
|
||||
|
||||
- **Dependencies:** `@azure/keyvault-keys`, `@azure/identity` (peer), `zod` (peer)
|
||||
- **Dev dependencies:** `vitest`
|
||||
|
||||
- [ ] **1.2.2** Implement core AES-256-GCM encrypt/decrypt
|
||||
- `aes-gcm.ts` — `encryptField()`, `decryptField()` using `node:crypto`
|
||||
- Input: plaintext string + key buffer + optional AAD
|
||||
- Output: `EncryptedField` object `{ __encrypted, v, alg, ct, iv, tag, dekId }`
|
||||
- **Tests:** encrypt → decrypt roundtrip, invalid authTag rejection, empty string, unicode, large payload (1 MB)
|
||||
|
||||
- [ ] **1.2.3** Implement envelope encryption (DEK management)
|
||||
- `envelope.ts` — `generateDek()`, `wrapDek()`, `unwrapDek()`
|
||||
- DEK: 32-byte random AES key, identified by `dek_{userId}_{context}`
|
||||
- Wrap: RSA-OAEP with MEK in AKV
|
||||
- Stored: `{ dekId, wrappedKey, mekVersion, createdAt }` in a `_encryption_keys` Cosmos container
|
||||
- **Tests:** generate → wrap → unwrap roundtrip, MEK version tracking
|
||||
|
||||
- [ ] **1.2.4** Implement key providers
|
||||
- `key-provider-akv.ts` — Production: `@azure/keyvault-keys` `CryptographyClient`
|
||||
- `key-provider-env.ts` — Dev/staging: raw hex key from `FIELD_ENCRYPT_KEY` env var (like current MFA pattern)
|
||||
- `key-provider-memory.ts` — Unit tests: random in-memory key, no external deps
|
||||
- **Tests:** provider interface contract tests for each
|
||||
|
||||
- [ ] **1.2.5** Implement DEK cache
|
||||
- `key-cache.ts` — In-memory LRU cache with configurable TTL (default 15 min)
|
||||
- Cache key: `dekId`, value: unwrapped DEK buffer
|
||||
- Auto-evict on TTL expiry, manual `invalidate(dekId)` for rotation
|
||||
- **Tests:** cache hit/miss, TTL expiry, manual invalidation, max size eviction
|
||||
|
||||
- [ ] **1.2.6** Implement `createFieldEncryptor()` factory
|
||||
- Wires key provider + cache + AES-GCM
|
||||
- Public API: `encrypt()`, `decrypt()`, `encryptBatch()`, `decryptBatch()`, `isEncrypted()`
|
||||
- **Tests:** full integration test with memory provider
|
||||
|
||||
- [ ] **1.2.7** Implement migration helpers
|
||||
- `migration.ts` — `migrateField()`: read plaintext, encrypt, write back (idempotent)
|
||||
- Supports batch size, dry run, progress callback
|
||||
- Detects already-encrypted fields via `__encrypted` sentinel
|
||||
- **Tests:** migrate plaintext → encrypted, skip already-encrypted, dry run mode
|
||||
|
||||
- [ ] **1.2.8** Add `isEncrypted()` type guard
|
||||
- Exported utility for repositories to detect encrypted vs plaintext during migration period
|
||||
- **Tests:** true for EncryptedField, false for string, false for null/undefined
|
||||
|
||||
**Sprint 1 deliverable:** `@bytelyst/field-encrypt` package with all 3 key providers, ~30 tests, `pnpm build && pnpm test` passing.
|
||||
|
||||
**Commit:** `feat(field-encrypt): create @bytelyst/field-encrypt package with AES-256-GCM envelope encryption`
|
||||
|
||||
---
|
||||
|
||||
### Sprint 2 — First Product Integrations: LysnrAI + JarvisJr (Week 3-4)
|
||||
|
||||
**Goal:** Encrypt the highest-sensitivity fields in 2 products. Validate the pattern end-to-end.
|
||||
|
||||
#### 2.1 LysnrAI Backend (port 4015)
|
||||
|
||||
- [ ] **2.1.1** Add `@bytelyst/field-encrypt` dependency to `backend/package.json`
|
||||
|
||||
```json
|
||||
"@bytelyst/field-encrypt": "file:../../learning_ai_common_plat/packages/field-encrypt"
|
||||
```
|
||||
|
||||
- [ ] **2.1.2** Create `backend/src/lib/field-encrypt.ts` — service-level encryptor singleton
|
||||
|
||||
```typescript
|
||||
import { createFieldEncryptor } from '@bytelyst/field-encrypt';
|
||||
import { config } from './config.js';
|
||||
|
||||
export const encryptor = createFieldEncryptor({
|
||||
keyProvider: config.FIELD_ENCRYPT_KEY_PROVIDER ?? 'memory',
|
||||
mekName: 'lysnr-mek',
|
||||
keyVaultUrl: config.AZURE_KEYVAULT_URL,
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **2.1.3** Encrypt `transcriptText` in transcripts module
|
||||
- `repository.ts`: encrypt on `create()`, decrypt on `findById()` and `findByUserId()`
|
||||
- Keep `transcriptText` field name but value changes from string to EncryptedField
|
||||
- **Migration:** Add `POST /api/admin/migrate-encryption` endpoint (admin-only) that batch-encrypts existing plaintext
|
||||
|
||||
- [ ] **2.1.4** Update config schema with encryption env vars
|
||||
|
||||
```typescript
|
||||
FIELD_ENCRYPT_KEY_PROVIDER: z.enum(['akv', 'env', 'memory']).default('memory'),
|
||||
FIELD_ENCRYPT_KEY: z.string().optional(), // only for 'env' provider
|
||||
AZURE_KEYVAULT_URL: z.string().url().optional(),
|
||||
```
|
||||
|
||||
- [ ] **2.1.5** Update tests — ensure existing tests pass with memory provider (no env vars needed)
|
||||
|
||||
**Commit:** `feat(lysnrai): encrypt transcriptText field with @bytelyst/field-encrypt`
|
||||
|
||||
#### 2.2 JarvisJr Backend (port 4012)
|
||||
|
||||
- [ ] **2.2.1** Add `@bytelyst/field-encrypt` dependency
|
||||
|
||||
- [ ] **2.2.2** Create `backend/src/lib/field-encrypt.ts` encryptor singleton
|
||||
|
||||
- [ ] **2.2.3** Encrypt fields in jarvis-sessions module
|
||||
- `transcript` field — session voice transcripts
|
||||
- `coachingNotes` field — extracted coaching insights
|
||||
|
||||
- [ ] **2.2.4** Encrypt fields in jarvis-memory module
|
||||
- `content` field — per-agent persistent memory entries
|
||||
|
||||
- [ ] **2.2.5** Update config + tests
|
||||
|
||||
**Commit:** `feat(jarvisjr): encrypt session transcripts, coaching notes, and agent memory`
|
||||
|
||||
#### 2.3 Refactor MFA to Use Shared Package
|
||||
|
||||
- [ ] **2.3.1** Migrate `platform-service/src/modules/auth/mfa/repository.ts` from inline AES-256-GCM to `@bytelyst/field-encrypt`
|
||||
- Replace `encryptSecret()` / `decryptSecret()` with `encryptor.encrypt()` / `encryptor.decrypt()`
|
||||
- Remove inline `createCipheriv` / `createDecipheriv` code
|
||||
- Keep `AUTH_TOTP_ENCRYPTION_KEY` as `env` key provider for backward compatibility
|
||||
- **Tests:** All existing MFA tests must still pass
|
||||
|
||||
**Commit:** `refactor(auth): migrate MFA encryption to @bytelyst/field-encrypt`
|
||||
|
||||
**Sprint 2 deliverable:** 2 product backends fully encrypted, MFA migrated, all tests green.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Ecosystem Rollout (Weeks 5-10)
|
||||
|
||||
### Sprint 3 — Four More Product Backends (Week 5-6)
|
||||
|
||||
**Goal:** Encrypt sensitive fields in NoteLett, MindLyst, NomGap, and ActionTrail.
|
||||
|
||||
#### 3.1 NoteLett Backend (port 4016)
|
||||
|
||||
- [ ] **3.1.1** Add dependency + create encryptor singleton
|
||||
- [ ] **3.1.2** Encrypt `body` field in notes module
|
||||
- [ ] **3.1.3** Encrypt `content` field in note-artifacts module
|
||||
- [ ] **3.1.4** Update MCP tools to decrypt before processing
|
||||
- [ ] **3.1.5** Update tests
|
||||
|
||||
**Commit:** `feat(notelett): encrypt note body and artifact content`
|
||||
|
||||
#### 3.2 MindLyst Backend (port 4014)
|
||||
|
||||
- [ ] **3.2.1** Add dependency + create encryptor singleton
|
||||
- [ ] **3.2.2** Encrypt `content` and `voiceTranscriptText` in memory_items module
|
||||
- [ ] **3.2.3** Encrypt `content` in reflections module
|
||||
- [ ] **3.2.4** Update tests
|
||||
|
||||
**Commit:** `feat(mindlyst): encrypt memory content, voice transcripts, and reflections`
|
||||
|
||||
#### 3.3 NomGap Backend (port 4013)
|
||||
|
||||
- [ ] **3.3.1** Add dependency + create encryptor singleton
|
||||
- [ ] **3.3.2** Encrypt `notes` field in meal-logs module (low sensitivity, but consistent)
|
||||
- [ ] **3.3.3** Update tests
|
||||
|
||||
**Commit:** `feat(nomgap): encrypt meal log notes`
|
||||
|
||||
#### 3.4 ActionTrail Backend (port 4018)
|
||||
|
||||
- [ ] **3.4.1** Add dependency + create encryptor singleton
|
||||
- [ ] **3.4.2** Encrypt `beforeSnapshot` and `afterSnapshot` in actions module
|
||||
- [ ] **3.4.3** Encrypt `beforeSnapshot` and `afterSnapshot` in reverts module
|
||||
- [ ] **3.4.4** Update tests
|
||||
|
||||
**Commit:** `feat(actiontrail): encrypt action and revert snapshots`
|
||||
|
||||
#### 3.5 Remaining Backends (Low Priority — Defer or Skip)
|
||||
|
||||
| Product | Backend Port | Encrypted Fields | Decision |
|
||||
| ---------------------- | ------------------ | ----------------------------------- | -------- |
|
||||
| **FlowMonk** (4017) | `tasks.notes` | Defer to Sprint 4 — low sensitivity |
|
||||
| **ChronoMind** (4011) | None | Skip — timer configs not sensitive |
|
||||
| **PeakPulse** (4010) | None | Skip — GPS/stats not sensitive |
|
||||
| **LocalMemGPT** (4019) | `messages.content` | Sprint 5 (SQLite-specific approach) |
|
||||
|
||||
**Sprint 3 deliverable:** 6 product backends encrypted (LysnrAI, JarvisJr, NoteLett, MindLyst, NomGap, ActionTrail).
|
||||
|
||||
---
|
||||
|
||||
### Sprint 4 — Native SDKs + Web Secure Storage + FlowMonk (Week 7-8)
|
||||
|
||||
**Goal:** Add client-side encryption primitives to all SDKs. Secure web localStorage.
|
||||
|
||||
#### 4.1 Swift Platform SDK — `BLFieldEncrypt`
|
||||
|
||||
- [ ] **4.1.1** Create `Sources/BLFieldEncrypt.swift` in `packages/swift-platform-sdk/`
|
||||
|
||||
```swift
|
||||
import CryptoKit
|
||||
|
||||
public struct BLFieldEncrypt {
|
||||
/// Encrypt a string field with AES-256-GCM
|
||||
public static func encrypt(_ plaintext: String, key: SymmetricKey) -> EncryptedField
|
||||
/// Decrypt an encrypted field
|
||||
public static func decrypt(_ field: EncryptedField, key: SymmetricKey) -> String?
|
||||
/// Check if a JSON value is an encrypted field
|
||||
public static func isEncrypted(_ value: Any) -> Bool
|
||||
}
|
||||
|
||||
public struct EncryptedField: Codable {
|
||||
public let __encrypted: Bool // always true
|
||||
public let v: Int // version
|
||||
public let alg: String // "aes-256-gcm"
|
||||
public let ct: String // ciphertext (base64)
|
||||
public let iv: String // IV (hex)
|
||||
public let tag: String // auth tag (hex)
|
||||
public let dekId: String // DEK identifier
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **4.1.2** Key derivation from Keychain-stored secret
|
||||
- [ ] **4.1.3** Unit tests (XCTest)
|
||||
|
||||
**Commit:** `feat(swift-sdk): add BLFieldEncrypt for client-side AES-256-GCM encryption`
|
||||
|
||||
#### 4.2 Kotlin Platform SDK — `BLFieldEncrypt`
|
||||
|
||||
- [ ] **4.2.1** Create `src/main/.../BLFieldEncrypt.kt` in `packages/kotlin-platform-sdk/`
|
||||
```kotlin
|
||||
object BLFieldEncrypt {
|
||||
fun encrypt(plaintext: String, key: SecretKeySpec): EncryptedField
|
||||
fun decrypt(field: EncryptedField, key: SecretKeySpec): String?
|
||||
fun isEncrypted(value: Any?): Boolean
|
||||
}
|
||||
```
|
||||
- [ ] **4.2.2** Key derivation from BLSecureStore
|
||||
- [ ] **4.2.3** Unit tests (JUnit5)
|
||||
|
||||
**Commit:** `feat(kotlin-sdk): add BLFieldEncrypt for client-side AES-256-GCM encryption`
|
||||
|
||||
#### 4.3 TypeScript Client Package — `@bytelyst/client-encrypt`
|
||||
|
||||
- [ ] **4.3.1** Create `packages/client-encrypt/` for browser + React Native
|
||||
```typescript
|
||||
// Uses Web Crypto API (SubtleCrypto) — works in browsers and React Native
|
||||
export function encryptField(plaintext: string, key: CryptoKey): Promise<EncryptedField>;
|
||||
export function decryptField(field: EncryptedField, key: CryptoKey): Promise<string>;
|
||||
export function deriveKey(passphrase: string, salt: Uint8Array): Promise<CryptoKey>;
|
||||
```
|
||||
- [ ] **4.3.2** PBKDF2 key derivation for web (600,000 iterations)
|
||||
- [ ] **4.3.3** Tests with vitest + happy-dom
|
||||
|
||||
**Commit:** `feat(client-encrypt): create @bytelyst/client-encrypt for browser/RN encryption`
|
||||
|
||||
#### 4.4 Web Secure Storage
|
||||
|
||||
- [ ] **4.4.1** Create `@bytelyst/secure-storage-web` package
|
||||
- IndexedDB-backed storage with `SubtleCrypto` non-extractable AES key
|
||||
- API: `secureGet(key)`, `secureSet(key, value)`, `secureDelete(key)`, `secureClear()`
|
||||
- Key is generated once, stored as non-extractable CryptoKey in IndexedDB
|
||||
- Falls back to `localStorage` if `SubtleCrypto` unavailable
|
||||
|
||||
- [ ] **4.4.2** Migrate auth tokens from `localStorage` to secure storage in all web apps
|
||||
- Update `@bytelyst/auth-client` to use secure storage
|
||||
- **Products affected:** All 7 web dashboards/apps
|
||||
|
||||
**Commit:** `feat(secure-storage-web): IndexedDB + SubtleCrypto encrypted storage for web`
|
||||
|
||||
#### 4.5 FlowMonk Backend (port 4017)
|
||||
|
||||
- [ ] **4.5.1** Add dependency + encrypt `tasks.notes` field
|
||||
- [ ] **4.5.2** Update tests
|
||||
|
||||
**Commit:** `feat(flowmonk): encrypt task notes`
|
||||
|
||||
**Sprint 4 deliverable:** All native SDKs have encryption primitives. Web secure storage. 7 backends encrypted.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Advanced Encryption (Weeks 9-14)
|
||||
|
||||
### Sprint 5 — True E2EE Private Vaults + SQLite Encryption (Week 9-10)
|
||||
|
||||
**Goal:** Implement zero-knowledge encryption for opt-in private spaces in JarvisJr and NoteLett. Encrypt LocalMemGPT SQLite.
|
||||
|
||||
#### 5.1 E2EE Key Management Infrastructure
|
||||
|
||||
- [ ] **5.1.1** Design E2EE key exchange protocol
|
||||
- User generates X25519 keypair on device
|
||||
- Public key uploaded to platform-service `user_public_keys` container
|
||||
- Private key stored in Keychain (iOS/Mac) / Keystore (Android) / derived from passphrase (Web)
|
||||
- Recovery: 12-word BIP-39 mnemonic → seed → keypair
|
||||
|
||||
- [ ] **5.1.2** Create `@bytelyst/e2ee-core` package
|
||||
|
||||
```typescript
|
||||
export function generateKeypair(): { publicKey: Uint8Array; privateKey: Uint8Array };
|
||||
export function generateRecoveryPhrase(): string; // 12 BIP-39 words
|
||||
export function keypairFromRecovery(phrase: string): { publicKey; privateKey };
|
||||
export function encryptForUser(
|
||||
plaintext: string,
|
||||
recipientPublicKey: Uint8Array
|
||||
): EncryptedPayload;
|
||||
export function decryptForUser(payload: EncryptedPayload, privateKey: Uint8Array): string;
|
||||
```
|
||||
|
||||
- Uses X25519 for key agreement, HKDF-SHA256 for derivation, AES-256-GCM for encryption
|
||||
- **Dependencies:** `@noble/curves` (X25519), `@noble/hashes` (HKDF, SHA256)
|
||||
- **Tests:** ~20 tests
|
||||
|
||||
- [ ] **5.1.3** Add `user_public_keys` container to platform-service
|
||||
- Schema: `{ id, userId, productId, publicKey (base64), deviceId, createdAt }`
|
||||
- Partition key: `/userId`
|
||||
- Routes: `POST /auth/e2ee/register-key`, `GET /auth/e2ee/public-key/:userId`
|
||||
|
||||
- [ ] **5.1.4** Multi-device key sync
|
||||
- iOS: Share private key via iCloud Keychain (requires entitlement)
|
||||
- Android: Share via Google Backup (EncryptedSharedPreferences)
|
||||
- Cross-platform: QR code containing encrypted private key (encrypted with passphrase)
|
||||
|
||||
**Commit:** `feat(e2ee): create @bytelyst/e2ee-core + platform key registration`
|
||||
|
||||
#### 5.2 JarvisJr E2EE Private Agents
|
||||
|
||||
- [ ] **5.2.1** Add `isPrivate` boolean field to agent config
|
||||
- When `true`, all session transcripts and memory for this agent are E2EE
|
||||
- Server stores only ciphertext — cannot decrypt
|
||||
- AI features (PromptBuilder, memory retrieval) disabled for private agents
|
||||
- UI shows "Private Agent" badge + "AI features unavailable" warning
|
||||
|
||||
- [ ] **5.2.2** Client-side encrypt before API calls (iOS + Web)
|
||||
- Session transcript → `e2eeCore.encryptForUser(transcript, userPublicKey)`
|
||||
- Memory content → `e2eeCore.encryptForUser(content, userPublicKey)`
|
||||
- Client-side decrypt after reads
|
||||
|
||||
- [ ] **5.2.3** Update JarvisJr web + iOS to handle E2EE toggle
|
||||
|
||||
**Commit:** `feat(jarvisjr): E2EE private agents with zero-knowledge encryption`
|
||||
|
||||
#### 5.3 NoteLett E2EE Private Workspaces
|
||||
|
||||
- [ ] **5.3.1** Add `isEncrypted` boolean field to workspace config
|
||||
- When `true`, all notes in workspace are E2EE
|
||||
- MCP tools return encrypted content (agent cannot read)
|
||||
- Search disabled for encrypted workspaces
|
||||
|
||||
- [ ] **5.3.2** Client-side encrypt before API calls (Web + Mobile)
|
||||
- [ ] **5.3.3** Update UI with "Encrypted Workspace" badge + trade-off warnings
|
||||
|
||||
**Commit:** `feat(notelett): E2EE private workspaces with zero-knowledge encryption`
|
||||
|
||||
#### 5.4 LocalMemGPT SQLite Encryption
|
||||
|
||||
- [ ] **5.4.1** Evaluate `better-sqlite3-sqlcipher` vs application-level encryption
|
||||
- If SQLCipher: swap `better-sqlite3` → `better-sqlite3-sqlcipher`, add `PRAGMA key`
|
||||
- If app-level: encrypt `content` column via `@bytelyst/field-encrypt` (env provider)
|
||||
|
||||
- [ ] **5.4.2** Implement chosen approach
|
||||
- [ ] **5.4.3** Handle FTS5 index (see design doc §7.2)
|
||||
- [ ] **5.4.4** Migration: encrypt existing unencrypted databases on first startup
|
||||
|
||||
**Commit:** `feat(localmemgpt): encrypt SQLite message content`
|
||||
|
||||
**Sprint 5 deliverable:** E2EE private vaults for JarvisJr + NoteLett. SQLite encryption for LocalMemGPT.
|
||||
|
||||
---
|
||||
|
||||
### Sprint 6 — Blob CMK + Audit Tooling + Documentation (Week 11-14)
|
||||
|
||||
**Goal:** Complete the encryption story with blob storage, monitoring, migration tooling, and docs.
|
||||
|
||||
#### 6.1 Azure Blob Storage Customer-Managed Key (CMK)
|
||||
|
||||
- [ ] **6.1.1** Create `bytelyst-blob-mek` RSA key in AKV
|
||||
- [ ] **6.1.2** Configure `bytelystblobs` storage account to use CMK from AKV
|
||||
- Azure Portal → Storage account → Encryption → Customer-managed keys
|
||||
- Select `bytelyst-blob-mek` from `kv-mywisprai`
|
||||
- [ ] **6.1.3** Verify all blob operations still work (upload, download, SAS tokens)
|
||||
- [ ] **6.1.4** Update `AZURE_RESOURCE_INVENTORY.md`
|
||||
|
||||
**Commit:** `feat(infra): enable customer-managed encryption key for Azure Blob Storage`
|
||||
|
||||
#### 6.2 Encryption Migration CLI
|
||||
|
||||
- [ ] **6.2.1** Create `scripts/encrypt-migrate.ts` — CLI tool for batch encrypting existing plaintext
|
||||
|
||||
```bash
|
||||
npx tsx scripts/encrypt-migrate.ts \
|
||||
--product lysnrai \
|
||||
--container transcripts \
|
||||
--field transcriptText \
|
||||
--partition-key /userId \
|
||||
--batch-size 100 \
|
||||
--dry-run
|
||||
```
|
||||
|
||||
- Uses `@bytelyst/field-encrypt` migration helpers
|
||||
- Progress bar, resumable (tracks last processed ID)
|
||||
- Dry run mode (report count without modifying)
|
||||
|
||||
- [ ] **6.2.2** Create per-product migration scripts
|
||||
```
|
||||
scripts/migrations/
|
||||
├── encrypt-lysnrai.sh
|
||||
├── encrypt-jarvisjr.sh
|
||||
├── encrypt-notelett.sh
|
||||
├── encrypt-mindlyst.sh
|
||||
├── encrypt-nomgap.sh
|
||||
├── encrypt-actiontrail.sh
|
||||
├── encrypt-flowmonk.sh
|
||||
└── encrypt-localmemgpt.sh
|
||||
```
|
||||
|
||||
**Commit:** `feat(devops): encryption migration CLI and per-product migration scripts`
|
||||
|
||||
#### 6.3 Monitoring & Audit
|
||||
|
||||
- [ ] **6.3.1** Add encryption metrics to telemetry
|
||||
- `field_encrypt_count` — encryptions performed
|
||||
- `field_decrypt_count` — decryptions performed
|
||||
- `dek_cache_hit_rate` — DEK cache efficiency
|
||||
- `dek_unwrap_latency_ms` — AKV round-trip time
|
||||
- `migration_progress` — % of documents encrypted per container
|
||||
|
||||
- [ ] **6.3.2** Add AKV diagnostic logging
|
||||
- Enable AKV diagnostics → Log Analytics workspace
|
||||
- Alert on: unexpected IP access, high error rate, key deletion attempts
|
||||
|
||||
- [ ] **6.3.3** Add encryption audit endpoint to platform-service
|
||||
- `GET /api/admin/encryption-status` — returns per-product encryption coverage
|
||||
|
||||
```json
|
||||
{
|
||||
"products": [
|
||||
{
|
||||
"productId": "lysnrai",
|
||||
"containers": [
|
||||
{
|
||||
"name": "transcripts",
|
||||
"field": "transcriptText",
|
||||
"total": 10000,
|
||||
"encrypted": 9500,
|
||||
"coverage": "95%"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **6.3.4** Add encryption status to admin dashboard Ops page
|
||||
- Table showing encryption coverage per product/container
|
||||
- Migration trigger button (admin-only)
|
||||
|
||||
**Commit:** `feat(monitoring): encryption telemetry, AKV diagnostics, audit endpoint`
|
||||
|
||||
#### 6.4 Key Rotation Runbooks
|
||||
|
||||
- [ ] **6.4.1** Write MEK rotation runbook
|
||||
|
||||
```
|
||||
1. Create new MEK version in AKV (az keyvault key rotate)
|
||||
2. Run DEK re-wrap script: scripts/rotate-mek.ts --product lysnrai
|
||||
3. Verify decryption still works (run tests)
|
||||
4. Update AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md
|
||||
```
|
||||
|
||||
- [ ] **6.4.2** Write DEK rotation runbook (per-user key refresh)
|
||||
- [ ] **6.4.3** Write emergency key revocation runbook
|
||||
- [ ] **6.4.4** Add rotation schedule to `AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md`
|
||||
|
||||
**Commit:** `docs(devops): encryption key rotation runbooks`
|
||||
|
||||
#### 6.5 Documentation Updates
|
||||
|
||||
- [ ] **6.5.1** Update all product `AGENTS.md` files with encryption conventions
|
||||
|
||||
```markdown
|
||||
### Encryption
|
||||
|
||||
- Sensitive fields encrypted via `@bytelyst/field-encrypt`
|
||||
- Encryption singleton in `backend/src/lib/field-encrypt.ts`
|
||||
- Tests use `memory` key provider (no env vars needed)
|
||||
- Production uses `akv` key provider with product-specific MEK
|
||||
```
|
||||
|
||||
- [ ] **6.5.2** Update `AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md` with MEK section
|
||||
- [ ] **6.5.3** Update `ENVIRONMENT_VARIABLES_AND_KEYVAULT_AUDIT.md` with new env vars
|
||||
- [ ] **6.5.4** Update `AZURE_RESOURCE_INVENTORY.md` with new AKV keys + containers
|
||||
- [ ] **6.5.5** Write `docs/BEST_PRACTICES/ENCRYPTION.md` — developer guide for adding encryption to new fields
|
||||
|
||||
**Commit:** `docs: comprehensive encryption documentation across all repos`
|
||||
|
||||
**Sprint 6 deliverable:** Complete encryption story — blob CMK, migration tooling, monitoring, key rotation runbooks, full documentation.
|
||||
|
||||
---
|
||||
|
||||
## Test Plan
|
||||
|
||||
### Package Tests
|
||||
|
||||
| Package | Tests (estimated) | Coverage |
|
||||
| ------------------------------ | :---------------: | ----------------------------------------------------------- |
|
||||
| `@bytelyst/field-encrypt` | 30 | AES-GCM, envelope, providers, cache, migration, type guard |
|
||||
| `@bytelyst/client-encrypt` | 15 | Web Crypto encrypt/decrypt, PBKDF2, roundtrip |
|
||||
| `@bytelyst/e2ee-core` | 20 | Keypair gen, recovery phrase, encrypt/decrypt, key exchange |
|
||||
| `@bytelyst/secure-storage-web` | 10 | IndexedDB CRUD, fallback, CryptoKey persistence |
|
||||
|
||||
### Integration Tests (per product)
|
||||
|
||||
Each product backend gets 3-5 new tests:
|
||||
|
||||
- Encrypt → store → read → decrypt roundtrip
|
||||
- Migration: plaintext → encrypted (idempotent)
|
||||
- Encrypted field detection (`isEncrypted()`)
|
||||
- Batch encrypt/decrypt
|
||||
- Config: memory provider works without env vars
|
||||
|
||||
**Total estimated new tests: ~120**
|
||||
|
||||
### E2E Verification
|
||||
|
||||
- [ ] Manual: Create encrypted note in NoteLett, verify ciphertext in Cosmos Data Explorer
|
||||
- [ ] Manual: Create private agent in JarvisJr, verify server cannot read transcript
|
||||
- [ ] Manual: Rotate MEK, verify all reads still work
|
||||
- [ ] Manual: Run migration script on test container, verify 100% coverage
|
||||
|
||||
---
|
||||
|
||||
## Risk Register
|
||||
|
||||
| # | Risk | Likelihood | Impact | Mitigation |
|
||||
| --- | ----------------------------------------------------- | ---------- | -------- | ---------------------------------------------------------------------------------------------- |
|
||||
| R1 | AKV RBAC migration breaks existing services | Medium | High | Test in staging first. Keep access policy backup. Immediate rollback plan. |
|
||||
| R2 | DEK cache memory pressure on high-user-count services | Low | Medium | Configurable max cache size. LRU eviction. Monitor memory usage. |
|
||||
| R3 | Encryption migration corrupts data | Low | Critical | Dry-run mode first. Backup before migration. Idempotent (skip already-encrypted). |
|
||||
| R4 | AKV outage blocks all reads/writes | Low | Critical | DEK cache (15 min) provides buffer. Env key provider as emergency fallback. |
|
||||
| R5 | Performance regression on high-throughput endpoints | Low | Medium | AES-256-GCM is hardware-accelerated. DEK cache avoids AKV round-trips. Benchmark before/after. |
|
||||
| R6 | E2EE key loss = permanent data loss | Medium | Critical | Recovery phrase (12 words). Prominent UI warnings. Confirmation flow before enabling. |
|
||||
| R7 | Cosmos RU increase from larger encrypted documents | Low | Low | ~15-20% doc size increase. Monitor RU consumption. |
|
||||
| R8 | FTS5/search breaks on encrypted fields | Medium | Medium | Maintain plaintext search index or use extraction-service for keywords. |
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
| Metric | Target | Measured By |
|
||||
| ------------------------ | -------------------------------------------------------- | --------------------------- |
|
||||
| **Encryption coverage** | 100% of P0 fields encrypted within 8 weeks | Audit endpoint |
|
||||
| **Zero plaintext leaks** | 0 unencrypted sensitive fields in Cosmos after migration | Audit scan |
|
||||
| **Performance impact** | < 5% latency increase on encrypted endpoints | Backend metrics |
|
||||
| **Test coverage** | 120+ new encryption tests | Vitest/XCTest/JUnit reports |
|
||||
| **DEK cache hit rate** | > 95% | Telemetry |
|
||||
| **Key rotation time** | < 30 min for full MEK rotation | Runbook timing |
|
||||
| **Developer experience** | < 10 LOC to encrypt a new field | Code review |
|
||||
|
||||
---
|
||||
|
||||
## Cost Estimate
|
||||
|
||||
| Item | Monthly Cost | Notes |
|
||||
| -------------------------- | :-------------: | ------------------------------------------ |
|
||||
| AKV RSA keys (10 MEKs) | $10-50 | $1/key (software) or $5/key (HSM) |
|
||||
| AKV operations (DEK wraps) | $0 | Within free tier (10K/month) |
|
||||
| Cosmos RU increase | ~$2-5 | 15-20% larger docs on encrypted containers |
|
||||
| Compute (AES-GCM) | ~$0 | Hardware-accelerated, negligible |
|
||||
| **Total** | **< $60/month** | |
|
||||
|
||||
---
|
||||
|
||||
## Decision Log
|
||||
|
||||
| Date | Decision | Rationale |
|
||||
| ---------- | ---------------------------------------- | -------------------------------------------------------------------------- |
|
||||
| 2026-03-21 | Use AES-256-GCM for all field encryption | AEAD, native on all platforms, NIST approved, matches existing MFA pattern |
|
||||
| 2026-03-21 | Envelope encryption (MEK → DEK) | Enables key rotation without re-encrypting all data; per-user isolation |
|
||||
| 2026-03-21 | Tier 3 E2EE only for JarvisJr + NoteLett | Server-side AI in other products requires plaintext access |
|
||||
| 2026-03-21 | Skip ChronoMind + PeakPulse encryption | Timer configs and GPS stats are not sensitive data |
|
||||
| 2026-03-21 | RSA-4096 for MEKs | Quantum-resistant margin; AKV native support |
|
||||
| 2026-03-21 | 15-min DEK cache TTL | Balances AKV rate limits vs key exposure window |
|
||||
| 2026-03-21 | `__encrypted` sentinel in documents | Enables gradual migration without schema changes |
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Sprint Checkpoint Verification
|
||||
|
||||
After each sprint, run:
|
||||
|
||||
```bash
|
||||
# ── Package tests ──────────────────────────────────
|
||||
cd learning_ai_common_plat && pnpm build && pnpm test
|
||||
|
||||
# ── All product backends ───────────────────────────
|
||||
cd learning_voice_ai_agent/backend && npm test # LysnrAI
|
||||
cd learning_ai_jarvis_jr/backend && npm test # JarvisJr
|
||||
cd learning_ai_notes/backend && npm test # NoteLett
|
||||
cd learning_multimodal_memory_agents/backend && npm test # MindLyst
|
||||
cd learning_ai_fastgap/backend && npm test # NomGap
|
||||
cd learning_ai_trails/backend && npm test # ActionTrail
|
||||
cd learning_ai_flowmonk/backend && npm test # FlowMonk
|
||||
cd learning_ai_local_memory_gpt/backend && npm test # LocalMemGPT
|
||||
|
||||
# ── Platform service ──────────────────────────────
|
||||
cd learning_ai_common_plat && pnpm --filter @lysnrai/platform-service test
|
||||
|
||||
# ── Type check everything ──────────────────────────
|
||||
cd learning_ai_common_plat && pnpm typecheck
|
||||
```
|
||||
Loading…
Reference in New Issue
Block a user