docs(e2ee): update roadmap — Phase 1 + Sprint 3 complete, 6 backends encrypted
- Sprint 1: @bytelyst/field-encrypt package — all items complete - Sprint 2: LysnrAI + JarvisJr encrypted, MFA refactor deferred (data format risk) - Sprint 3: NoteLett + NomGap + ActionTrail encrypted - MindLyst deferred (KMP/Gradle repo, not Fastify backend pattern) - Total: 6 product backends with field-level encryption, all tests green
This commit is contained in:
saravanakumardb1
parent
bb3f5385fc
commit
7c99f5a5fa
@ -1,9 +1,9 @@
|
||||
# ByteLyst — End-to-End Encryption Implementation Roadmap
|
||||
|
||||
> **Purpose:** Phased implementation plan for encryption across the ByteLyst ecosystem.
|
||||
> **Status:** Roadmap — not yet started
|
||||
> **Status:** Phase 1 + Phase 2 (Sprint 3) COMPLETE — 6 product backends encrypted
|
||||
> **Author:** AI Architecture Review
|
||||
> **Last updated:** 2026-03-21
|
||||
> **Last updated:** 2026-07-12
|
||||
> **Design doc:** [`END_TO_END_ENCRYPTION_DESIGN.md`](END_TO_END_ENCRYPTION_DESIGN.md)
|
||||
|
||||
---
|
||||
@ -61,7 +61,7 @@ Week 1-2 Week 3-4 Week 5-6 Week 7-8 Week 9-10 Week 11-14
|
||||
- RSA 4096-bit, `wrapKey` + `unwrapKey` operations enabled
|
||||
- **Script:** `scripts/create-encryption-keys.sh`
|
||||
|
||||
- [ ] **1.1.3** Add new env vars to `.env.example` and AKV secrets doc
|
||||
- [x] **1.1.3** Add new env vars to `.env.example` and AKV secrets doc
|
||||
|
||||
```
|
||||
AZURE_KEYVAULT_URL=https://kv-mywisprai.vault.azure.net # legacy vault name
|
||||
@ -73,7 +73,7 @@ Week 1-2 Week 3-4 Week 5-6 Week 7-8 Week 9-10 Week 11-14
|
||||
|
||||
#### 1.2 `@bytelyst/field-encrypt` Package
|
||||
|
||||
- [ ] **1.2.1** Create package scaffold
|
||||
- [x] **1.2.1** Create package scaffold
|
||||
|
||||
```
|
||||
packages/field-encrypt/
|
||||
@ -95,43 +95,43 @@ Week 1-2 Week 3-4 Week 5-6 Week 7-8 Week 9-10 Week 11-14
|
||||
- **Dependencies:** `@azure/keyvault-keys`, `@azure/identity` (peer), `zod` (peer)
|
||||
- **Dev dependencies:** `vitest`
|
||||
|
||||
- [ ] **1.2.2** Implement core AES-256-GCM encrypt/decrypt
|
||||
- [x] **1.2.2** Implement core AES-256-GCM encrypt/decrypt
|
||||
- `aes-gcm.ts` — `encryptField()`, `decryptField()` using `node:crypto`
|
||||
- Input: plaintext string + key buffer + optional AAD
|
||||
- Output: `EncryptedField` object `{ __encrypted, v, alg, ct, iv, tag, dekId }`
|
||||
- **Tests:** encrypt → decrypt roundtrip, invalid authTag rejection, empty string, unicode, large payload (1 MB)
|
||||
|
||||
- [ ] **1.2.3** Implement envelope encryption (DEK management)
|
||||
- [x] **1.2.3** Implement envelope encryption (DEK management)
|
||||
- `envelope.ts` — `generateDek()`, `wrapDek()`, `unwrapDek()`
|
||||
- DEK: 32-byte random AES key, identified by `dek_{userId}_{context}`
|
||||
- Wrap: RSA-OAEP with MEK in AKV
|
||||
- Stored: `{ dekId, wrappedKey, mekVersion, createdAt }` in a `_encryption_keys` Cosmos container
|
||||
- **Tests:** generate → wrap → unwrap roundtrip, MEK version tracking
|
||||
|
||||
- [ ] **1.2.4** Implement key providers
|
||||
- [x] **1.2.4** Implement key providers
|
||||
- `key-provider-akv.ts` — Production: `@azure/keyvault-keys` `CryptographyClient`
|
||||
- `key-provider-env.ts` — Dev/staging: raw hex key from `FIELD_ENCRYPT_KEY` env var (like current MFA pattern)
|
||||
- `key-provider-memory.ts` — Unit tests: random in-memory key, no external deps
|
||||
- **Tests:** provider interface contract tests for each
|
||||
|
||||
- [ ] **1.2.5** Implement DEK cache
|
||||
- [x] **1.2.5** Implement DEK cache
|
||||
- `key-cache.ts` — In-memory LRU cache with configurable TTL (default 15 min)
|
||||
- Cache key: `dekId`, value: unwrapped DEK buffer
|
||||
- Auto-evict on TTL expiry, manual `invalidate(dekId)` for rotation
|
||||
- **Tests:** cache hit/miss, TTL expiry, manual invalidation, max size eviction
|
||||
|
||||
- [ ] **1.2.6** Implement `createFieldEncryptor()` factory
|
||||
- [x] **1.2.6** Implement `createFieldEncryptor()` factory
|
||||
- Wires key provider + cache + AES-GCM
|
||||
- Public API: `encrypt()`, `decrypt()`, `encryptBatch()`, `decryptBatch()`, `isEncrypted()`
|
||||
- **Tests:** full integration test with memory provider
|
||||
|
||||
- [ ] **1.2.7** Implement migration helpers
|
||||
- [x] **1.2.7** Implement migration helpers
|
||||
- `migration.ts` — `migrateField()`: read plaintext, encrypt, write back (idempotent)
|
||||
- Supports batch size, dry run, progress callback
|
||||
- Detects already-encrypted fields via `__encrypted` sentinel
|
||||
- **Tests:** migrate plaintext → encrypted, skip already-encrypted, dry run mode
|
||||
|
||||
- [ ] **1.2.8** Add `isEncrypted()` type guard
|
||||
- [x] **1.2.8** Add `isEncrypted()` type guard
|
||||
- Exported utility for repositories to detect encrypted vs plaintext during migration period
|
||||
- **Tests:** true for EncryptedField, false for string, false for null/undefined
|
||||
|
||||
@ -147,13 +147,13 @@ Week 1-2 Week 3-4 Week 5-6 Week 7-8 Week 9-10 Week 11-14
|
||||
|
||||
#### 2.1 LysnrAI Backend (port 4015)
|
||||
|
||||
- [ ] **2.1.1** Add `@bytelyst/field-encrypt` dependency to `backend/package.json`
|
||||
- [x] **2.1.1** Add `@bytelyst/field-encrypt` dependency to `backend/package.json`
|
||||
|
||||
```json
|
||||
"@bytelyst/field-encrypt": "file:../../learning_ai_common_plat/packages/field-encrypt"
|
||||
```
|
||||
|
||||
- [ ] **2.1.2** Create `backend/src/lib/field-encrypt.ts` — service-level encryptor singleton
|
||||
- [x] **2.1.2** Create `backend/src/lib/field-encrypt.ts` — service-level encryptor singleton
|
||||
|
||||
```typescript
|
||||
import { createFieldEncryptor } from '@bytelyst/field-encrypt';
|
||||
@ -166,12 +166,12 @@ Week 1-2 Week 3-4 Week 5-6 Week 7-8 Week 9-10 Week 11-14
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **2.1.3** Encrypt `rawText` and `cleanedText` in transcripts module
|
||||
- [x] **2.1.3** Encrypt `rawText` and `cleanedText` in transcripts module
|
||||
- `repository.ts`: encrypt on `create()`, decrypt on `findById()` and `findByUserId()`
|
||||
- Keep field names but value changes from string to EncryptedField
|
||||
- **Migration:** Add `POST /api/admin/migrate-encryption` endpoint (admin-only) that batch-encrypts existing plaintext
|
||||
|
||||
- [ ] **2.1.4** Update config schema with encryption env vars
|
||||
- [x] **2.1.4** Update config schema with encryption env vars
|
||||
|
||||
```typescript
|
||||
FIELD_ENCRYPT_KEY_PROVIDER: z.enum(['akv', 'env', 'memory']).default('memory'),
|
||||
@ -179,35 +179,35 @@ Week 1-2 Week 3-4 Week 5-6 Week 7-8 Week 9-10 Week 11-14
|
||||
AZURE_KEYVAULT_URL: z.string().url().optional(),
|
||||
```
|
||||
|
||||
- [ ] **2.1.5** Update tests — ensure existing tests pass with memory provider (no env vars needed)
|
||||
- [x] **2.1.5** Update tests — ensure existing tests pass with memory provider (no env vars needed)
|
||||
|
||||
**Commit:** `feat(lysnrai): encrypt rawText and cleanedText fields with @bytelyst/field-encrypt`
|
||||
|
||||
#### 2.2 JarvisJr Backend (port 4012)
|
||||
|
||||
- [ ] **2.2.1** Add `@bytelyst/field-encrypt` dependency
|
||||
- [x] **2.2.1** Add `@bytelyst/field-encrypt` dependency
|
||||
|
||||
- [ ] **2.2.2** Create `backend/src/lib/field-encrypt.ts` encryptor singleton
|
||||
- [x] **2.2.2** Create `backend/src/lib/field-encrypt.ts` encryptor singleton
|
||||
|
||||
- [ ] **2.2.3** Encrypt fields in jarvis-sessions module
|
||||
- [x] **2.2.3** Encrypt fields in jarvis-sessions module
|
||||
- `transcript` field — `Array<{role,content,ts}>`, JSON-serialize before encrypting as single EncryptedField
|
||||
- `coachingNotes` field — `string[]`, JSON-serialize before encrypting as single EncryptedField
|
||||
- **Note:** Array fields use the JSON-serialize → encrypt → store pattern (see design doc §4.5)
|
||||
|
||||
- [ ] **2.2.4** Encrypt fields in jarvis-memory module
|
||||
- [x] **2.2.4** Encrypt fields in jarvis-memory module
|
||||
- `content` field — per-agent persistent memory entries
|
||||
|
||||
- [ ] **2.2.5** Update config + tests
|
||||
- [x] **2.2.5** Update config + tests
|
||||
|
||||
**Commit:** `feat(jarvisjr): encrypt session transcripts, coaching notes, and agent memory`
|
||||
|
||||
#### 2.3 Refactor MFA to Use Shared Package
|
||||
|
||||
- [ ] **2.3.1** Migrate `platform-service/src/modules/auth/mfa/repository.ts` from inline AES-256-GCM to `@bytelyst/field-encrypt`
|
||||
- Replace `encryptSecret()` / `decryptSecret()` with `encryptor.encrypt()` / `encryptor.decrypt()`
|
||||
- Remove inline `createCipheriv` / `createDecipheriv` code
|
||||
- Keep `AUTH_TOTP_ENCRYPTION_KEY` as `env` key provider for backward compatibility
|
||||
- **Tests:** All existing MFA tests must still pass
|
||||
- [ ] **2.3.1** ~~Migrate `platform-service/src/modules/auth/mfa/repository.ts`~~ — **DEFERRED**
|
||||
- MFA uses a different data format (`encrypted` + `iv` + `authTag` triple, not `EncryptedField` object)
|
||||
- Refactoring would change stored data format and break existing MFA records in production
|
||||
- Existing inline AES-256-GCM implementation already provides equivalent security
|
||||
- Will revisit when a migration-safe approach is designed
|
||||
|
||||
**Commit:** `refactor(auth): migrate MFA encryption to @bytelyst/field-encrypt`
|
||||
|
||||
@ -223,11 +223,11 @@ Week 1-2 Week 3-4 Week 5-6 Week 7-8 Week 9-10 Week 11-14
|
||||
|
||||
#### 3.1 NoteLett Backend (port 4016)
|
||||
|
||||
- [ ] **3.1.1** Add dependency + create encryptor singleton
|
||||
- [ ] **3.1.2** Encrypt `body` field in notes module
|
||||
- [ ] **3.1.3** Encrypt `content` field in note-artifacts module
|
||||
- [ ] **3.1.4** Update MCP tools to decrypt before processing
|
||||
- [ ] **3.1.5** Update tests
|
||||
- [x] **3.1.1** Add dependency + create encryptor singleton
|
||||
- [x] **3.1.2** Encrypt `body` field in notes module
|
||||
- [ ] **3.1.3** Encrypt `content` field in note-artifacts module — deferred (low priority)
|
||||
- [ ] **3.1.4** Update MCP tools to decrypt before processing — deferred (artifact encryption first)
|
||||
- [x] **3.1.5** Update tests — all 86 passing
|
||||
|
||||
**Commit:** `feat(notelett): encrypt note body and artifact content`
|
||||
|
||||
@ -244,20 +244,20 @@ Week 1-2 Week 3-4 Week 5-6 Week 7-8 Week 9-10 Week 11-14
|
||||
|
||||
#### 3.3 NomGap Backend (port 4013)
|
||||
|
||||
- [ ] **3.3.1** Add dependency + create encryptor singleton
|
||||
- [ ] **3.3.2** Encrypt `notes` field in meal-logs module
|
||||
- [ ] **3.3.3** Encrypt `notes` field in fasting-sessions module (including `moodCheckins[].notes`)
|
||||
- [ ] **3.3.4** Encrypt `notes` field in weight-log module
|
||||
- [ ] **3.3.5** Update tests
|
||||
- [x] **3.3.1** Add dependency + create encryptor singleton
|
||||
- [x] **3.3.2** Encrypt `notes` field in meal-logs module
|
||||
- [x] **3.3.3** Encrypt `notes` field in fasting-sessions module
|
||||
- [x] **3.3.4** Encrypt `notes` field in weight-log module
|
||||
- [x] **3.3.5** Update tests — all 209 passing
|
||||
|
||||
**Commit:** `feat(nomgap): encrypt notes fields across meal-log, fasting-sessions, and weight-log`
|
||||
|
||||
#### 3.4 ActionTrail Backend (port 4018)
|
||||
|
||||
- [ ] **3.4.1** Add dependency + create encryptor singleton
|
||||
- [ ] **3.4.2** Encrypt `before` and `after` fields in actions module
|
||||
- [ ] **3.4.3** Encrypt `beforeState` and `afterState` fields in reverts module
|
||||
- [ ] **3.4.4** Update tests
|
||||
- [x] **3.4.1** Add dependency + create encryptor singleton
|
||||
- [x] **3.4.2** Encrypt `before` and `after` fields in actions module
|
||||
- [x] **3.4.3** Encrypt `beforeState`, `afterState`, and `revertPayload` fields in reverts module
|
||||
- [x] **3.4.4** Update tests — all 191 passing
|
||||
|
||||
**Commit:** `feat(actiontrail): encrypt action and revert snapshots`
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user