docs(e2ee): detailed SQLCipher + AKV implementation plan for LocalMemGPT Sprint 5.4

- Decision: SQLCipher full-DB encryption (preserves FTS5 search)
- Key hierarchy: AKV secret > env var > auto-generated file > unencrypted dev
- Existing DB migration via sqlcipher_export
- 6 implementation steps documented
This commit is contained in:
saravanakumardb1 2026-03-21 13:39:01 -07:00
parent 10b48a3800
commit b1af8e550a

View File

@ -421,15 +421,51 @@ Week 1-2 Week 3-4 Week 5-6 Week 7-8 Week 9-10 Week 11-14
#### 5.4 LocalMemGPT SQLite Encryption
- [ ] **5.4.1** Evaluate `better-sqlite3-sqlcipher` vs application-level encryption
- If SQLCipher: swap `better-sqlite3``better-sqlite3-sqlcipher`, add `PRAGMA key`
- If app-level: encrypt `content` column via `@bytelyst/field-encrypt` (env provider)
**Decision: SQLCipher (full-database encryption)**
- [ ] **5.4.2** Implement chosen approach
- [ ] **5.4.3** Handle FTS5 index (see design doc §7.2)
- [ ] **5.4.4** Migration: encrypt existing unencrypted databases on first startup
SQLCipher encrypts the entire SQLite file with AES-256-CBC, preserving FTS5 search, triggers, and all SQL features transparently. Application-level encryption was rejected because it breaks FTS5 full-text search on `messages.content`.
**Commit:** `feat(localmemgpt): encrypt SQLite message content`
**Sensitive fields protected:** `messages.content`, `conversations.system_prompt`, `document_chunks.content`, `document_chunks.embedding`
**Key management strategy:**
| Source | Priority | Description |
|--------|----------|-------------|
| Azure Key Vault secret `localmemgpt-sqlite-key` | 1st | Production — resolved via `@bytelyst/config` `resolveKeyVaultSecrets()` at startup |
| `SQLITE_ENCRYPTION_KEY` env var | 2nd | CI / staging / Docker override |
| `~/.LysnrAI/localmemgpt-sqlite.key` file | 3rd | Auto-generated on first run (64-char hex) for local dev |
| No key | 4th | Skip encryption — unencrypted dev mode for backwards compat |
> **Why AKV?** Consistent with all other ByteLyst products (MEKs stored in `kv-mywisprai`). Key never touches the repo, survives container restarts, supports rotation via AKV versioning.
**Implementation steps:**
- [ ] **5.4.1** Swap `better-sqlite3``better-sqlite3-sqlcipher` in `backend/package.json`
- Native module — requires OpenSSL + C toolchain for build
- Verify Docker build (`backend/Dockerfile`) still compiles
- Verify CI (`ci.yml`) has build tools available
- [ ] **5.4.2** Add key management to `backend/src/lib/sqlite.ts`
- Read key from `SQLITE_ENCRYPTION_KEY` env var, fall back to `~/.LysnrAI/localmemgpt-sqlite.key`
- If neither exists: auto-generate 64-char hex key, write to key file, log warning with backup reminder
- If key exists: `db.pragma(\`key = '\${key}'\`)`as first statement after`new Database(dbPath)`
- In-memory test DB (`getTestDb()`) skips encryption
- [ ] **5.4.3** Existing database migration (one-time, on startup)
- Detect if `db/memory.db` is unencrypted (try open without key, check if readable)
- If unencrypted + key available: run SQLCipher export migration:
```sql
ATTACH DATABASE 'memory-encrypted.db' AS encrypted KEY 'the-key';
SELECT sqlcipher_export('encrypted');
DETACH DATABASE encrypted;
```
- Swap `memory.db``memory-unencrypted.db.bak`, `memory-encrypted.db``memory.db`
- Log: "Database encrypted successfully. Backup at memory-unencrypted.db.bak"
- [ ] **5.4.4** Update `backend/.env.example` with `SQLITE_ENCRYPTION_KEY` documentation
- [ ] **5.4.5** Verify FTS5 search, document upload/chunking, and export/import still work
- [ ] **5.4.6** Update `docker-compose.yml` to mount key file or pass env var
**Commit:** `feat(localmemgpt): SQLCipher full-database encryption with auto-key management`
**Sprint 5 deliverable:** E2EE private vaults for JarvisJr + NoteLett. SQLite encryption for LocalMemGPT.