From e5a481fb05e45724a53e1467e619e6ad3752b217 Mon Sep 17 00:00:00 2001 From: saravanakumardb1 Date: Sat, 14 Feb 2026 23:24:41 -0800 Subject: [PATCH] =?UTF-8?q?docs:=20holistic=20devops=20docs=20review=20?= =?UTF-8?q?=E2=80=94=20fix=20stale=20refs,=20rewrite=20KV=20rotation=20doc?= =?UTF-8?q?,=20delete=20scratchpad=20+=20outdated=20files?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../AUTH_SERVICE_DESIGN__OUTDATED_CANCELED.md | 86 ----- .../AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md | 350 +++++++++--------- docs/devops/AZURE_PORTAL_SETUP.md | 6 +- docs/devops/AZURE_RESOURCE_INVENTORY.md | 11 +- ...NVIRONMENT_VARIABLES_AND_KEYVAULT_AUDIT.md | 51 +-- docs/devops/immediate_action_feb14.md | 9 - 6 files changed, 204 insertions(+), 309 deletions(-) delete mode 100644 docs/WINDSURF/AUTH_SERVICE_DESIGN__OUTDATED_CANCELED.md delete mode 100644 docs/devops/immediate_action_feb14.md diff --git a/docs/WINDSURF/AUTH_SERVICE_DESIGN__OUTDATED_CANCELED.md b/docs/WINDSURF/AUTH_SERVICE_DESIGN__OUTDATED_CANCELED.md deleted file mode 100644 index 95d5fc23..00000000 --- a/docs/WINDSURF/AUTH_SERVICE_DESIGN__OUTDATED_CANCELED.md +++ /dev/null @@ -1,86 +0,0 @@ -# ⛔ OUTDATED & CANCELED — Auth Service Design (WindSurf) - -> **Status:** CANCELED — Auth is handled by `platform-service/src/modules/auth/` and `@bytelyst/auth` package. -> This doc predates the current architecture. See `SERVICE_CONSOLIDATION_ROADMAP.md` for the current plan. -> **Canceled:** 2026-02-14 - -## Goal (Original — No Longer Applicable) - -Create a shared authentication microservice under `learning_ai_common_plat/services/auth-service` so all BytelystAI apps (MindLyst, LysnrAI, Windhawk extensions, etc.) rely on one canonical JWT-based auth + licensing stack rather than duplicating logic in each client. This service will become the source of truth for user identity, subscriptions, JWT minting/refreshing, and Azure resource guardianship. - -## Section 1: Scope & Constraints - -- **Who calls it**: All mobile/web/desktop apps, plus internal backend services that need user context (cloud sync, tracker, platform, bios, etc.). -- **Stack**: FastAPI (TypeScript?). Mirror existing platform stack (Fastify + TypeScript) or re-use Python FastAPI depending on team familiarity; pick whichever aligns with most current services. -- **Secrets**: Must read JWT signing keys, Stripe keys, and Azure connection strings from Key Vault via shared environment (per WindSurf scripts). Avoid hard-coded secrets and integrate with `env.dev` + `keyvault` wrappers. -- **Deployment**: Host inside `rg-mywisprai`. Expose via Traefik under `/api/auth`. - -## Section 2: API surface - -### Routes - -1. `POST /api/auth/register` — accept `name`, `email`, `password`; validate against duplicate; hash password; store user doc in Cosmos `users` container; return JWT pair + user info. -2. `POST /api/auth/login` — accept `email`, `password`; verify password; return `access_token`, `refresh_token`, and `user` (including plan/roles). This token is used by all clients. -3. `POST /api/auth/refresh` — refresh `access_token` using `refresh_token`; rotate tokens. -4. `POST /api/auth/logout` (optional) — optionally blacklist refresh token (maybe via TTL cache) or rotate. -5. `GET /api/auth/me` — validate bearer token; return user + plan info (used by clients to hydrate UI, lubrication). This endpoint is currently used by CloudSync and mobile settings screens. - -### Tokens - -- Access token: short-lived (~15m) JWT (HMAC-SHA256) signed with secret from Key Vault; contains `sub`, `plan`, `roles`, `prod` flag, `resourceGroup` references for telemetry. -- Refresh token: opaque UUID stored in Cosmos table or Redis; rotate per login; share via `AuthService` for `AuthState`. Provide TTL (7d) and ability to revoke. - -### Payload Example -``` -{ - "sub": "user-id", - "email": "hello@bytelyst.ai", - "plan": "pro", - "roles": ["admin", "beta"], - "resource": "rg-mywisprai", - "iat": 1670000000, - "exp": 1670000900 -} -``` - -## Section 3: Storage & Licensing - -- **Cosmos**: `users` container with fields `id`, `email`, `passwordHash`, `plan`, `stripeCustomer`, `licenses`, `settings`, `createdAt`, `updatedAt`. -- **Azure Storage/Blob**: (if storing avatars) keep path; not needed initially. -- **Licenses**: On login/register, query `billing-service`/Stripe via HTTP (shared module) to check active subscription; store plan info in user doc. -- **Auth claims** include `licenseId` so downstream services (tracker, transcripts) can enforce usage. - -## Section 4: Shared SDK & Reusable Middleware - -- Provide a small `auth` utility package in Plat, consumed by other services: - - `verifyToken(req)` – extracts Bearer token, verifies signature, returns user claims. - - `requirePlan('pro')` – decorator to guard premium routes. - - `withAuth(api, handler)` – plug into FastAPI/Fastify. - -- Export an `authClient.postAuth(endpoint, body)` used by mobile apps; all clients share the same base URL. - -## Section 5: Integration Checklist for WindSurf Agents - -1. Create `services/auth-service` folder with package management + README (copy pattern from `billing-service`). -2. Define `.env.example` with env vars: `JWT_SECRET`, `COSMOS_ENDPOINT`, `COSMOS_KEY`, `API_BASE_URL`, `STRIPE_KEY`, `AZURE_STORAGE_SAS`, etc. -3. Implement FastAPI routes as above; reuse existing Cosmos helpers from other services. -4. Build middleware/commons for other services: place shared logic under `packages/auth` or similar; reference from `platform-service`, `billing-service`, `cloud sync`, etc. -5. Reconfigure mobile apps to call `/api/auth` on new service; update `common/env.dev` to include new `AUTH_BASE_URL` if splitting. -6. Migrate existing auth logic (AuthService/AuthViewModel) to consume new endpoints; keep current local storage strategy (AppStorage+Shared App Group) unchanged. -7. Document onboarding in `docs/WINDSURF/AUTH_SERVICE_DESIGN.md` and mention guardrails for secrets. - -## Section 6: Monitoring & Observability - -- Send login/register attempts + failures to App Insights (`bytelyst-appinsights`). -- Register alerts in `rg-mywisprai` (via Action Group) for unusual auth failure spikes. -- Log to Grafana/Loki via `loki` service (existing docker stack) to keep consistency. - -## Section 7: WindSurf Agent Tasks - -1. Scaffold the new service; mirror the structure of `billing-service` (DI, repo, routes). -2. Implement tests: login, refresh, invalid creds, missing tokens. -3. Update `shared packages` (TS) to export middleware/client. -4. Update mobile env documentation and sample dev env. -5. Validate with `pnpm lint`, `pnpm test`, `./gradlew` to keep build green. - -Keep this doc with the same tone as existing `WINDSURF` guidance so future agents can pick up the auth refactor. If you need extra detail on request/response shapes or license checks, let me know and I’ll add appendices. diff --git a/docs/devops/AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md b/docs/devops/AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md index 5c8248a5..3a472467 100644 --- a/docs/devops/AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md +++ b/docs/devops/AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md @@ -1,265 +1,245 @@ -# BytelystAI — Azure Key Vault & Secrets Rotation (MindLyst + LysnrAI) +# BytelystAI — Azure Key Vault & Secrets Rotation -> **Purpose:** Centralize **all secrets** in Azure Key Vault and establish a repeatable rotation process. -> **Scope:** Staging first (current RG: `rg-mywisprai`, vault: `kv-mywisprai`), then production. +> **Purpose:** Centralize all secrets in Azure Key Vault and establish a repeatable rotation process. +> **Vault:** `kv-mywisprai` in `rg-mywisprai` (East US) > **Last updated:** 2026-02-14 --- -## Goals +## Overview -- Use **Azure Key Vault** as the source of truth for secrets (no secrets in docs, code, or git). -- Make secret rotation **low/no-downtime** using primary/secondary keys where available. -- Have a single checklist-driven runbook for rotation and incident response. +All ByteLyst products (LysnrAI, MindLyst, legacy MyWisprAI) share a **single Key Vault**: `kv-mywisprai`. Secrets are prefixed by product. -## Non-Goals (for now) +### Goals -- Full CI/CD + automated rotation (we’ll outline it, but can implement later). +- Azure Key Vault is the **source of truth** for all secrets (no secrets in docs, code, or git). +- Secret rotation is **low/no-downtime** using primary/secondary keys where available. +- All services resolve secrets at startup with **env var fallback**. + +### Non-Goals (for now) + +- Automated rotation via Azure Event Grid (manual runbooks for now). - Client-side secret usage (mobile apps should not embed service keys). --- -## Current State (Staging) +## Current State -- Resource Group: `rg-mywisprai` -- Key Vault: `kv-mywisprai` -- MindLyst Azure secrets are centralized in Key Vault under `mindlyst-*` (see “Secret Inventory” below). -- `docs/WINDSURF/AZURE_PORTAL_SETUP.md` does not contain secret values; it references Key Vault secret names and includes scripts to pull values when needed. +| Product | Prefix | Secrets in KV | Status | +|---------|--------|---------------|--------| +| **MindLyst** | `mindlyst-*` | 12 | Fully populated | +| **MyWisprAI** (legacy) | `wispr-*` | 5 | Legacy desktop secrets | +| **LysnrAI** | `lysnr-*` | 0 | **NOT SEEDED** — code is ready, vault is empty | -> Because secrets have been committed into git at least once, treat this as a **compromise** and rotate keys ASAP. +**Total secrets:** 17 (12 MindLyst + 5 MyWisprAI + 0 LysnrAI) -### Rotation TODO (Deferred) +### Code Integration Status -As of **2026-02-14**, secrets were moved into Key Vault, but the underlying service keys have **not** been rotated yet. +All components have AKV resolution implemented: -- [ ] Rotate Cosmos DB keys (`cosmos-mywisprai`) and update `mindlyst-cosmos-key` -- [ ] Rotate Storage account keys (`bytelystblobs`) and update `mindlyst-blob-connection-string` -- [ ] Rotate Azure OpenAI keys (`mywisprai-openai-sweden`) and update `mindlyst-openai-key` -- [ ] Rotate Speech keys (`mywisprai-speech`) and update `mindlyst-speech-key` -- [ ] Rotate Notification Hub SAS keys (`lysnnai`/`mindlyst-hub`) and update `mindlyst-notification-hub-connection-string` -- [ ] If treating App Insights connection string as leaked: create a new App Insights component and update `mindlyst-appinsights-connection-string` +| Component | Integration | File | +|-----------|------------|------| +| platform-service | `resolveKeyVaultSecrets()` at startup | `services/platform-service/src/server.ts` | +| extraction-service | `resolveKeyVaultSecrets()` at startup | `services/extraction-service/src/server.ts` | +| Admin dashboard | Next.js `instrumentation.ts` hook | `admin-dashboard-web/src/instrumentation.ts` | +| User dashboard | Next.js `instrumentation.ts` hook | `user-dashboard-web/src/instrumentation.ts` | +| Tracker dashboard | Next.js `instrumentation.ts` hook | `tracker-dashboard-web/src/instrumentation.ts` | +| Python backend | `SecretResolver` + pydantic `model_validator` | `backend/src/secrets/keyvault.py` | +| Desktop app | `SecretResolver` + pydantic `model_validator` | `src/secrets/keyvault.py` | ---- +### Admin Secrets Manager -## Target State - -### 1) Secret Sources - -- **All secrets** stored in Key Vault (`kv-`). -- Apps receive secrets via: - - Azure compute platform **Key Vault references** (preferred), or - - Environment variables injected at deploy time from Key Vault (acceptable), or - - A runtime secret loader using Managed Identity (only if platform references aren’t available). - -### 2) Environment Separation - -- Separate vaults per environment: - - `kv-bytelyst-staging` - - `kv-bytelyst-prod` - -### 3) No Secrets in Repo - -- `docs/WINDSURF/AZURE_PORTAL_SETUP.md` should contain: - - Resource names, endpoints, regions, container names, partition keys - - **Key Vault secret names** (not values) - - Commands/scripts to fetch values when needed +The admin dashboard at **`/ops/secrets`** provides live CRUD access to Key Vault: +- List all secrets with metadata (status, expiry, last updated) +- Read secret values (masked, with copy-to-clipboard) +- Create new secrets +- Edit existing secret values +- Rotate secrets (auto-generates random value) +- Delete secrets (soft-delete with confirmation) --- ## Secret Inventory (Canonical) -Use these as the canonical secret names. If names already exist, keep them; otherwise create them. +### LysnrAI — `lysnr-*` (13 secrets, code in `packages/config/src/keyvault.ts`) -### MindLyst (server-side) +| KV Secret Name | Env Var | Used By | Priority | +|----------------|---------|---------|----------| +| `lysnr-cosmos-key` | `COSMOS_KEY` | All services, dashboards | Critical | +| `lysnr-cosmos-endpoint` | `COSMOS_ENDPOINT` | All services, dashboards | Critical | +| `lysnr-jwt-secret` | `JWT_SECRET` | All services, dashboards | Critical | +| `lysnr-stripe-secret-key` | `STRIPE_SECRET_KEY` | platform-service | Critical | +| `lysnr-stripe-webhook-secret` | `STRIPE_WEBHOOK_SECRET` | platform-service | Critical | +| `lysnr-billing-internal-key` | `BILLING_INTERNAL_KEY` | platform-service | High | +| `lysnr-blob-connection-string` | `AZURE_BLOB_CONNECTION_STRING` | platform-service | Critical | +| `lysnr-blob-account-key` | `AZURE_BLOB_ACCOUNT_KEY` | platform-service | Critical | +| `lysnr-gemini-api-key` | `GEMINI_API_KEY` | extraction-service | Critical | +| `lysnr-seed-secret` | `SEED_SECRET` | admin dashboard | Medium | +| `lysnr-azure-speech-key` | `AZURE_SPEECH_KEY` | desktop, backend | High | +| `lysnr-azure-openai-key` | `AZURE_OPENAI_KEY` | extraction-service | High | +| `lysnr-azure-openai-endpoint` | `AZURE_OPENAI_ENDPOINT` | extraction-service | High | -| Category | Env var(s) today | Key Vault secret name | Notes | -|---|---|---|---| -| Cosmos | `COSMOS_KEY` | `mindlyst-cosmos-key` | Rotate using primary/secondary keys | -| Cosmos | `COSMOS_ENDPOINT` | `mindlyst-cosmos-endpoint` | Not secret, but ok to store | -| Cosmos | `COSMOS_DATABASE` | `mindlyst-cosmos-database` | Not secret, but ok to store | -| OpenAI | `AZURE_OPENAI_KEY` | `mindlyst-openai-key` | Rotate using key1/key2 | -| OpenAI | `AZURE_OPENAI_ENDPOINT` | `mindlyst-openai-endpoint` | Config | -| OpenAI | `AZURE_OPENAI_DEPLOYMENT` | `mindlyst-openai-deployment` | Config | -| OpenAI | `AZURE_OPENAI_API_VERSION` | `mindlyst-openai-api-version` | Config | -| Speech | `AZURE_SPEECH_KEY` | `mindlyst-speech-key` | Rotate using key1/key2 | -| Speech | `AZURE_SPEECH_REGION` | `mindlyst-speech-region` | Config | -| Blob | `AZURE_BLOB_CONNECTION_STRING` | `mindlyst-blob-connection-string` | Prefer SAS later; keys rotate key1/key2 | -| Notifications | `ANH_CONNECTION_STRING` | `mindlyst-notification-hub-connection-string` | Rotate SAS keys on the auth rule | -| Insights | `APPLICATIONINSIGHTS_CONNECTION_STRING` | `mindlyst-appinsights-connection-string` | If leaked, easiest “rotation” is new App Insights resource | -| Stripe | `STRIPE_SECRET_KEY` | `mindlyst-stripe-secret-key` | Not Azure; still belongs in Key Vault | -| Stripe | `STRIPE_WEBHOOK_SECRET` | `mindlyst-stripe-webhook-secret` | Not Azure; still belongs in Key Vault | +### MindLyst — `mindlyst-*` (12 secrets, all populated) -### LysnrAI (server-side) +| KV Secret Name | Env Var | Notes | +|----------------|---------|-------| +| `mindlyst-cosmos-endpoint` | `COSMOS_ENDPOINT` | Config | +| `mindlyst-cosmos-key` | `COSMOS_KEY` | Rotate via key1/key2 | +| `mindlyst-cosmos-database` | `COSMOS_DATABASE` | Config (`mindlyst`) | +| `mindlyst-openai-endpoint` | `AZURE_OPENAI_ENDPOINT` | Config | +| `mindlyst-openai-key` | `AZURE_OPENAI_KEY` | Rotate via key1/key2 | +| `mindlyst-openai-deployment` | `AZURE_OPENAI_DEPLOYMENT` | Config | +| `mindlyst-openai-api-version` | `AZURE_OPENAI_API_VERSION` | Config | +| `mindlyst-speech-key` | `AZURE_SPEECH_KEY` | Rotate via key1/key2 | +| `mindlyst-speech-region` | `AZURE_SPEECH_REGION` | Config | +| `mindlyst-blob-connection-string` | `AZURE_BLOB_CONNECTION_STRING` | Rotate via key1/key2 | +| `mindlyst-notification-hub-connection-string` | `ANH_CONNECTION_STRING` | Rotate SAS keys | +| `mindlyst-appinsights-connection-string` | `APPLICATIONINSIGHTS_CONNECTION_STRING` | Treat as sensitive | -Keep existing `wispr-*` secrets as-is, but standardize any missing ones to the same pattern: +### MyWisprAI (legacy) — `wispr-*` (5 secrets) -- `wispr-azure-openai-endpoint` -- `wispr-azure-openai-key` -- `wispr-azure-openai-deployment` -- `wispr-azure-speech-key` -- `wispr-azure-speech-region` +| KV Secret Name | Env Var | +|----------------|---------| +| `wispr-azure-openai-endpoint` | `AZURE_OPENAI_ENDPOINT` | +| `wispr-azure-openai-key` | `AZURE_OPENAI_KEY` | +| `wispr-azure-openai-deployment` | `AZURE_OPENAI_DEPLOYMENT` | +| `wispr-azure-speech-key` | `AZURE_SPEECH_KEY` | +| `wispr-azure-speech-region` | `AZURE_SPEECH_REGION` | --- -## Access Model (Recommended) +## Immediate Action Items -Pick one model and stick to it per environment. +### 1. Seed LysnrAI Secrets (BLOCKING) -### Option A: Key Vault RBAC (recommended) +All 13 `lysnr-*` secrets must be added to `kv-mywisprai`: -- Enable `enableRbacAuthorization=true` on the vault -- Grant: - - Humans: `Key Vault Secrets Officer` (or least privilege) for provisioning - - Apps (Managed Identity): `Key Vault Secrets User` for runtime reads -- Advantage: consistent with Azure RBAC governance +```bash +# Option A: Use the seed script (reads from .env) +cp .env.example .env # fill in real values +./scripts/seed-keyvault.sh -### Option B: Access Policies (simple for small teams) +# Option B: Use the admin dashboard Secrets Manager UI +# Navigate to /ops/secrets → Add Secret for each -- Keep access policies and grant explicit secret permissions to: - - The deployment user/service principal - - The app’s managed identity -- Advantage: very explicit, fewer moving parts +# Option C: Use az cli directly +az keyvault secret set --vault-name kv-mywisprai --name lysnr-cosmos-key --value "" -o none +``` ---- +### 2. Rotate Leaked Keys (DEFERRED) -## Migration Plan (Staging First) +Secrets have appeared in git history. Rotate after seeding: -### Phase 0: Stop The Bleed (Immediate) +- [ ] Cosmos DB keys (`cosmos-mywisprai`) → update `mindlyst-cosmos-key` + `lysnr-cosmos-key` +- [ ] Storage account keys (`bytelystblobs`) → update blob connection strings +- [ ] Azure OpenAI keys (`mywisprai-openai-sweden`) → update OpenAI key secrets +- [ ] Speech keys (`mywisprai-speech`) → update speech key secrets +- [ ] Notification Hub SAS keys → update `mindlyst-notification-hub-connection-string` -- [ ] Rotate all secrets that have appeared in git history (see “Rotation Runbooks”). -- [ ] Reduce Key Vault blast radius: - - [ ] Enable Key Vault diagnostics to Log Analytics - - [ ] Review Key Vault access policies / RBAC assignments -- [x] Add missing MindLyst secrets to Key Vault: - - [x] `mindlyst-openai-endpoint` - - [x] `mindlyst-openai-deployment` - - [x] `mindlyst-openai-api-version` - - [x] `mindlyst-speech-region` - - [x] `mindlyst-notification-hub-connection-string` - - [x] `mindlyst-appinsights-connection-string` -- [x] Docs hygiene: - - [x] Remove plaintext secrets from `docs/WINDSURF/AZURE_PORTAL_SETUP.md` (leave names + scripts) - - [x] Add a short “fetch from Key Vault” snippet per secret category +### 3. Enable KV Diagnostics -### Phase 1: App Integration (Hosted Environments) - -- [ ] Decide hosting target(s): - - [ ] MindLyst web: App Service / Container Apps / other - - [ ] LysnrAI backend: where it runs -- [ ] For each app, configure a Managed Identity and grant Key Vault read access: - - [ ] MindLyst web runtime identity can read `mindlyst-*` secrets - - [ ] LysnrAI runtime identity can read `wispr-*` secrets -- [ ] Inject secrets into the app runtime: - - [ ] Prefer platform Key Vault references (App Service Key Vault reference, Container Apps secrets from Key Vault) - - [ ] Fall back to deployment-time env var injection from Key Vault -- [ ] Validate with an end-to-end smoke test: - - [ ] `/api/triage` (Azure OpenAI) - - [ ] `/api/brain-chat` (Azure OpenAI) - - [ ] `/api/memory` + `/api/brains` (Cosmos) - -### Phase 2: Operationalize (Rotation + Audits) - -- [ ] Define rotation cadence: - - [ ] Staging: monthly (or after any leak) - - [ ] Prod: quarterly (or after any leak) -- [ ] Add a “rotation log”: - - [ ] Create `docs/WINDSURF/SECRETS_ROTATION_LOG.md` (optional) and record dates/owners -- [ ] Add automated checks: - - [ ] Secret scanning in CI (git-secrets / gitleaks) - - [x] Block commits containing patterns like `AccountKey=` / `SharedAccessKey=` (Husky: `scripts/secret-scan-staged.sh`) +- [ ] Enable Key Vault diagnostics to Log Analytics +- [ ] Review Key Vault access policies / RBAC assignments --- ## Rotation Runbooks -General rule: rotate by switching to the **secondary** key first (if supported), regenerate the primary, then switch back. +**General rule:** Rotate by switching to the **secondary** key first (if supported), regenerate the primary, then switch back. -### Cosmos DB Key Rotation (Serverless) +### Cosmos DB Key Rotation Supports primary/secondary keys. -- [ ] Determine current key used by apps (usually “primary”). -- [ ] Update Key Vault `mindlyst-cosmos-key` to the **secondary** Cosmos key. -- [ ] Redeploy / restart apps that use Cosmos. -- [ ] Verify Cosmos reads/writes: - - [ ] `GET /api/memory` - - [ ] `POST /api/memory` -- [ ] Regenerate the Cosmos **primary** key in Azure. -- [ ] Update Key Vault `mindlyst-cosmos-key` back to the new primary (or keep secondary as active if you prefer). -- [ ] Redeploy / restart again. +1. Update KV secrets to the **secondary** Cosmos key +2. Restart all services that use Cosmos +3. Verify reads/writes: `GET /health` on platform-service + extraction-service +4. Regenerate the Cosmos **primary** key in Azure Portal +5. (Optional) Switch KV back to new primary + restart again -Rollback: switch Key Vault secret back to the other key and restart. +**Secrets to update:** `lysnr-cosmos-key`, `mindlyst-cosmos-key` +**Rollback:** Switch KV secret back to the other key and restart. ### Storage Account Key Rotation Supports key1/key2. -- [ ] Update Key Vault `mindlyst-blob-connection-string` to use the **other** storage key. -- [ ] Redeploy / restart apps that use Blob. -- [ ] Regenerate the original key in Azure. -- [ ] Update Key Vault secret again (optional switch-back). +1. Update KV blob connection strings to use the **other** storage key +2. Restart platform-service +3. Regenerate the original key in Azure Portal + +**Secrets to update:** `lysnr-blob-connection-string`, `lysnr-blob-account-key`, `mindlyst-blob-connection-string` ### Azure OpenAI Key Rotation Supports key1/key2. -- [ ] Update Key Vault `mindlyst-openai-key` to use the **other** key. -- [ ] Verify `/api/triage` and `/api/brain-chat`. -- [ ] Regenerate the old key in Azure. +1. Update KV OpenAI key secrets to use the **other** key +2. Verify extraction endpoints work +3. Regenerate the old key in Azure Portal + +**Secrets to update:** `lysnr-azure-openai-key`, `mindlyst-openai-key` ### Speech Key Rotation Supports key1/key2. -- [ ] Update Key Vault `mindlyst-speech-key` to use the **other** key. -- [ ] Verify STT/TTS pipeline (when implemented). -- [ ] Regenerate the old key in Azure. +1. Update KV speech key secrets to the **other** key +2. Verify STT/TTS pipeline +3. Regenerate the old key -### Notification Hub SAS Key Rotation - -Rotate the SAS keys on the authorization rule used by `ANH_CONNECTION_STRING`. - -- [ ] Switch to the **secondary** connection string for the auth rule (update Key Vault `mindlyst-notification-hub-connection-string`). -- [ ] Verify push send (server-side) and device receive (once implemented). -- [ ] Regenerate the primary SAS key on the auth rule. -- [ ] Switch back if desired. - -### Application Insights “Rotation” - -App Insights doesn’t have an easy “rotate key” flow that’s comparable to key1/key2 (treat the connection string as sensitive anyway). - -Preferred approach if leaked: - -- [ ] Create a new App Insights resource (workspace-based recommended). -- [ ] Update Key Vault `mindlyst-appinsights-connection-string`. -- [ ] Redeploy apps. -- [ ] Keep old App Insights temporarily for historical traces, then delete if desired. +**Secrets to update:** `lysnr-azure-speech-key`, `mindlyst-speech-key` ### Stripe Keys Rotation (External) -- [ ] Rotate keys in Stripe dashboard. -- [ ] Update Key Vault `mindlyst-stripe-secret-key` / `mindlyst-stripe-webhook-secret`. -- [ ] Redeploy apps. +1. Rotate keys in Stripe dashboard +2. Update KV secrets +3. Restart platform-service + +**Secrets to update:** `lysnr-stripe-secret-key`, `lysnr-stripe-webhook-secret` + +### Recommended Rotation Schedule + +| Secret Type | Frequency | Method | +|-------------|-----------|--------| +| Cosmos DB keys | Quarterly | Azure Portal → rotate → update KV | +| JWT secret | Quarterly | Generate new → update KV → rolling deploy | +| Stripe keys | Annually or on breach | Stripe Dashboard → update KV | +| API keys (Gemini, OpenAI) | Annually or on breach | Provider portal → update KV | +| Blob storage keys | Quarterly | Azure Portal → rotate → update KV | --- -## Implementation Tasks (Checkbox List) +## Access Model -### Key Vault Completion (Staging) +### Current: Access Policies (simple) -- [ ] Add missing `mindlyst-*` secrets to `kv-mywisprai` (see inventory) -- [ ] Add missing `wispr-*` secrets to `kv-mywisprai` (if needed) -- [ ] Ensure the `mindlyst-*` secret names match what Bicep creates (`infra/azure/bytelyst-shared/`) +- Access policies grant explicit secret permissions to the deployer user +- Apps use `DefaultAzureCredential` (managed identity in prod, `az cli` locally) -### App Config Changes +### Future: Key Vault RBAC (recommended for production) -- [ ] Add a “Key Vault-first” config strategy for hosted environments: - - [ ] App Service Key Vault references (if using App Service) - - [ ] Container Apps secrets from Key Vault (if using Container Apps) -- [ ] Ensure no app requires a secret at build-time (only at runtime) +- Enable `enableRbacAuthorization=true` on the vault +- Grant humans `Key Vault Secrets Officer` +- Grant app managed identities `Key Vault Secrets User` (read-only) -### Repo Hygiene +--- -- [ ] After rotation, remove plaintext secret values from `docs/WINDSURF/AZURE_PORTAL_SETUP.md` -- [ ] Add CI secret scanning -- [ ] (Optional) Rewrite git history to remove old secrets (only if you’re prepared for the repo impact) +## Related Files + +| File | Purpose | +|------|---------| +| `packages/config/src/keyvault.ts` | `resolveKeyVaultSecrets()` + `LYSNR_SECRETS` constant | +| `scripts/seed-keyvault.sh` | Seed KV from `.env` values | +| `docs/devops/AZURE_RESOURCE_INVENTORY.md` | Complete Azure resource inventory | +| `docs/devops/AZURE_PORTAL_SETUP.md` | Step-by-step provisioning guide | +| `docs/devops/ENVIRONMENT_VARIABLES_AND_KEYVAULT_AUDIT.md` | Full env var + KV gap analysis | + +--- + +## Security Guardrails + +- [x] Husky pre-commit: `scripts/secret-scan-staged.sh` blocks `AccountKey=` / `SharedAccessKey=` +- [x] Husky pre-push: `scripts/secret-scan-repo.sh` scans tracked files +- [ ] CI secret scanning (gitleaks) — not yet configured +- [ ] (Optional) Rewrite git history to remove old secrets diff --git a/docs/devops/AZURE_PORTAL_SETUP.md b/docs/devops/AZURE_PORTAL_SETUP.md index 1b3d457e..9a18be58 100644 --- a/docs/devops/AZURE_PORTAL_SETUP.md +++ b/docs/devops/AZURE_PORTAL_SETUP.md @@ -6,7 +6,7 @@ > **Prerequisites:** An Azure account with an active subscription > **Last updated:** 2026-02-14 > -> **Security note (staging):** This document does **not** include secret values. Store secrets in Azure Key Vault (`kv-mywisprai`) and reference them by name (see **MindLyst Environment Variables** and **Key Vault** sections). Rotation is deferred; see `docs/WINDSURF/AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md`. +> **Security note (staging):** This document does **not** include secret values. Store secrets in Azure Key Vault (`kv-mywisprai`) and reference them by name (see **MindLyst Environment Variables** and **Key Vault** sections). Rotation is deferred; see `docs/devops/AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md`. --- @@ -476,7 +476,7 @@ After completing all steps, create `mindlyst-native/web/.env.local`: > **Note:** MindLyst web API routes use `x-user-id` (or `MINDLYST_USER_ID`) as the Cosmos DB partition key for containers with `/userId`. > -> **Hosted envs:** Prefer Key Vault references / Managed Identity (see `docs/WINDSURF/AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md`). For local dev, generate `.env.local` from Key Vault. +> **Hosted envs:** Prefer Key Vault references / Managed Identity (see `docs/devops/AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md`). For local dev, generate `.env.local` from Key Vault. ```bash KV_NAME="kv-mywisprai" @@ -914,4 +914,4 @@ Phase 0.3 (Azure Infrastructure): --- -> **This document is shared between both repos.** When updating Azure infrastructure, update this doc in `learning_multimodal_memory_agents` and reference it from `learning_voice_ai_agent`. +> **This document lives in `learning_ai_common_plat/docs/devops/`.** It is the single source of truth for Azure infrastructure setup shared by all products. diff --git a/docs/devops/AZURE_RESOURCE_INVENTORY.md b/docs/devops/AZURE_RESOURCE_INVENTORY.md index 2c1f26dd..b8155e27 100644 --- a/docs/devops/AZURE_RESOURCE_INVENTORY.md +++ b/docs/devops/AZURE_RESOURCE_INVENTORY.md @@ -1,6 +1,6 @@ # Azure Resource Inventory -> **Last Updated:** 2026-02-15 +> **Last Updated:** 2026-02-14 > **Purpose:** Complete inventory of Azure resources for ByteLyst AI products --- @@ -97,7 +97,7 @@ AZURE_BLOB_CONNECTION_STRING= **🔑 Key Environment Variables:** ```bash -AZURE_KEY_VAULT_URI=https://kv-mywisprai.vault.azure.net/ +AZURE_KEYVAULT_URL=https://kv-mywisprai.vault.azure.net/ ``` **📝 Usage in Code:** @@ -230,7 +230,7 @@ COSMOS_KEY= COSMOS_DATABASE=lysnrai # or mindlyst/mywisprai # Key Vault -AZURE_KEY_VAULT_URI=https://kv-mywisprai.vault.azure.net/ +AZURE_KEYVAULT_URL=https://kv-mywisprai.vault.azure.net/ # Blob Storage AZURE_BLOB_ACCOUNT_NAME=bytelystblobs @@ -295,10 +295,11 @@ APPINSIGHTS_INSTRUMENTATIONKEY= |----------|----------| | Azure Portal Setup | `docs/devops/AZURE_PORTAL_SETUP.md` | | Key Vault & Secrets Rotation | `docs/devops/AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md` | -| Cosmos DB Usage | `packages/cosmos/README.md` | -| Blob Storage Usage | `packages/blob/README.md` | +| Environment & KV Audit | `docs/devops/ENVIRONMENT_VARIABLES_AND_KEYVAULT_AUDIT.md` | +| Cosmos DB Package | `packages/cosmos/` | | Environment Variables Template | `.env.example` | | Key Vault Integration | `packages/config/src/keyvault.ts` | +| Admin Secrets Manager | Admin dashboard → `/ops/secrets` (live CRUD for KV) | --- diff --git a/docs/devops/ENVIRONMENT_VARIABLES_AND_KEYVAULT_AUDIT.md b/docs/devops/ENVIRONMENT_VARIABLES_AND_KEYVAULT_AUDIT.md index 440e195c..09391f6c 100644 --- a/docs/devops/ENVIRONMENT_VARIABLES_AND_KEYVAULT_AUDIT.md +++ b/docs/devops/ENVIRONMENT_VARIABLES_AND_KEYVAULT_AUDIT.md @@ -1,6 +1,6 @@ # Environment Variables & Azure Key Vault Audit -> **Last Updated:** 2026-02-15 +> **Last Updated:** 2026-02-14 > **Purpose:** Complete audit of environment variables, Azure Key Vault secrets, and gap analysis --- @@ -10,7 +10,7 @@ ### Critical Findings: 1. ❌ **ZERO LysnrAI secrets** exist in Azure Key Vault despite code expecting them 2. ✅ **MindLyst secrets** are fully populated (12 secrets) -3. ✅ **MyWisprAI secrets** are partially populated (7 secrets) +3. ✅ **MyWisprAI secrets** are partially populated (5 secrets) 4. ⚠️ **Mismatch** between code expectations and actual Key Vault state 5. ⚠️ **Missing Stripe secrets** for billing functionality 6. ⚠️ **Missing Gemini API key** for extraction service @@ -292,18 +292,14 @@ These secrets are **required** for LysnrAI services to function and are **comple - Notification Hub (connection string) - Application Insights (connection string) -### MyWisprAI Product (7 secrets) ⚠️ +### MyWisprAI Product (5 secrets) ⚠️ **Prefix:** `wispr-*` -**Status:** Partially populated +**Status:** Partially populated (legacy — pre-rebrand desktop app secrets) **Secrets:** - Azure OpenAI (endpoint, key, deployment) - Azure Speech (key, region) -**Missing:** -- Cosmos DB secrets -- Blob Storage secrets -- Stripe secrets -- JWT secret +**Note:** These are legacy secrets from the original MyWisprAI product. LysnrAI services use `lysnr-*` prefixed secrets instead. ### LysnrAI Product (0 secrets) ❌ **Prefix:** `lysnr-*` @@ -342,13 +338,20 @@ Add to each service's startup (already implemented in code): ```typescript import { resolveKeyVaultSecrets, LYSNR_SECRETS } from '@bytelyst/config'; -// Before loading config -await resolveKeyVaultSecrets(Object.values(LYSNR_SECRETS)); +// Before loading config — pick the subset your service needs +await resolveKeyVaultSecrets([ + LYSNR_SECRETS.COSMOS_KEY, + LYSNR_SECRETS.COSMOS_ENDPOINT, + LYSNR_SECRETS.JWT_SECRET, + // ... add service-specific secrets +]); -// Then load config (will use KV values) +// Then load config (will use resolved KV values from process.env) const config = loadConfig(); ``` +**Admin Dashboard:** The Secrets Manager UI at `/ops/secrets` provides live CRUD access to all Key Vault secrets (list, read, set, rotate, delete). + ### Step 3: Test Key Vault Integration ```bash @@ -410,13 +413,18 @@ pnpm --filter @lysnrai/platform-service dev | File | Purpose | |------|---------| -| `packages/config/src/keyvault.ts` | Key Vault integration code | -| `scripts/seed-keyvault.sh` | Script to populate Key Vault | -| `.env.example` | Template for required variables | -| `services/platform-service/src/lib/config.ts` | Platform service config schema | -| `services/extraction-service/src/lib/config.ts` | Extraction service config schema | +| `packages/config/src/keyvault.ts` | Key Vault integration code (`resolveKeyVaultSecrets` + `LYSNR_SECRETS`) | +| `scripts/seed-keyvault.sh` | Script to populate Key Vault from `.env` values | +| `.env.example` | Template for required environment variables | +| `services/platform-service/src/server.ts` | Platform service — resolves KV secrets at startup | +| `services/extraction-service/src/server.ts` | Extraction service — resolves KV secrets at startup | +| `admin-dashboard-web/src/instrumentation.ts` | Admin dashboard — resolves KV secrets via Next.js hook | +| `user-dashboard-web/src/instrumentation.ts` | User dashboard — resolves KV secrets via Next.js hook | +| `tracker-dashboard-web/src/instrumentation.ts` | Tracker dashboard — resolves KV secrets via Next.js hook | +| `backend/src/secrets/keyvault.py` | Python backend SecretResolver | +| `src/secrets/keyvault.py` | Desktop app SecretResolver | | `docs/devops/AZURE_PORTAL_SETUP.md` | Azure portal configuration guide | -| `docs/devops/AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md` | Key Vault setup guide | +| `docs/devops/AZURE_KEY_VAULT_AND_SECRETS_ROTATION.md` | Key Vault setup + rotation runbooks | | `docs/devops/AZURE_RESOURCE_INVENTORY.md` | Complete Azure resource inventory | --- @@ -431,8 +439,8 @@ pnpm --filter @lysnrai/platform-service dev ### Key Vault Secrets: - **Total Secrets in KV:** 17 - **MindLyst Secrets:** 12 ✅ -- **MyWisprAI Secrets:** 7 ⚠️ -- **LysnrAI Secrets:** 0 ❌ +- **MyWisprAI Secrets:** 5 ⚠️ (legacy `wispr-*` prefix) +- **LysnrAI Secrets:** 0 ❌ (`lysnr-*` prefix) - **Expected LysnrAI Secrets:** 13 - **Coverage Gap:** 100% @@ -482,4 +490,5 @@ node -e "process.env.AZURE_KEYVAULT_URL && console.log('✅ KV enabled') || cons **Generated by:** Environment audit automation **Maintained by:** ByteLyst DevOps Team -**Next Audit:** After LysnrAI secrets are seeded +**Next Audit:** After LysnrAI `lysnr-*` secrets are seeded into `kv-mywisprai` +**Admin UI:** Secrets Manager at admin dashboard → `/ops/secrets` diff --git a/docs/devops/immediate_action_feb14.md b/docs/devops/immediate_action_feb14.md deleted file mode 100644 index 6941fc73..00000000 --- a/docs/devops/immediate_action_feb14.md +++ /dev/null @@ -1,9 +0,0 @@ -# 1. Get actual keys from Azure -az cosmosdb keys list --name cosmos-mywisprai --resource-group rg-mywisprai - -# 2. Populate .env file with real values -# 3. Run the seed script -./scripts/seed-keyvault.sh - -# 4. Verify secrets were created -az keyvault secret list --vault-name kv-mywisprai --query "[?contains(name, 'lysnr')]" --output table