diff --git a/docs/WINDSURF/TELEMETRY_ROADMAP.md b/docs/WINDSURF/TELEMETRY_ROADMAP.md index 83ad4503..2f10ea2a 100644 --- a/docs/WINDSURF/TELEMETRY_ROADMAP.md +++ b/docs/WINDSURF/TELEMETRY_ROADMAP.md @@ -113,8 +113,8 @@ ### Error Clustering & Alerting -- [ ] Automated error fingerprinting (hash of `platform + channel + module + eventName + errorDomain + errorCode`) -- [ ] Cluster severity escalation (`warn` → `error` → `fatal` based on count + affected users) +- [x] Automated error fingerprinting (hash of `platform + channel + module + eventName + errorDomain + errorCode`) — Phase 1 ([`ce4c4ff`](https://github.com/saravanakumardb1/learning_ai_common_plat/commit/ce4c4ff)) +- [x] Cluster severity escalation (`warn` → `error` → `fatal` based on count + affected users) — Phase 1 ([`ce4c4ff`](https://github.com/saravanakumardb1/learning_ai_common_plat/commit/ce4c4ff)) - [ ] Slack/email alerting when cluster severity escalates - [ ] Dashboard: cluster timeline chart showing occurrence rate over time - [ ] Dashboard: "Resolve" / "Ignore" actions on clusters @@ -128,26 +128,27 @@ ### Collection Policy Builder UI -- [ ] Admin page: `/ops/telemetry-policies` -- [ ] CRUD UI for collection policies (name, enabled, targeting rules, sampling rates) -- [ ] Targeting builder: platform checkboxes, version range inputs, percentage slider +- [x] Admin page: `/ops/telemetry-policies` ([`c7732c9`](https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/c7732c9)) +- [x] CRUD UI for collection policies (name, enabled, targeting rules, sampling rates) ([`c7732c9`](https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/c7732c9)) +- [x] Targeting builder: platform checkboxes, channel badges, release channel selection, percentage slider ([`c7732c9`](https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/c7732c9)) - [ ] Live preview: "N clients would match this policy" -- [ ] Policy activation/deactivation toggle -- [ ] Scheduling: `startsAt` / `expiresAt` date pickers +- [x] Policy activation/deactivation toggle ([`c7732c9`](https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/c7732c9)) +- [x] Scheduling: `startsAt` / `expiresAt` date pickers ([`c7732c9`](https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/c7732c9)) ### Privacy & Compliance -- [ ] PII regex scanner on ingestion (email, phone, SSN patterns → redact before storage) -- [ ] Admin UI: GDPR erasure tool (search by userId → delete all events) -- [ ] Retention policy enforcement (TTL-based auto-expiry per container) +- [x] PII regex scanner on ingestion (email, phone, SSN, credit card patterns → reject before storage) — Phase 1 ([`ce4c4ff`](https://github.com/saravanakumardb1/learning_ai_common_plat/commit/ce4c4ff)) +- [x] Admin API: GDPR erasure endpoint `DELETE /telemetry/user/:userId` — Phase 1 ([`ce4c4ff`](https://github.com/saravanakumardb1/learning_ai_common_plat/commit/ce4c4ff)) +- [x] Admin UI: GDPR erasure proxy route `/api/telemetry/erasure` ([`c7732c9`](https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/c7732c9)) +- [x] Retention policy enforcement (TTL-based auto-expiry, `TELEMETRY_EVENT_TTL_DAYS` env var) — Phase 1 ([`ce4c4ff`](https://github.com/saravanakumardb1/learning_ai_common_plat/commit/ce4c4ff)) - [ ] Audit log entries for policy changes and data deletions ### Performance & Scale -- [ ] Client-side config caching (poll `/api/telemetry/config` with `If-None-Match` ETag) -- [ ] Server-side rate limiting per `installId` (100 events/min default) +- [x] ETag caching on `GET /telemetry/config` (`If-None-Match` → 304, `Cache-Control: private, max-age=60`) ([`2fb3410`](https://github.com/saravanakumardb1/learning_ai_common_plat/commit/2fb3410)) +- [x] Server-side rate limiting per `installId` (100 events/min, in-memory sliding window) ([`2fb3410`](https://github.com/saravanakumardb1/learning_ai_common_plat/commit/2fb3410)) - [ ] Cosmos DB indexing policy tuning for `telemetry_events` (composite indexes on query patterns) -- [ ] Batch ingestion deduplication by `event.id` +- [x] Batch ingestion deduplication by `event.id` ([`2fb3410`](https://github.com/saravanakumardb1/learning_ai_common_plat/commit/2fb3410)) - [ ] Prometheus metrics for ingestion throughput and error rates --- @@ -201,20 +202,20 @@ ## Test Coverage -| Component | Test File | Tests | Coverage | -| -------------------------------- | ------------------------------------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------- | -| **Platform-service schemas** | `telemetry.test.ts` | 34 | Zod schemas: event, ingest, policy CRUD, query | -| **Platform-service route logic** | `telemetry.test.ts` | 43 | `containsPII` (6), `computePk` (4), `normalizeMessage` (7), `generateFingerprint` (8), `policyMatchesContext` (13), `mergePolicies` (5) | -| **Desktop Python client** | `tests/cloud/test_platform_telemetry.py` | 19 | Event format (6), queue behavior (2), session mgmt (2), flush/HTTP (5), install ID (2), singleton (2) | -| **Web dashboard client** | `user-dashboard-web/src/__tests__/telemetry.test.ts` | 12 | `trackEvent` (3), `trackPageView` (1), `flush` (4), install ID (2), `initTelemetry` (2) | -| **Tracker dashboard client** | `tracker-dashboard-web/src/__tests__/telemetry.test.ts` | 10 | `trackEvent` (3), `trackPageView` (1), `flush` (4), `initTelemetry` (2) | -| **Admin dashboard client** | `admin-dashboard-web/src/__tests__/telemetry.test.ts` | 10 | `trackEvent` (3), `trackPageView` (1), `flush` (4), `initTelemetry` (2) | -| **Total** | | **128** | | +| Component | Test File | Tests | Coverage | +| -------------------------------- | ------------------------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Platform-service schemas** | `telemetry.test.ts` | 34 | Zod schemas: event, ingest, policy CRUD, query | +| **Platform-service route logic** | `telemetry.test.ts` | 46 | `containsPII` (6), `computePk` (4), `normalizeMessage` (7), `generateFingerprint` (8), `policyMatchesContext` (13), `mergePolicies` (5), `checkRateLimit` (3) | +| **Desktop Python client** | `tests/cloud/test_platform_telemetry.py` | 19 | Event format (6), queue behavior (2), session mgmt (2), flush/HTTP (5), install ID (2), singleton (2) | +| **Web dashboard client** | `user-dashboard-web/src/__tests__/telemetry.test.ts` | 12 | `trackEvent` (3), `trackPageView` (1), `flush` (4), install ID (2), `initTelemetry` (2) | +| **Tracker dashboard client** | `tracker-dashboard-web/src/__tests__/telemetry.test.ts` | 10 | `trackEvent` (3), `trackPageView` (1), `flush` (4), `initTelemetry` (2) | +| **Admin dashboard client** | `admin-dashboard-web/src/__tests__/telemetry.test.ts` | 10 | `trackEvent` (3), `trackPageView` (1), `flush` (4), `initTelemetry` (2) | +| **Total** | | **131** | | ### Verification commands ```bash -# Platform-service (77 telemetry tests within 611 total) +# Platform-service (80 telemetry tests within 614 total) cd ../learning_ai_common_plat && pnpm --filter @lysnrai/platform-service test # Desktop Python (19 tests) @@ -271,3 +272,5 @@ The following bugs were discovered during systematic review of the roadmap again | 2026-02-17 | voice-agent | [`08efdb6`](https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/08efdb6) | Tests: Python client (19) + web dashboard (12) telemetry tests | | 2026-02-17 | voice-agent | [`a102609`](https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/a102609) | Tracker + admin self-telemetry clients + tests (20 tests) | | 2026-02-17 | voice-agent | [`9196f48`](https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/9196f48) | Android TelemetryClient + keyboard instrumentation + ProcessLifecycleOwner | +| 2026-02-17 | voice-agent | [`c7732c9`](https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/c7732c9) | Phase 3: Policy Builder UI + GDPR erasure proxy + sidebar nav | +| 2026-02-17 | common-plat | [`2fb3410`](https://github.com/saravanakumardb1/learning_ai_common_plat/commit/2fb3410) | Phase 3: Rate limiting, batch dedup, ETag config caching (614 tests) |