Commit Graph

99 Commits

Author SHA1 Message Date
saravanakumardb1
d5e857dbf7 test(e2e): docker compose E2E test + seed scripts + 9-step verification
Implements the full E2E flow against the deployed docker stack and
documents it as a repeatable test playbook.

Surfaced and fixed three real issues while building the E2E:

1. JWT secret mismatch — docker-compose.override.yml backend was using
   a NoteLett-only JWT_SECRET that platform-service did not share, so
   every Authorization: Bearer call returned 'Invalid or expired token'.
   Aligned the override to use platform-service's actual secret
   (dev-ecosystem-secret-do-not-use-in-production).

2. CORS preflight missing PATCH/DELETE — @bytelyst/fastify-core registers
   @fastify/cors with only { origin }, which leaves Access-Control-Allow-
   Methods at the @fastify/cors default of 'GET,HEAD,POST'. Real browser
   PATCH/DELETE preflights would fail. Added an onSend hook in
   backend/src/server.ts that rewrites the header to
   'GET,HEAD,POST,PATCH,PUT,DELETE,OPTIONS' on CORS preflight responses.

3. Product 'notelett' wasn't registered with platform-service — auth
   register/login both error with 'Unknown or disabled product: notelett'.
   The seed script now POSTs to /api/products idempotently.

Deliverables:

- scripts/e2e-docker-seed.sh — idempotent: registers the notelett product
  and creates two test users (admin@notelett.app with role=admin who can
  write, user@notelett.app with role=user who is read-only). Re-runs are
  no-ops once seeded.

- scripts/e2e-docker-test.sh — 9-step E2E that drives the deployed stack
  via HTTP only (no browser): login → CORS preflight for PATCH →
  workspace create → note create → note read → note PATCH (status:
  draft→active) → note list → note delete → workspace delete.

- docs/testing/E2E_DOCKER_TESTING.md — full playbook covering prereqs,
  seed, automated E2E, manual UI smoke, stack architecture diagram,
  troubleshooting (JWT mismatch, unknown product, role rejection,
  CORS, port conflict, data loss), tear-down, CI wiring guidance.

- package.json — pnpm e2e:docker:seed and pnpm e2e:docker:test
  shortcuts.

Verified live on this host's deployed stack:

  $ bash scripts/e2e-docker-seed.sh
  ↷ product 'notelett' already exists
  ↷ admin user already registered + login works
  ✓ user created
  🟢 Seed complete.

  $ bash scripts/e2e-docker-test.sh
  ✓ user=usr_e094e0c2-... role=admin
  ✓ CORS allows PATCH
  ✓ workspace created
  ✓ note created
  ✓ note read matches
  ✓ note patched (status: draft → active)
  ✓ note list returned (1 item)
  ✓ note deleted (HTTP 204)
  ✓ workspace deleted (HTTP 204)
  🟢 All 9 E2E steps passed.

Backend regression suite still green: 380/380.
2026-05-23 01:16:19 -07:00
saravanakumardb1
79e936bd68 feat(ci): Cosmos emulator smoke job exercises partition-key paths
The existing 380-test backend suite runs entirely against the in-memory
datastore provider, which treats every partition-key value as equivalent.
This hid one entire class of bug — partition-key mismatches — until
production. D7 closes that gap.

Implementation:

- backend/src/test-helpers.ts adds useCosmosDatastore() that swaps the
  active provider for CosmosDatastoreProvider using COSMOS_ENDPOINT /
  COSMOS_KEY / COSMOS_DATABASE. Throws synchronously when env is missing
  so a misconfigured run fails loudly instead of silently falling back
  to in-memory.

- backend/vitest.config.ts now excludes src/**/*.cosmos.test.ts so the
  default 'pnpm test' run stays green for contributors without Docker.

- backend/vitest.cosmos.config.ts (new) includes ONLY *.cosmos.test.ts,
  bumps testTimeout to 30s / hookTimeout to 60s for the real client
  round-trips, and locks DB_PROVIDER=cosmos in test env.

- backend/src/cosmos.smoke.cosmos.test.ts (new) covers the four most
  important partition-key contracts in NoteLett:
    workspaces      /userId
    notes           /workspaceId
    note_tasks      /workspaceId
    note_shares     /workspaceId  (full create → resolve → delete → null)
  Each test also asserts that a wrong-partition-key lookup returns null,
  which is the failure mode the in-memory provider cannot simulate.

- backend/package.json adds 'test:cosmos' script.

- .github/workflows/ci.yml gains a backend-cosmos job that boots the
  official mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator
  container as a service, waits for it to be ready (60 × 5s polls of
  /_explorer/emulator.pem), then runs pnpm test:cosmos against it.
  The job depends on the existing backend job so the emulator only
  spins up after unit tests pass.

Verified locally:
- pnpm --filter @notelett/backend test: 380/380 (cosmos suite excluded)
- vitest list --config vitest.cosmos.config.ts: 4 tests under the cosmos
  smoke suite, as designed
- pnpm run verify: end-to-end green (backend 380/380, web 96/96,
  mobile 97/97)
- ci.yml passes Python yaml.safe_load

CI verification: the new job will execute on the next push. Local
verification against the emulator requires Docker on the dev host.
2026-05-23 00:25:24 -07:00
saravanakumardb1
1258d49488 feat(backend): emit task.created + workspace.created events; add share revocation regression test
Sprint B — closes audit items B6 (event-bus completeness) and B3
(public-share revocation regression).

Event bus:
- note-tasks/repository.ts createNoteTask now emits task.created with
  taskId, noteId, workspaceId, userId, title
- workspaces/repository.ts createWorkspace now emits workspace.created
  with workspaceId, userId, name

The event-bus already declared these event types (event-bus.ts) and
webhook subscribers can target them, but they were never emitted —
making the contract dead. Emissions follow the same .catch(() => {})
pattern used by note.created/updated/deleted in notes/repository.ts so
a subscriber failure cannot break the create flow.

Regression tests:
- note-tasks/repository.test.ts and workspaces/repository.test.ts
  exercise the emission paths end-to-end through the in-memory
  datastore.
- note-shares/repository.integration.test.ts adds a 5-test integration
  suite for the public-share revocation path: token resolves before
  revocation; token returns null after deleteShare (hard delete);
  expired token returns null; cross-product token rejected;
  listSharesForNote does not include revoked shares.

Verified:
- pnpm --filter @notelett/backend run test: 380/380 (was 373, +7 new)
- pnpm run verify end-to-end green
2026-05-22 23:23:08 -07:00
9bded74567 fix(lint): clear production lint blockers 2026-05-05 14:04:49 -07:00
efcc5fa044 feat(data): add seed bootstrap strategy 2026-05-05 13:45:43 -07:00
e71febe51a fix(sharing): harden note share revocation flows 2026-05-05 12:25:16 -07:00
09f30c003e feat(notes): harden import export readiness 2026-05-05 12:15:11 -07:00
3aa385774e test(backend): verify scheduler webhook lifecycle 2026-05-05 11:37:28 -07:00
205c44bc3f test(backend): document cosmos query scope 2026-05-05 11:17:27 -07:00
ff33a057ab test(backend): cover structured error mapping 2026-05-05 11:15:02 -07:00
874dda2f73 feat(backend): add dependency readiness diagnostics 2026-05-05 11:12:23 -07:00
c31f51ddbd fix(backend): propagate outbound request ids 2026-05-05 11:06:02 -07:00
7016fc1ad5 refactor(backend): route startup logs through logger 2026-05-05 11:00:38 -07:00
6b37b728aa fix(security): encrypt sensitive note metadata 2026-05-05 10:12:40 -07:00
8a53dfd489 fix(mcp): harden agent write paths 2026-05-05 10:08:58 -07:00
ee4a8ab2ea fix(security): add backend abuse rate limits 2026-05-05 10:05:33 -07:00
56a051a422 fix(security): gate production diagnostics 2026-05-05 09:51:29 -07:00
8007fac947 test(config): cover production validation 2026-05-05 09:42:20 -07:00
e7d381f071 fix(config): harden production backend config 2026-05-05 09:40:55 -07:00
saravanakumardb1
ba89cfd643 refactor(embeddings): replace local cosineSimilarity with @bytelyst/palace re-export
Remove duplicate cosine similarity implementation in favor of the
shared @bytelyst/palace primitive. All consumers import from
embeddings.ts unchanged.
2026-04-13 12:33:52 -07:00
saravanakumardb1
7d7e445135 feat(prompts): inject palace wake-up context into Smart Actions
Enrich the system prompt with L0/L1/L2 workspace context from the
palace when running Smart Actions. Best-effort — failures never block
prompt execution. Gives LLM access to decisions, preferences, and
semantically relevant memories from the workspace.
2026-04-13 12:33:34 -07:00
saravanakumardb1
06590b9175 feat(palace): extract KG triples from entities + refresh L1 cache on note save
After storing memories, generate subject-predicate-object triples from
entity pairs found in extracted memories. Also trigger L1 critical facts
cache regeneration so wake-up context stays fresh.
2026-04-13 12:33:16 -07:00
saravanakumardb1
18b680ad53 fix(backend): fix sync throw isolation in event bus + add 6 tests
Promise.allSettled only catches rejected promises, not synchronous throws.
Wrap handler calls in Promise.resolve().then() to isolate sync errors.
Add 6 unit tests covering delivery, unsubscribe, error isolation,
singleton, reset, and removeAll.
2026-04-13 11:42:52 -07:00
saravanakumardb1
61de6ce94a feat(backend): wire @bytelyst/webhook-dispatch to domain event bus
- New lib/webhook-subscriber.ts: bridges event bus to webhook dispatch.
  Registers listeners on all 5 domain events (note.created, updated,
  deleted, task.created, workspace.created). Dispatches to registered
  targets with HMAC-SHA256 signing, retry, and delivery log.
- server.ts: init webhook subscriber on startup, stop on close.
- Adds @bytelyst/webhook-dispatch dependency.
2026-04-13 10:29:43 -07:00
saravanakumardb1
190d23280f feat(backend): add domain event bus and wire note CRUD events
- New lib/event-bus.ts: lightweight typed pub/sub with error isolation
  via Promise.allSettled. Supports note.created, note.updated,
  note.deleted, task.created, workspace.created events.
- notes/repository.ts: emit events on create, update, and delete.
- .env.example: add TELEMETRY_ENABLED, FEATURE_FLAGS_ENABLED,
  FIELD_ENCRYPT_ENABLED, PALACE_ENABLED, PALACE_EXTRACTION_ENABLED.
2026-04-13 10:00:12 -07:00
saravanakumardb1
1bf9896ea9 fix(palace): pass stripped plainText to extractor instead of raw HTML 2026-04-10 01:42:00 -07:00
saravanakumardb1
1ae35a7e21 fix(palace): wire palace MCP tools into register-note-tools registry 2026-04-10 01:41:28 -07:00
saravanakumardb1
97662d1dc0 fix(palace): deleteMemory now decrements wing/room memoryCount 2026-04-10 01:40:34 -07:00
saravanakumardb1
c7c1ebad74 feat(palace): MCP memory tools — search, store, wake-up, KG query, timeline, list wings (N6) 2026-04-10 01:37:31 -07:00
saravanakumardb1
be2f4ff0ad feat(palace): REST API routes with search, CRUD, KG, wake-up, maintenance (N5) 2026-04-10 01:35:20 -07:00
saravanakumardb1
31bdb0ada9 feat(palace): knowledge graph triple CRUD with temporal queries, contradiction detection (N3) 2026-04-10 01:30:02 -07:00
saravanakumardb1
a5dbeac77e feat(palace): wake-up context builder with L0/L1/L2 layers, L1 cache regeneration (N2) 2026-04-10 01:27:13 -07:00
saravanakumardb1
0af5f875bb feat(palace): auto-save hooks — extract memories on note create/update (N4) 2026-04-10 01:25:00 -07:00
saravanakumardb1
632b5df1ac feat(palace): add 30 tests with cross-user isolation, fix extractor regex fallback 2026-04-10 01:19:35 -07:00
saravanakumardb1
13020bc72f feat(palace): add LLM memory extractor with regex fallback 2026-04-10 01:16:55 -07:00
saravanakumardb1
44d8867aa5 feat(palace): add user-isolated repository with CRUD, search, dedup, prune, encryption 2026-04-10 01:15:41 -07:00
saravanakumardb1
38006af1a3 feat(palace): add @bytelyst/palace dep, palace types, 6 cosmos containers, config vars 2026-04-10 01:14:00 -07:00
saravanakumardb1
b7bdfd97bc fix(test): resolve all TS lint errors in note-collaborators routes test — properly typed mock fns 2026-04-06 20:56:20 -07:00
saravanakumardb1
338e80fc33 fix(intake): support comma-separated status filter in GET /intake/jobs — fixes mobile polling contract mismatch 2026-04-06 20:48:26 -07:00
saravanakumardb1
599d68e116 feat(sharing): add collaborative shares, export-text, deep-link helper — note-collaborators module (11 new tests) 2026-04-06 20:31:31 -07:00
saravanakumardb1
0e16714da1 feat(intake): add URL intake pipeline — classifier, extractors, rules engine, routes, 6 new prompt templates (27 total), 26 new tests 2026-04-06 20:28:36 -07:00
saravanakumardb1
6f262a5218 fix: register missing note_shares+note_versions containers, add multi-note input type 2026-04-06 20:20:12 -07:00
saravanakumardb1
18646fac6d fix(tests): fix all failing tests — server mock, ecosystem schema stubs, CreateNoteModal body assertion 2026-04-06 19:13:35 -07:00
saravanakumardb1
96015be313 fix: add /userId partition key to scheduler schedule+webhook collections 2026-04-06 16:29:24 -07:00
saravanakumardb1
192c7baf2f fix: align web+mobile types with backend (categories, output types, field names, durationMs telemetry) 2026-04-06 16:24:31 -07:00
saravanakumardb1
6095f1d985 feat(smart-actions): add prompt-result screen, capture sub-routes, Cmd+Shift+A shortcut, telemetry events (G16-G19) 2026-04-06 13:43:47 -07:00
saravanakumardb1
4fd6994fb0 test(web): add prompt-client, SmartActionsPanel, RunPromptModal, NoteEditor tests (G10-G13) 2026-04-06 13:34:04 -07:00
saravanakumardb1
2a7cfbb73e test(backend): add runner, reading-time, copilot-transform, note-hooks tests (G6-G9) 2026-04-06 13:29:33 -07:00
saravanakumardb1
093da76eee feat(smart-actions): add run-stream SSE, history endpoint, weekly-digest template, web client functions (G1-G5) 2026-04-06 13:27:02 -07:00
saravanakumardb1
e9f389a8b7 feat(backend): add 11 telemetry events for Smart Actions 2026-04-06 11:13:50 -07:00