Go to file
saravanakumardb1 d5e857dbf7 test(e2e): docker compose E2E test + seed scripts + 9-step verification
Implements the full E2E flow against the deployed docker stack and
documents it as a repeatable test playbook.

Surfaced and fixed three real issues while building the E2E:

1. JWT secret mismatch — docker-compose.override.yml backend was using
   a NoteLett-only JWT_SECRET that platform-service did not share, so
   every Authorization: Bearer call returned 'Invalid or expired token'.
   Aligned the override to use platform-service's actual secret
   (dev-ecosystem-secret-do-not-use-in-production).

2. CORS preflight missing PATCH/DELETE — @bytelyst/fastify-core registers
   @fastify/cors with only { origin }, which leaves Access-Control-Allow-
   Methods at the @fastify/cors default of 'GET,HEAD,POST'. Real browser
   PATCH/DELETE preflights would fail. Added an onSend hook in
   backend/src/server.ts that rewrites the header to
   'GET,HEAD,POST,PATCH,PUT,DELETE,OPTIONS' on CORS preflight responses.

3. Product 'notelett' wasn't registered with platform-service — auth
   register/login both error with 'Unknown or disabled product: notelett'.
   The seed script now POSTs to /api/products idempotently.

Deliverables:

- scripts/e2e-docker-seed.sh — idempotent: registers the notelett product
  and creates two test users (admin@notelett.app with role=admin who can
  write, user@notelett.app with role=user who is read-only). Re-runs are
  no-ops once seeded.

- scripts/e2e-docker-test.sh — 9-step E2E that drives the deployed stack
  via HTTP only (no browser): login → CORS preflight for PATCH →
  workspace create → note create → note read → note PATCH (status:
  draft→active) → note list → note delete → workspace delete.

- docs/testing/E2E_DOCKER_TESTING.md — full playbook covering prereqs,
  seed, automated E2E, manual UI smoke, stack architecture diagram,
  troubleshooting (JWT mismatch, unknown product, role rejection,
  CORS, port conflict, data loss), tear-down, CI wiring guidance.

- package.json — pnpm e2e:docker:seed and pnpm e2e:docker:test
  shortcuts.

Verified live on this host's deployed stack:

  $ bash scripts/e2e-docker-seed.sh
  ↷ product 'notelett' already exists
  ↷ admin user already registered + login works
  ✓ user created
  🟢 Seed complete.

  $ bash scripts/e2e-docker-test.sh
  ✓ user=usr_e094e0c2-... role=admin
  ✓ CORS allows PATCH
  ✓ workspace created
  ✓ note created
  ✓ note read matches
  ✓ note patched (status: draft → active)
  ✓ note list returned (1 item)
  ✓ note deleted (HTTP 204)
  ✓ workspace deleted (HTTP 204)
  🟢 All 9 E2E steps passed.

Backend regression suite still green: 380/380.
2026-05-23 01:16:19 -07:00
.gitea/workflows ci: fix YAML formatting — normalize blank lines 2026-03-29 11:04:16 -07:00
.github feat(ci): Cosmos emulator smoke job exercises partition-key paths 2026-05-23 00:25:24 -07:00
.husky chore: add Husky pre-commit hooks + secret-scan scripts 2026-03-27 23:07:30 -07:00
backend test(e2e): docker compose E2E test + seed scripts + 9-step verification 2026-05-23 01:16:19 -07:00
docs test(e2e): docker compose E2E test + seed scripts + 9-step verification 2026-05-23 01:16:19 -07:00
mobile test(mobile): verify platform lifecycle clients 2026-05-05 13:02:14 -07:00
scripts test(e2e): docker compose E2E test + seed scripts + 9-step verification 2026-05-23 01:16:19 -07:00
shared fix: normalize product.json + replace --ml-* with --nl-* CSS namespace 2026-03-21 20:20:40 -07:00
web test(e2e): fix 4 pre-existing E2E failures and make port-conflict-proof 2026-05-23 00:50:29 -07:00
.aider.conf.yml chore(docs): regenerate AI agent config files 2026-03-10 22:56:39 -07:00
.clinerules chore(docs): regenerate AI agent config files 2026-03-10 22:56:39 -07:00
.cursorrules chore(docs): regenerate AI agent config files 2026-03-10 22:56:39 -07:00
.dockerignore feat(repo): migrate notelett workspace to pnpm 2026-03-22 15:50:54 -07:00
.editorconfig chore(docs): regenerate AI agent config files 2026-03-10 22:56:39 -07:00
.gitignore chore: gitignore .docker-deps/ directory 2026-04-13 14:36:43 -07:00
.npmrc feat(ux): add UX testing setup guide and common platform integration 2026-05-09 22:09:43 +00:00
.npmrc.docker fix: Update docker configuration for production deployment 2026-05-12 08:20:12 +00:00
.nvmrc chore: add .nvmrc pinning Node 22 2026-03-29 10:48:30 -07:00
.pnpmfile.cjs fix(workspace): canonicalize common-plat path to ../learning_ai_common_plat 2026-05-22 15:08:30 -07:00
.windsurfrules chore(docs): regenerate AI agent config files 2026-03-10 22:56:39 -07:00
AGENTS.md docs(cleanup): move historical roadmaps to docs/archive/ and update AGENTS.md 2026-05-22 23:23:50 -07:00
CLAUDE.md feat(repo): migrate notelett workspace to pnpm 2026-03-22 15:50:54 -07:00
docker-compose.override.yml test(e2e): docker compose E2E test + seed scripts + 9-step verification 2026-05-23 01:16:19 -07:00
docker-compose.yml fix: Update docker configuration for production deployment 2026-05-12 08:20:12 +00:00
package.json test(e2e): docker compose E2E test + seed scripts + 9-step verification 2026-05-23 01:16:19 -07:00
pnpm-lock.yaml test(e2e): fix 4 pre-existing E2E failures and make port-conflict-proof 2026-05-23 00:50:29 -07:00
pnpm-workspace.yaml fix(workspace): canonicalize common-plat path to ../learning_ai_common_plat 2026-05-22 15:08:30 -07:00
README.md docs(sprint-a): record build restoration and refreshed sprint plan 2026-05-22 15:08:42 -07:00

NoteLett

Structured notes platform for humans and AI agents — part of the ByteLyst ecosystem.

Quick Start

Prerequisites:

  • pnpm 10.6.5
  • sibling common-platform checkout at ../learning_ai_common_plat (override with BYTELYST_COMMON_PLAT_ROOT if your layout differs)
  • optional local ByteLyst services from common platform:
    • platform-service on port 4003 for auth, flags, telemetry, diagnostics, billing, and blob
    • extraction-service on port 4005 for URL/task extraction
    • mcp-server on port 4007 for agent tooling
# Install against the local common-platform workspace packages.
pnpm run install:common-plat --frozen-lockfile

# Registry fallback when explicitly needed.
source ~/.zshrc
pnpm run install:gitea --frozen-lockfile

# Backend in local memory mode (port 4016)
DB_PROVIDER=memory JWT_SECRET=dev-secret-do-not-use-in-prod \
pnpm --filter @notelett/backend run dev

# Web (port 3000)
pnpm --filter @notelett/web run dev

# Mobile
pnpm --filter @notelett/mobile run start

Auth is expected to come from platform-service. Local development can use the repo's dev/test helpers and memory datastore, but production must use real platform auth, Cosmos, and encryption configuration.

Docker path:

docker compose build
docker compose up -d
curl -sf http://localhost:4016/health
curl -sf http://localhost:4016/api/bootstrap

Local production-readiness smoke:

pnpm run smoke:local

# Use an already-running backend or skip shared service checks when isolating product behavior.
SMOKE_START_BACKEND=0 pnpm run smoke:local -- --no-start
pnpm run smoke:local -- --skip-platform

# Docker compose build/start/health smoke
pnpm run smoke:compose

Architecture

Surface Stack Port
Backend Fastify 5 + TypeScript ESM 4016
Web Next.js 16 + React 19 3000
Mobile Expo + React Native 8081
Platform platform-service (shared) 4003
Extraction extraction-service (shared) 4005
MCP mcp-server (shared) 4007

Architecture Boundaries

NoteLett keeps product-local domain behavior in this repo: note/workspace CRUD, relationships, tasks, artifacts, agent action audit trails, saved views, prompt templates/runners, intake rules/jobs, note sharing/collaboration, version history, Palace memory/KG UX, and all NoteLett-specific web/mobile workflows.

Common-platform responsibilities come from ../learning_ai/learning_ai_common_plat: auth/session primitives, API clients, datastore/Cosmos abstractions, shared error/config/logging conventions, telemetry, diagnostics, feature flags, kill switch, blob access, extraction-service clients, design tokens, shared UI primitives where appropriate, MCP server integration, webhook dispatch, encryption helpers, and cross-repo automation scripts.

Package resolution defaults to local common-platform packages through .pnpmfile.cjs (BYTELYST_PACKAGE_SOURCE=common-plat). Use pnpm run install:common-plat for local development and pnpm run install:gitea only when you intentionally want registry versions. Override the local checkout with BYTELYST_COMMON_PLAT_ROOT=/path/to/learning_ai_common_plat.

Do not move NoteLett-specific notes logic into common platform unless another product has a concrete reuse need. Do not create repo-local substitutes for platform concerns already covered by @bytelyst/* packages or platform services.

Key Features

  • Backend modules: notes, workspaces, relationships, tasks, artifacts, agent actions, saved views, Smart Actions, intake, collaborators, shares, versions, Palace, and ecosystem import routes
  • 8 MCP tools: list, get, search, create_draft, update, link_notes, extract_tasks, attach_artifact
  • Smart Actions: 20 built-in prompt templates, copilot text transforms, scheduler, webhooks, URL extraction
  • Intake and Palace: URL/text intake jobs, intake rules, MemPalace memory/search/KG/wake-up routes
  • Agent audit trail: every write tool records agent action history
  • Datastore abstraction: Cosmos DB in production, in-memory for tests
  • Platform integrations: auth (JWT), telemetry, diagnostics, feature flags, kill-switch, blob storage
  • LLM integration: @bytelyst/llm with retry, timeout, and fallback heuristics

Environment

Copy backend/.env.example to backend/.env and web/.env.example to web/.env.local as needed.

Local memory mode:

  • NODE_ENV=development
  • DB_PROVIDER=memory
  • JWT_SECRET=dev-secret-do-not-use-in-prod
  • LLM_PROVIDER=mock
  • shared service URLs can point at local common-platform ports or remain disabled for isolated backend tests

Production requirements:

  • NODE_ENV=production
  • strong JWT_SECRET shared with platform-service or the production auth verification mechanism
  • DB_PROVIDER=cosmos
  • COSMOS_ENDPOINT, COSMOS_KEY, and COSMOS_DATABASE
  • FIELD_ENCRYPT_ENABLED=true with FIELD_ENCRYPT_KEY_PROVIDER=akv or a managed key provider and required key material
  • PLATFORM_SERVICE_URL, EXTRACTION_SERVICE_URL, and MCP_SERVER_URL
  • NEXT_PUBLIC_MCP_SERVER_URL=http://localhost:4007/api for local web settings/agent guidance
  • TELEMETRY_ENABLED=true and FEATURE_FLAGS_ENABLED=true when platform services are available
  • LLM_PROVIDER plus provider credentials (OPENAI_API_KEY or Azure OpenAI settings) for non-mock AI behavior

The May 5 production-readiness pass hardened production config to fail closed on unsafe defaults; see docs/PRODUCTION_READINESS_HANDOFF_ROADMAP.md.

Tests

pnpm --filter @notelett/backend run typecheck
pnpm --filter @notelett/backend run test
pnpm --filter @notelett/backend run lint
pnpm --filter @notelett/web run typecheck
pnpm --filter @notelett/web run test
pnpm --filter @notelett/web run lint
pnpm --filter @notelett/web run test:e2e
pnpm --filter @notelett/mobile run typecheck
pnpm --filter @notelett/mobile run test
pnpm --filter @notelett/mobile run lint
pnpm run verify

Current May 5 production-readiness status:

  • pnpm run verify passes backend/web/mobile typecheck and tests, plus backend/web production builds.
  • backend, web, and mobile lint commands exit 0 with tracked advisory warnings.
  • web Playwright release flows pass.
  • release-guard audits pass common-platform secret scan, hardcoded color/token checks, and active product/API drift checks.
  • Docker compose smoke and live shared-service smoke are explicit environment deferrals on this host because Docker is unavailable and platform-service/extraction-service/mcp-server are not running with Cosmos credentials.

See docs/PRODUCTION_READINESS_HANDOFF_ROADMAP.md Phase P10 for exact commands, commit hashes, warnings, and deferral notes.

Docs