test(e2e): docker compose E2E test + seed scripts + 9-step verification

Implements the full E2E flow against the deployed docker stack and
documents it as a repeatable test playbook.

Surfaced and fixed three real issues while building the E2E:

1. JWT secret mismatch — docker-compose.override.yml backend was using
   a NoteLett-only JWT_SECRET that platform-service did not share, so
   every Authorization: Bearer call returned 'Invalid or expired token'.
   Aligned the override to use platform-service's actual secret
   (dev-ecosystem-secret-do-not-use-in-production).

2. CORS preflight missing PATCH/DELETE — @bytelyst/fastify-core registers
   @fastify/cors with only { origin }, which leaves Access-Control-Allow-
   Methods at the @fastify/cors default of 'GET,HEAD,POST'. Real browser
   PATCH/DELETE preflights would fail. Added an onSend hook in
   backend/src/server.ts that rewrites the header to
   'GET,HEAD,POST,PATCH,PUT,DELETE,OPTIONS' on CORS preflight responses.

3. Product 'notelett' wasn't registered with platform-service — auth
   register/login both error with 'Unknown or disabled product: notelett'.
   The seed script now POSTs to /api/products idempotently.

Deliverables:

- scripts/e2e-docker-seed.sh — idempotent: registers the notelett product
  and creates two test users (admin@notelett.app with role=admin who can
  write, user@notelett.app with role=user who is read-only). Re-runs are
  no-ops once seeded.

- scripts/e2e-docker-test.sh — 9-step E2E that drives the deployed stack
  via HTTP only (no browser): login → CORS preflight for PATCH →
  workspace create → note create → note read → note PATCH (status:
  draft→active) → note list → note delete → workspace delete.

- docs/testing/E2E_DOCKER_TESTING.md — full playbook covering prereqs,
  seed, automated E2E, manual UI smoke, stack architecture diagram,
  troubleshooting (JWT mismatch, unknown product, role rejection,
  CORS, port conflict, data loss), tear-down, CI wiring guidance.

- package.json — pnpm e2e:docker:seed and pnpm e2e:docker:test
  shortcuts.

Verified live on this host's deployed stack:

  $ bash scripts/e2e-docker-seed.sh
  ↷ product 'notelett' already exists
  ↷ admin user already registered + login works
  ✓ user created
  🟢 Seed complete.

  $ bash scripts/e2e-docker-test.sh
  ✓ user=usr_e094e0c2-... role=admin
  ✓ CORS allows PATCH
  ✓ workspace created
  ✓ note created
  ✓ note read matches
  ✓ note patched (status: draft → active)
  ✓ note list returned (1 item)
  ✓ note deleted (HTTP 204)
  ✓ workspace deleted (HTTP 204)
  🟢 All 9 E2E steps passed.

Backend regression suite still green: 380/380.
This commit is contained in:
saravanakumardb1 2026-05-23 01:16:19 -07:00
parent e5221afb87
commit d5e857dbf7
6 changed files with 453 additions and 1 deletions

View File

@ -50,6 +50,20 @@ const app = await createServiceApp({
readiness: true,
});
// The shared @bytelyst/fastify-core registers @fastify/cors with just
// { origin }, which leaves Access-Control-Allow-Methods at the @fastify/cors
// default of 'GET,HEAD,POST'. NoteLett also uses PATCH and DELETE on notes,
// workspaces, relationships, tasks, artifacts, prompts, etc., so SPA
// preflights would fail in real browsers. Extend the methods list with
// an onSend hook that rewrites the header for CORS preflight responses.
const ALLOWED_METHODS = 'GET,HEAD,POST,PATCH,PUT,DELETE,OPTIONS';
app.addHook('onSend', async (req, reply, payload) => {
if (req.method === 'OPTIONS' && reply.getHeader('access-control-allow-origin')) {
reply.header('access-control-allow-methods', ALLOWED_METHODS);
}
return payload;
});
await registerOptionalJwtContext(app, {
verifyToken: async (token: string) => {
const { payload } = await jwtVerify(token, jwtSecret, { issuer: 'bytelyst-platform' });

View File

@ -26,7 +26,10 @@ services:
EXTRACTION_SERVICE_URL: "http://host.docker.internal:4005"
MCP_SERVER_URL: "http://host.docker.internal:4007"
DB_PROVIDER: memory
JWT_SECRET: "dev-secret-change-me-at-least-32-characters-long"
# MUST match the JWT_SECRET used by the sibling platform-service so
# tokens platform issues are accepted by NoteLett backend. The
# platform-service in this dev compose stack uses the value below.
JWT_SECRET: "dev-ecosystem-secret-do-not-use-in-production"
web:
ports: !override

View File

@ -0,0 +1,224 @@
# E2E Testing — Docker Compose Stack
> **Audience:** anyone testing NoteLett locally via the Docker compose stack.
> **Last verified:** May 23, 2026 against `docker-compose.override.yml` and platform-service `dev-ecosystem-secret-do-not-use-in-production`.
## What this exercises
The full critical path of a NoteLett user:
| Step | Surface | Endpoint / Action |
|---|---|---|
| 1 | **platform-service** (4003) | `POST /api/auth/login` — issues a JWT with `iss=bytelyst-platform` |
| 2 | **notelett-backend** (4016) | CORS preflight for PATCH from origin `localhost:3050` |
| 3 | notelett-backend | `POST /api/workspaces` — requires `admin` role |
| 4 | notelett-backend | `POST /api/notes` |
| 5 | notelett-backend | `GET /api/notes/:id?workspaceId=...` (Cosmos point read) |
| 6 | notelett-backend | `PATCH /api/notes/:id?workspaceId=...` — flips status `draft → active` |
| 7 | notelett-backend | `GET /api/notes?workspaceId=...` (list) |
| 8 | notelett-backend | `DELETE /api/notes/:id?workspaceId=...` |
| 9 | notelett-backend | `DELETE /api/workspaces/:id` |
The backend itself does NOT issue tokens; it only consumes JWTs the platform-service signs. The two services share a `JWT_SECRET` via `docker-compose.override.yml`. If those don't match, every authenticated call returns `Invalid or expired token`.
## Prerequisites
1. **Sibling services running** on host:
- `platform-service` on `:4003`
- `extraction-service` on `:4005` (optional for this E2E, used elsewhere)
- `mcp-server` on `:4007` (optional)
- `cosmos-emulator` on `:8081` (optional — `DB_PROVIDER=memory` is the default for compose)
2. **Docker images built**:
```bash
bash scripts/docker-prep.sh
docker compose build
bash scripts/docker-prep.sh --restore
```
3. **Compose up**:
```bash
docker compose up -d
```
This brings up `notelett-backend` (port 4016) and `notelett-web` (host port 3050 → container 3045) via `docker-compose.override.yml`.
4. **Verify health**:
```bash
curl -sS http://localhost:4016/health # → {"status":"ok",...}
curl -sS http://localhost:3050/dashboard # → HTTP 200
```
## One-time seed (idempotent)
```bash
bash scripts/e2e-docker-seed.sh
```
What it does:
| Resource | Endpoint | Skip if exists |
|---|---|---|
| `notelett` product on platform-service | `POST /api/products` | Yes (checks `GET /api/products/notelett`) |
| Admin user `admin@notelett.app` | `POST /api/auth/register` (role=admin) | Yes (probes login first) |
| Read-only user `user@notelett.app` | `POST /api/auth/register` (role=user) | Yes (probes login first) |
Test credentials produced:
| Email | Password | Role | Can write? |
|---|---|---|---|
| `admin@notelett.app` | `Notelett!Test#2026` | `admin` | ✅ yes |
| `user@notelett.app` | `Notelett!Test#2026` | `user` | ❌ no (`requireWriter` rejects `user` role) |
## Run the E2E test
```bash
bash scripts/e2e-docker-test.sh
```
Expected output (last verified May 23, 2026):
```
── 1. Login as admin@notelett.app ──
✓ user=usr_e094e0c2-ef52-4ad5-b7af-6a1cb6df700a role=admin
── 2. CORS preflight for PATCH ──
✓ CORS allows PATCH
── 3. Create workspace ──
id=ws-e2e-... name=E2E Workspace
✓ workspace created
── 4. Create note ──
id=note-e2e-... status=draft title=E2E Note
✓ note created
── 5. Read note ──
title=E2E Note tags=['e2e']
✓ note read matches
── 6. PATCH note (title + status → active) ──
title=E2E Note (edited) status=active
✓ note patched
── 7. List notes in workspace ──
total=1 items=['E2E Note (edited)']
✓ note list returned
── 8. DELETE note ──
delete HTTP 204
✓ note deleted
── 9. DELETE workspace ──
delete HTTP 204
✓ workspace deleted
🟢 All 9 E2E steps passed.
```
## Manual UI test
After seed + compose up:
1. Open **http://localhost:3050/login** in a browser.
2. Sign in with `admin@notelett.app` / `Notelett!Test#2026`.
3. You should be redirected to `/dashboard`.
4. Use the sidebar to navigate to **Workspaces** → click **Create Workspace** → name it, save.
5. Open the workspace → **Create Note** → title + body → save.
6. From the note detail page:
- Edit the title inline (auto-saves)
- Add a tag
- Try **Smart Actions** → "Shorten" (uses mock LLM in this deploy)
- Click **Share** → "Copy as Text"
7. Navigate to **Settings** → confirm the Profile card shows `admin@notelett.app`.
8. Sign out — should land on `/login`.
If any step fails see the troubleshooting section below.
## Stack architecture (this deployment)
```
┌────────────────────────────┐
Browser ───▶ │ notelett-web :3050 │ (Next.js standalone in Docker)
│ serves SPA + assets │
└─────────┬──────────────────┘
│ SPA calls
┌───────────────────────────────┐
│ platform-service :4003 │ (auth, products, flags)
│ signs JWT with iss= │
│ bytelyst-platform │
└─────────┬─────────────────────┘
│ POST /api/auth/login → {accessToken, user}
Browser stores accessToken in localStorage
│ All NoteLett API calls send
│ Authorization: Bearer <accessToken>
┌───────────────────────────────┐
│ notelett-backend :4016 │ (Fastify, DB_PROVIDER=memory)
│ verifies JWT with SAME │
│ JWT_SECRET as platform │
│ /api/workspaces, /api/notes, │
│ ... │
└───────────────────────────────┘
```
## Troubleshooting
### `Invalid or expired token` from notelett-backend
Cause: JWT signing secret on platform-service does not match NoteLett backend.
Fix:
```bash
docker exec mygh-platform-service-1 env | grep JWT_SECRET
# Copy that value into docker-compose.override.yml under services.backend.environment.JWT_SECRET
docker compose up -d --force-recreate backend
```
### `Unknown or disabled product: notelett`
Cause: platform-service product cache doesn't have `notelett`.
Fix: re-run `bash scripts/e2e-docker-seed.sh` — step 1 will register the product. If it already says "exists" but the error persists, the platform-service may need a cache reload — restart it: `docker restart mygh-platform-service-1`.
### `Write access requires editor, admin, or owner role`
Cause: logged-in user has `user` or `viewer` role, not `admin`.
Fix: use `admin@notelett.app`, not `user@notelett.app`. The platform-service register endpoint accepts roles `admin | viewer | user` while NoteLett's `requireWriter` accepts `editor | admin | owner`. **Only `admin` works for both write operations and platform-service registration.**
### CORS preflight rejects PATCH/DELETE
Cause: rare — only happens if backend was deployed before commit `<this-feature-commit>`.
Fix: rebuild backend image: `bash scripts/docker-prep.sh && docker compose build backend && bash scripts/docker-prep.sh --restore && docker compose up -d --force-recreate backend`.
### Port 3000 already in use
Cause: another container/service (Grafana, etc.) holds host port 3000.
Fix: this repo's `docker-compose.override.yml` already remaps web to port 3050. If you need a different port, edit the override file and re-run `docker compose up -d --force-recreate web`.
### Data resets on container restart
Expected — `DB_PROVIDER=memory`. To persist, set Cosmos credentials in `docker-compose.override.yml`:
```yaml
services:
backend:
environment:
DB_PROVIDER: cosmos
COSMOS_ENDPOINT: http://host.docker.internal:8081
COSMOS_KEY: C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==
COSMOS_DATABASE: notelett_local
```
Then `docker compose up -d --force-recreate backend`.
## Tear-down
```bash
docker compose down # stop containers, keep volumes
docker compose down --volumes # stop and clear data
```
## CI integration
`scripts/e2e-docker-test.sh` is suitable for a CI job that:
1. Starts the compose stack (or relies on services as GitHub Actions services)
2. Runs `e2e-docker-seed.sh` once
3. Runs `e2e-docker-test.sh`
The seed script is idempotent so reruns are safe. Currently the GitHub Actions `release-flows` Playwright job covers browser-level E2E; `e2e-docker-test.sh` covers the HTTP-API E2E and can be wired in as a separate job after the existing `docker-build` job.
## Related docs
- [`docs/PRODUCTION_READINESS_HANDOFF_ROADMAP.md`](../PRODUCTION_READINESS_HANDOFF_ROADMAP.md) — full production gate
- [`docs/runbooks/SECRET_MANAGEMENT.md`](../runbooks/SECRET_MANAGEMENT.md) — how to rotate the shared JWT secret
- [`docs/runbooks/MEK_ROTATION.md`](../runbooks/MEK_ROTATION.md) — field-encryption key rotation (when Cosmos is enabled)

View File

@ -10,6 +10,8 @@
"install:gitea": "BYTELYST_PACKAGE_SOURCE=gitea pnpm install -r",
"smoke:local": "bash scripts/local-smoke.sh",
"smoke:compose": "bash scripts/compose-smoke.sh",
"e2e:docker:seed": "bash scripts/e2e-docker-seed.sh",
"e2e:docker:test": "bash scripts/e2e-docker-test.sh",
"seed:bootstrap": "pnpm --filter @notelett/backend run bootstrap:seed",
"audit:release-guards": "bash scripts/release-guard-audit.sh",
"audit:ui": "bash scripts/ui-drift-audit.sh",

99
scripts/e2e-docker-seed.sh Executable file
View File

@ -0,0 +1,99 @@
#!/usr/bin/env bash
# NoteLett Docker compose E2E seed.
#
# Idempotent setup for scripts/e2e-docker-test.sh and for manual UI testing:
# 1. Register the `notelett` product with the running platform-service.
# 2. Register one admin test user (admin@notelett.app).
# 3. Register one read-only test user (user@notelett.app).
#
# Re-running is safe: each step skips if the resource already exists.
#
# Prerequisites:
# - platform-service is reachable at http://localhost:4003
# - NoteLett backend is reachable at http://localhost:4016 (or run
# `docker compose up -d` first)
#
# Usage:
# bash scripts/e2e-docker-seed.sh
set -e
PLATFORM="${PLATFORM:-http://localhost:4003}"
ADMIN_EMAIL="admin@notelett.app"
ADMIN_PASS="Notelett!Test#2026"
USER_EMAIL="user@notelett.app"
USER_PASS="Notelett!Test#2026"
step() { echo ""; echo "── $* ──"; }
ok() { echo "$*"; }
skip() { echo "$*"; }
fail() { echo "$*" >&2; exit 1; }
step "1. Register product 'notelett' with platform-service"
EXISTING=$(curl -sS "$PLATFORM/api/products/notelett" -o /dev/null -w "%{http_code}")
if [[ "$EXISTING" == "200" ]]; then
skip "product 'notelett' already exists"
else
RESP=$(curl -sS -X POST "$PLATFORM/api/products" -H "Content-Type: application/json" \
-d '{
"productId": "notelett",
"displayName": "NoteLett",
"licensePrefix": "NL",
"packageName": "@notelett/web",
"defaultPlan": "free",
"trialDays": 14,
"websiteUrl": "https://notelett.app",
"status": "active"
}')
echo "$RESP" | python3 -c "import json,sys; d=json.load(sys.stdin); print(f' id={d.get(\"id\")} status={d.get(\"status\")}')" \
|| fail "product create failed: $RESP"
ok "product created"
fi
step "2. Register admin test user ($ADMIN_EMAIL)"
LOGIN_PROBE=$(curl -sS -X POST "$PLATFORM/api/auth/login" -H "Content-Type: application/json" \
-d "{\"email\":\"$ADMIN_EMAIL\",\"password\":\"$ADMIN_PASS\",\"productId\":\"notelett\"}")
if echo "$LOGIN_PROBE" | grep -q "accessToken"; then
skip "admin user already registered + login works"
else
REG=$(curl -sS -X POST "$PLATFORM/api/auth/register" -H "Content-Type: application/json" \
-d "{
\"email\":\"$ADMIN_EMAIL\",
\"password\":\"$ADMIN_PASS\",
\"displayName\":\"Admin User\",
\"role\":\"admin\",
\"productId\":\"notelett\"
}")
echo "$REG" | python3 -c "import json,sys; d=json.load(sys.stdin); u=d.get('user',{}); print(f' email={u.get(\"email\")} role={u.get(\"role\")}')" \
|| fail "admin register failed: $REG"
ok "admin user created"
fi
step "3. Register read-only test user ($USER_EMAIL)"
LOGIN_PROBE=$(curl -sS -X POST "$PLATFORM/api/auth/login" -H "Content-Type: application/json" \
-d "{\"email\":\"$USER_EMAIL\",\"password\":\"$USER_PASS\",\"productId\":\"notelett\"}")
if echo "$LOGIN_PROBE" | grep -q "accessToken"; then
skip "user already registered + login works"
else
REG=$(curl -sS -X POST "$PLATFORM/api/auth/register" -H "Content-Type: application/json" \
-d "{
\"email\":\"$USER_EMAIL\",
\"password\":\"$USER_PASS\",
\"displayName\":\"Read-only User\",
\"role\":\"user\",
\"productId\":\"notelett\"
}")
echo "$REG" | python3 -c "import json,sys; d=json.load(sys.stdin); u=d.get('user',{}); print(f' email={u.get(\"email\")} role={u.get(\"role\")}')" \
|| fail "user register failed: $REG"
ok "user created"
fi
echo ""
echo "🟢 Seed complete."
echo ""
echo "Test credentials:"
echo " Admin (read+write): $ADMIN_EMAIL / $ADMIN_PASS"
echo " Read-only: $USER_EMAIL / $USER_PASS"
echo ""
echo "Next: bash scripts/e2e-docker-test.sh"
echo "Or open the web app: http://localhost:3050/login"

110
scripts/e2e-docker-test.sh Executable file
View File

@ -0,0 +1,110 @@
#!/usr/bin/env bash
# NoteLett Docker compose E2E test.
#
# Drives the full critical-path auth + notes flow against the deployed
# Docker stack via HTTP:
# 1. Login to platform-service (port 4003)
# 2. CORS preflight for PATCH (port 4016)
# 3. Create workspace (port 4016, requires admin role)
# 4. Create note (port 4016)
# 5. Read note by id (point read, /workspaceId partition)
# 6. PATCH note (title + status)
# 7. List notes in workspace
# 8. DELETE note
# 9. DELETE workspace
#
# Prerequisites:
# - `docker compose up -d` has been run from the repo root.
# - The sibling platform-service is running on http://localhost:4003 and
# has been seeded with the `notelett` product + a test admin user.
# Run scripts/e2e-docker-seed.sh once to do both.
# - JWT_SECRET in docker-compose.override.yml MUST match platform-service.
#
# Usage:
# bash scripts/e2e-docker-test.sh
#
# Environment overrides:
# PLATFORM default http://localhost:4003
# BACKEND default http://localhost:4016
# EMAIL default admin@notelett.app
# PASS default Notelett!Test#2026 (matches scripts/e2e-docker-seed.sh)
set -e
PLATFORM="${PLATFORM:-http://localhost:4003}"
BACKEND="${BACKEND:-http://localhost:4016}"
EMAIL="${EMAIL:-admin@notelett.app}"
PASS="${PASS:-Notelett!Test#2026}"
step() { echo ""; echo "── $* ──"; }
ok() { echo "$*"; }
fail() { echo "$*" >&2; exit 1; }
step "1. Login as $EMAIL"
LOGIN=$(curl -sS -X POST "$PLATFORM/api/auth/login" -H "Content-Type: application/json" \
-d "{\"email\":\"$EMAIL\",\"password\":\"$PASS\",\"productId\":\"notelett\"}")
TOKEN=$(echo "$LOGIN" | python3 -c "import json,sys; print(json.load(sys.stdin).get('accessToken',''))")
USER_ID=$(echo "$LOGIN" | python3 -c "import json,sys; d=json.load(sys.stdin); print(d.get('user',{}).get('id',''))")
ROLE=$(echo "$LOGIN" | python3 -c "import json,sys; d=json.load(sys.stdin); print(d.get('user',{}).get('role',''))")
if [[ -z "$TOKEN" ]]; then
echo "$LOGIN" >&2
fail "login failed — run scripts/e2e-docker-seed.sh first"
fi
ok "user=$USER_ID role=$ROLE"
step "2. CORS preflight for PATCH"
curl -sS -X OPTIONS "$BACKEND/api/notes/x" \
-H "Origin: http://localhost:3050" \
-H "Access-Control-Request-Method: PATCH" -D - -o /dev/null \
| grep -i "access-control-allow-methods" \
| grep -qE "PATCH" \
&& ok "CORS allows PATCH" \
|| fail "CORS does not advertise PATCH"
step "3. Create workspace"
WS_ID="ws-e2e-$(date +%s)"
WS=$(curl -sS -X POST "$BACKEND/api/workspaces" \
-H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
-d "{\"id\":\"$WS_ID\",\"name\":\"E2E Workspace\",\"description\":\"Created by e2e-docker-test.sh\"}")
echo "$WS" | python3 -c "import json,sys; d=json.load(sys.stdin); print(f' id={d[\"id\"]} name={d[\"name\"]}')"
ok "workspace created"
step "4. Create note"
NOTE_ID="note-e2e-$(date +%s)"
NOTE=$(curl -sS -X POST "$BACKEND/api/notes" \
-H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
-d "{\"id\":\"$NOTE_ID\",\"workspaceId\":\"$WS_ID\",\"title\":\"E2E Note\",\"body\":\"<p>created by script</p>\",\"tags\":[\"e2e\"],\"sourceType\":\"manual\"}")
echo "$NOTE" | python3 -c "import json,sys; d=json.load(sys.stdin); print(f' id={d[\"id\"]} status={d[\"status\"]} title={d[\"title\"]}')"
ok "note created"
step "5. Read note"
GOT=$(curl -sS "$BACKEND/api/notes/$NOTE_ID?workspaceId=$WS_ID" -H "Authorization: Bearer $TOKEN")
echo "$GOT" | python3 -c "import json,sys; d=json.load(sys.stdin); print(f' title={d[\"title\"]} tags={d[\"tags\"]}')"
[[ "$(echo "$GOT" | python3 -c "import json,sys; print(json.load(sys.stdin)[\"id\"])")" == "$NOTE_ID" ]] \
&& ok "note read matches" \
|| fail "note read mismatch"
step "6. PATCH note (title + status → active)"
PATCHED=$(curl -sS -X PATCH "$BACKEND/api/notes/$NOTE_ID?workspaceId=$WS_ID" \
-H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
-d '{"title":"E2E Note (edited)","status":"active"}')
echo "$PATCHED" | python3 -c "import json,sys; d=json.load(sys.stdin); print(f' title={d[\"title\"]} status={d[\"status\"]}')"
ok "note patched"
step "7. List notes in workspace"
LIST=$(curl -sS "$BACKEND/api/notes?workspaceId=$WS_ID" -H "Authorization: Bearer $TOKEN")
echo "$LIST" | python3 -c "import json,sys; d=json.load(sys.stdin); print(f' total={d[\"total\"]} items={[n[\"title\"] for n in d[\"items\"]]}')"
ok "note list returned"
step "8. DELETE note"
curl -sS -X DELETE "$BACKEND/api/notes/$NOTE_ID?workspaceId=$WS_ID" \
-H "Authorization: Bearer $TOKEN" -o /dev/null -w " delete HTTP %{http_code}\n"
ok "note deleted"
step "9. DELETE workspace"
curl -sS -X DELETE "$BACKEND/api/workspaces/$WS_ID" \
-H "Authorization: Bearer $TOKEN" -o /dev/null -w " delete HTTP %{http_code}\n"
ok "workspace deleted"
echo ""
echo "🟢 All 9 E2E steps passed."