From 95fc26200617cf1e47d72a78d78bb3f19a41b9e2 Mon Sep 17 00:00:00 2001 From: root Date: Wed, 27 May 2026 18:41:17 +0000 Subject: [PATCH] chore: add agent context docs --- .claude/AGENTS.md | 1 + .cline/AGENTS.md | 1 + .cursor/AGENTS.md | 1 + .devin/AGENTS.md | 206 +++++++++++++++++++++++++++++++++++ .devin/CONTEXT.md | 226 +++++++++++++++++++++++++++++++++++++++ .windsurf/AGENTS.md | 1 + REPO_CONTEXT.md | 132 +++++++++++++++++++++++ backend/vitest.config.ts | 9 +- 8 files changed, 576 insertions(+), 1 deletion(-) create mode 100644 .claude/AGENTS.md create mode 100644 .cline/AGENTS.md create mode 100644 .cursor/AGENTS.md create mode 100644 .devin/AGENTS.md create mode 100644 .devin/CONTEXT.md create mode 100644 .windsurf/AGENTS.md create mode 100644 REPO_CONTEXT.md diff --git a/.claude/AGENTS.md b/.claude/AGENTS.md new file mode 100644 index 0000000..86e2953 --- /dev/null +++ b/.claude/AGENTS.md @@ -0,0 +1 @@ +Refer to ../REPO_CONTEXT.md for repository overview and ../AGENTS.md for complete project instructions. \ No newline at end of file diff --git a/.cline/AGENTS.md b/.cline/AGENTS.md new file mode 100644 index 0000000..86e2953 --- /dev/null +++ b/.cline/AGENTS.md @@ -0,0 +1 @@ +Refer to ../REPO_CONTEXT.md for repository overview and ../AGENTS.md for complete project instructions. \ No newline at end of file diff --git a/.cursor/AGENTS.md b/.cursor/AGENTS.md new file mode 100644 index 0000000..86e2953 --- /dev/null +++ b/.cursor/AGENTS.md @@ -0,0 +1 @@ +Refer to ../REPO_CONTEXT.md for repository overview and ../AGENTS.md for complete project instructions. \ No newline at end of file diff --git a/.devin/AGENTS.md b/.devin/AGENTS.md new file mode 100644 index 0000000..9bb6ade --- /dev/null +++ b/.devin/AGENTS.md @@ -0,0 +1,206 @@ +# ByteLyst NoteLett - Agent Instructions + +> This file is read automatically by Claude Code. For full project instructions +> shared across all AI tools, see [`CONTEXT.md`](CONTEXT.md). + +## Critical Deployment Patterns (Learned from Production Deployment) + +### 1. Docker Build Preparation +**Before building Docker images, you MUST run the dependency packing script:** +```bash +bash scripts/docker-prep.sh +``` + +This script: +- Builds all @bytelyst/* packages from learning_ai_common_plat +- Packs them as tarballs in `.docker-deps/` +- Rewrites package.json files to reference the tarballs +- Injects pnpm.overrides for version consistency + +**After building, ALWAYS restore the package.json files:** +```bash +bash scripts/docker-prep.sh --restore +``` + +### 2. Dockerfile Configuration +**Dockerfiles MUST be updated to remove Gitea secrets and use .docker-deps:** + +**Backend Dockerfile changes:** +- Remove `--mount=type=secret,id=gitea_npm_token` lines +- Remove TOKEN environment setup +- Add `COPY .docker-deps/ ../.docker-deps/` before install +- Simplified install command: `RUN pnpm install --ignore-scripts --lockfile=false` + +**Web Dockerfile changes:** +- Remove `--mount=type=secret,id=gitea_npm_token` lines +- Remove GITEA_NPM_TOKEN environment setup +- Ensure .docker-deps is copied before install + +### 3. Public API URL Configuration +**NEXT_PUBLIC_ environment variables MUST use public URLs, not internal Docker network names:** + +**Incorrect:** +```yaml +NEXT_PUBLIC_NOTES_API_URL: http://localhost:4016/api +NEXT_PUBLIC_NOTES_API_URL: http://backend:4011 +``` + +**Correct:** +```yaml +NEXT_PUBLIC_NOTES_API_URL: https://api.bytelyst.com/notelett +``` + +This is because browsers cannot access internal Docker network names. The web app needs to connect to the public API endpoint. + +### 4. Docker Compose Environment Variable Format +**Use YAML mapping syntax, NOT array syntax:** + +**Incorrect:** +```yaml +environment: + - NODE_ENV=production + - PORT=4016 +``` + +**Correct:** +```yaml +environment: + NODE_ENV: production + PORT: 4016 +``` + +### 5. Production vs Development Mode +**Production mode (NODE_ENV=production) requires:** +- DB_PROVIDER: cosmos (not memory) +- FIELD_ENCRYPT_ENABLED: true (not false) +- FIELD_ENCRYPT_KEY_PROVIDER: akv or env (not memory) + +**If these are not configured, use development mode:** +```yaml +NODE_ENV: development +``` + +### 6. Container Names for Caddy Routing +**Container names MUST match Caddyfile configuration:** +- Backend: `notelett-backend` +- Web: `notelett-web` + +Add to docker-compose.yml: +```yaml +services: + backend: + container_name: notelett-backend + web: + container_name: notelett-web +``` + +### 7. Caddy Network Connectivity +**Caddy MUST be connected to the app networks for reverse proxy:** +```bash +docker network connect learning_ai_notes_default caddy +docker network connect learning_ai_common_plat_default caddy +``` + +### 8. DNS Configuration +**For new subdomains, add DNS records via GoDaddy API:** +```bash +# Example for notes.bytelyst.com +curl -X PUT "https://api.godaddy.com/v1/domains/bytelyst.com/records/A/notes" \ + -H "Authorization: sso-key $GODADDY_API_KEY:$GODADDY_API_SECRET" \ + -H "Content-Type: application/json" \ + -d '[{"data": "187.124.159.82", "name": "notes", "ttl": 600, "type": "A"}]' +``` + +Use the same IP as other bytelyst.com subdomains (187.124.159.82). + +### 9. Caddy Configuration Pattern +**Add subdomain routing to /opt/bytelyst/Caddyfile:** +```caddy +notes.bytelyst.com { + encode gzip + reverse_proxy notelett-web:3045 +} +``` + +Then reload Caddy: +```bash +docker exec caddy caddy reload --config /etc/caddy/Caddyfile +``` + +### 10. Deployment Script Usage +**ALWAYS use the deployment script from learning_ai_devops_tools:** +```bash +cd ../learning_ai_devops_tools +bash deploy-notes.sh +``` + +Options: +- Option 1: Normal deployment (with cache, with health checks) +- Option 5: Force + No-cache (skip checks, force rebuild) - use when config changes +- Option 6: Force + Skip health checks (skip both) - use when health checks are flaky + +## Deployment Checklist + +Before deploying: +1. [ ] Run `bash scripts/docker-prep.sh` to package dependencies +2. [ ] Update Dockerfiles to remove Gitea secrets and use .docker-deps +3. [ ] Set NEXT_PUBLIC_ variables to use public API URLs +4. [ ] Use YAML mapping syntax for environment variables +5. [ ] Set correct NODE_ENV (development unless Cosmos/AKV configured) +6. [ ] Set container names to match Caddyfile +7. [ ] Update Caddyfile with subdomain routing +8. [ ] Add DNS records via GoDaddy API +9. [ ] Connect Caddy to app networks +10. [ ] Run deployment script with appropriate options + +After deploying: +1. [ ] Run `bash scripts/docker-prep.sh --restore` to restore package.json +2. [ ] Test public URL accessibility +3. [ ] Test API endpoint accessibility +4. [ ] Verify SSL certificates are working +5. [ ] Check container logs for errors +6. [ ] Commit and push changes + +## Common Deployment Errors and Solutions + +### Error: "failed to calculate checksum of ref: /.docker-deps: not found" +**Solution:** Run `bash scripts/docker-prep.sh` before building + +### Error: "HTTP 400 urn:ietf:params:acme:error:dns - DNS problem: NXDOMAIN" +**Solution:** DNS records not propagated yet. Wait 5-30 minutes after adding DNS records, then restart Caddy. + +### Error: "Production DB_PROVIDER must be cosmos" +**Solution:** Either configure Cosmos DB credentials or set NODE_ENV to development + +### Error: "services.web.environment.[3]: unexpected type map[string]interface {}" +**Solution:** Convert environment variables from array syntax to YAML mapping syntax + +### Error: React hydration errors or backend connection failures +**Solution:** Check that NEXT_PUBLIC_ variables use public API URLs, not internal Docker network names + +## Quick Deployment Commands + +```bash +# Prepare dependencies +bash scripts/docker-prep.sh + +# Deploy +cd ../learning_ai_devops_tools +echo "5" | bash deploy-notes.sh # Force + No-cache + +# Restore after deployment +bash scripts/docker-prep.sh --restore + +# Test +curl -I https://notes.bytelyst.com +curl -s https://api.bytelyst.com/notelett/health +``` + +## Key Files for Deployment + +- `docker-compose.yml` - Container configuration +- `backend/Dockerfile` - Backend container build +- `web/Dockerfile` - Web container build +- `scripts/docker-prep.sh` - Dependency packaging script +- `/opt/bytelyst/Caddyfile` - Reverse proxy configuration +- `../learning_ai_devops_tools/deploy-notes.sh` - Deployment script \ No newline at end of file diff --git a/.devin/CONTEXT.md b/.devin/CONTEXT.md new file mode 100644 index 0000000..e823a79 --- /dev/null +++ b/.devin/CONTEXT.md @@ -0,0 +1,226 @@ +# ByteLyst NoteLett - Project Context + +> **Purpose**: This file provides project-specific context for AI coding agents working in this repository. + +## Companion Repositories (Always Available) + +This repository has access to these companion repositories: + +- **learning_ai_common_plat** - Platform packages (@bytelyst/*) + - Context: `learning_ai_common_plat/REPO_CONTEXT.md` + - Provides: Shared libraries, configuration, auth, cosmos, telemetry, etc. + - Location: `../learning_ai_common_plat/` + +- **learning_ai_devops_tools** - Deployment and operational scripts + - Context: `learning_ai_devops_tools/REPO_CONTEXT.md` + - Provides: Deployment scripts, GitHub tools, operational automation + - Location: `../learning_ai_devops_tools/` + +## Project Overview + +**ByteLyst NoteLett** - Agentic note-taking workspace for notes, tasks, relationships, and workspaces with: +- AI-powered note organization and linking +- Smart Actions with prompt templates +- Cross-product transcript and trail report imports +- MCP (Model Context Protocol) integration for external tools +- Multi-platform support (Web, iOS, Android) + +## Key Locations + +### Source Code +- **Backend**: `backend/` - Fastify 5 + TypeScript ESM backend (port 4016) +- **Web**: `web/` - Next.js 16 + React 19 web app (port 3045) +- **Mobile**: `mobile/` - React Native + Expo companion app +- **Shared**: `shared/` - Shared TypeScript utilities + +### Configuration +- **Backend Config**: `backend/src/lib/config.ts` - Zod-validated env config +- **Docker Config**: `docker-compose.yml` - Container orchestration +- **TypeScript Config**: `backend/tsconfig.json`, `web/tsconfig.json` + +### Documentation +- **Product Docs**: `docs/PRD.md`, `docs/ROADMAP.md` +- **Production Readiness**: `docs/PRODUCTION_READINESS_HANDOFF_ROADMAP.md` +- **UI/UX Migration**: `docs/UI_UX_PLATFORM_CORE_ROADMAP.md` + +## Deployment Patterns + +### Primary Deployment Method +- **Script**: `learning_ai_devops_tools/deploy-notes.sh` +- **Usage**: Always check this script first before attempting custom deployment +- **Pattern**: Interactive menu with options for cache, force, and health checks +- **Features**: + - Package publication checks + - --no-cache support + - Health verification + - Dirty checks + +### Deployment Requirements +- **GITEA_NPM_TOKEN**: Required for @bytelyst/* package installation +- **Network**: Access to Gitea registry for package installation +- **Build Order**: learning_ai_common_plat → this project + +## MCP Integration + +### MCP Components +- **MCP Tool Contracts**: `backend/src/mcp/note-tool-contracts.ts` +- **MCP Tools**: `backend/src/mcp/note-tools.ts` +- **MCP Registration**: `backend/src/mcp/register-note-tools.ts` + +### MCP Features +- **8 MCP Tools**: Note CRUD, search, relationships, artifacts, etc. +- **Zod Schemas**: Type-safe tool definitions +- **Integration**: Fastify plugin for automatic registration + +### MCP Configuration +- **Environment Variables**: + - `MCP_SERVER_URL`: MCP server endpoint + - `MCP_ENABLED`: Feature flag (default: true) + +## Architecture Patterns + +### Backend Architecture +- **Framework**: Fastify 5 with TypeScript ESM +- **Database**: Azure Cosmos DB via @bytelyst/datastore +- **Platform Integration**: platform-service (port 4003) for auth, flags, telemetry +- **Extraction**: extraction-service (port 4005) for AI-powered task extraction +- **LLM**: @bytelyst/llm with retry + timeout hardening + +### Frontend Architecture +- **Framework**: Next.js 16 (App Router) + React 19 +- **State Management**: Zustand + localStorage +- **UI Components**: @bytelyst/ui + custom components +- **Styling**: TailwindCSS v4 + design tokens (--nl-*) + +### Mobile Architecture +- **Framework**: React Native + Expo +- **State Management**: Zustand + MMKV +- **Navigation**: Expo Router +- **Theme**: NoteLystTheme from design tokens + +## Common Operations + +### Backend Development +```bash +cd backend +npm run dev # Start backend in dev mode +npm run build # Build TypeScript +npm run test # Run tests +``` + +### Web Development +```bash +cd web +npm run dev # Start Next.js dev server +npm run build # Production build (uses --webpack flag) +npm run test # Run Vitest tests +``` + +### Mobile Development +```bash +cd mobile +npm start # Start Expo dev server +``` + +### Docker Development +```bash +docker compose up # Start all containers +docker compose down # Stop all containers +``` + +## Important Notes + +### Platform Integration +- **platform-service**: Used for auth, billing, flags, telemetry, blob storage +- **extraction-service**: Used for AI-powered task extraction from notes +- **Product ID**: "notelett" (required for all Cosmos documents) + +### Feature Flags +- **20 Feature Flags**: Core + Smart Actions features +- **Client**: @bytelyst/feature-flag-client +- **Backend**: backend/src/lib/feature-flags.ts + +### Design System +- **Tokens**: @bytelyst/design-tokens (--nl-* CSS custom properties) +- **Components**: @bytelyst/ui (Button, Card, Badge, Toast, etc.) +- **Never hardcode colors**: Always use theme tokens + +## Testing + +### Backend Tests +- **Framework**: Vitest +- **Coverage**: All modules (notes, workspaces, relationships, tasks, etc.) +- **Location**: `backend/src/modules/*/test.ts` + +### Web Tests +- **Framework**: Vitest + Playwright +- **Coverage**: Components, E2E flows +- **Location**: `web/src/*.dom.test.tsx`, `web/e2e/` + +### Mobile Tests +- **Framework**: Expo Test (planned) + +## Common Issues and Solutions + +### Platform Integration Issues +- **"platform-service unreachable"**: Check PLATFORM_SERVICE_URL +- **"Feature flag not found"**: Verify feature flag configuration +- **"Telemetry disabled"**: Check TELEMETRY_ENABLED setting + +### Deployment Issues +- **"Cannot find module @bytelyst/..."**: Build learning_ai_common_plat packages first +- **"Gitea registry unauthorized"**: Check GITEA_NPM_TOKEN +- **"Docker build network error"**: Ensure proper network access + +### Development Issues +- **Port conflicts**: Backend (4016), Web (3045) +- **Environment variables**: Copy backend/.env.example to backend/.env +- **TypeScript errors**: Run `npm run build` to see full error messages + +## Quick Reference + +| Task | Command | +|------|---------| +| Start development | `cd backend && npm run dev` | +| Build backend | `cd backend && npm run build` | +| Build web | `cd web && npm run build -- --webpack` | +| Deploy | `../learning_ai_devops_tools/deploy-notes.sh` | +| Run tests | `cd backend && npm run test` | +| Docker compose | `docker compose up -d` | + +## First Steps for Any Task + +1. **Check companion repo contexts**: + - Read `learning_ai_common_plat/REPO_CONTEXT.md` for package patterns + - Read `learning_ai_devops_tools/REPO_CONTEXT.md` for deployment patterns + +2. **Check deployment scripts**: + - Look for `learning_ai_devops_tools/deploy-notes.sh` + - Don't build custom deployment solutions + +3. **Check project-specific context**: + - Product docs: `docs/PRD.md`, `docs/ROADMAP.md` + - Production readiness: `docs/PRODUCTION_READINESS_HANDOFF_ROADMAP.md` + +4. **Verify environment setup**: + - Check backend/.env is configured + - Ensure GITEA_NPM_TOKEN is set for deployment + +## Project-Specific Patterns + +### Note CRUD Pattern +1. Types: Define Zod schemas in `types.ts` +2. Repository: Use @bytelyst/datastore getCollection() +3. Routes: Fastify routes with validation +4. All documents include `productId: "notelett"` + +### Smart Actions Pattern +1. Prompt Templates: Stored in note-prompts module +2. Runner: executePrompt() with retry + timeout +3. Scheduler: Cron schedules + webhook triggers +4. Integration: Web UI for template management + +### Cross-Product Integration +1. Phase 1: Transcript import from LysnrAI +2. Phase 3: Trail report import from ActionTrail +3. Helpers: backend/src/lib/ecosystem-phase*.ts \ No newline at end of file diff --git a/.windsurf/AGENTS.md b/.windsurf/AGENTS.md new file mode 100644 index 0000000..86e2953 --- /dev/null +++ b/.windsurf/AGENTS.md @@ -0,0 +1 @@ +Refer to ../REPO_CONTEXT.md for repository overview and ../AGENTS.md for complete project instructions. \ No newline at end of file diff --git a/REPO_CONTEXT.md b/REPO_CONTEXT.md new file mode 100644 index 0000000..cb2d29d --- /dev/null +++ b/REPO_CONTEXT.md @@ -0,0 +1,132 @@ +# ByteLyst NoteLett - Repository Context + +> **Purpose**: This file provides quick context for AI coding agents about what this repository contains and how to use it effectively. + +## What This Repository Provides + +This is the **NoteLett product repository** for the ByteLyst ecosystem. It contains: + +- **Agentic note-taking platform** with notes, tasks, relationships, and workspaces +- **Multi-platform support**: Web (Next.js 16), iOS (React Native), Android (React Native) +- **Smart Actions**: AI-powered prompt templates and automation +- **MCP Integration**: Model Context Protocol for external tool integration +- **Cross-product imports**: Transcript and trail report imports from other ByteLyst products + +## Companion Repositories (Always Available) + +This repository has access to these companion repositories: + +- **learning_ai_common_plat** - Platform packages (@bytelyst/*) + - Context: `../learning_ai_common_plat/REPO_CONTEXT.md` + - Provides: Shared libraries, configuration, auth, cosmos, telemetry, etc. + - Location: `../learning_ai_common_plat/` + +- **learning_ai_devops_tools** - Deployment and operational scripts + - Context: `../learning_ai_devops_tools/REPO_CONTEXT.md` + - Provides: Deployment scripts, GitHub tools, operational automation + - Location: `../learning_ai_devops_tools/` + +## Key Locations + +### Source Code +- **Backend**: `backend/` - Fastify 5 + TypeScript ESM backend (port 4016) + - `backend/src/modules/` - Feature modules (notes, workspaces, relationships, tasks, etc.) + - `backend/src/lib/` - Shared backend utilities +- **Web**: `web/` - Next.js 16 + React 19 web app (port 3045) + - `web/src/app/` - App Router pages + - `web/src/components/` - React UI components + - `web/src/lib/` - Pure TS client libraries +- **Mobile**: `mobile/` - React Native + Expo companion app + - `mobile/src/app/` - Expo Router screens + - `mobile/src/api/` - API clients + - `mobile/src/store/` - Zustand stores +- **Shared**: `shared/` - Shared TypeScript utilities and product config + +### Configuration +- **Backend Config**: `backend/src/lib/config.ts` - Zod-validated env config +- **Web Config**: `web/src/lib/product-config.ts` - Product identity + API URLs +- **Docker Config**: `docker-compose.yml` - Container orchestration +- **TypeScript Config**: `backend/tsconfig.json`, `web/tsconfig.json`, `mobile/tsconfig.json` + +### Documentation +- **Product Docs**: `docs/PRD.md`, `docs/ROADMAP.md` +- **Production Readiness**: `docs/PRODUCTION_READINESS_HANDOFF_ROADMAP.md` +- **UI/UX Migration**: `docs/UI_UX_PLATFORM_CORE_ROADMAP.md` +- **AI Agent Instructions**: `AGENTS.md` (complete instructions for all tools) + +### Deployment +- **Docker Prep**: `scripts/docker-prep.sh` - Package @bytelyst/* dependencies +- **Deployment Script**: `../learning_ai_devops_tools/deploy-notes.sh` +- **Dockerfiles**: `backend/Dockerfile`, `web/Dockerfile` + +## Common Usage Patterns + +### Development +```bash +# Install dependencies +pnpm install + +# Backend development +pnpm --filter @notelett/backend run dev + +# Web development +pnpm --filter @notelett/web run dev + +# Mobile development +pnpm --filter @notelett/mobile run start + +# Full verification +pnpm run verify +``` + +### Testing +```bash +# Backend tests +pnpm --filter @notelett/backend run test + +# Web tests +pnpm --filter @notelett/web run test + +# Mobile tests +pnpm --filter @notelett/mobile run test +``` + +### Docker Build +```bash +# Prepare dependencies (REQUIRED before Docker build) +bash scripts/docker-prep.sh + +# Build containers +docker-compose build + +# Restore after build +bash scripts/docker-prep.sh --restore +``` + +## Tech Stack + +| Layer | Technology | +|-------|-----------| +| **Backend** | Fastify 5, TypeScript ESM, Zod, Azure Cosmos DB | +| **Web** | Next.js 16 (App Router), React 19, TailwindCSS v4, Zustand | +| **Mobile** | React Native (Expo), TypeScript, expo-router, Zustand | +| **Shared packages** | @bytelyst/* from learning_ai_common_plat | +| **Platform** | platform-service (port 4003) for auth, flags, telemetry | +| **Database** | Azure Cosmos DB via @bytelyst/datastore | +| **LLM** | @bytelyst/llm (mock/openai/azure providers) | + +## Important Notes + +- **Product ID**: `notelett` (used in all Cosmos documents) +- **Package scope**: @notelett/* for workspace packages +- **Shared packages**: Uses @bytelyst/* from learning_ai_common_plat +- **Docker prep**: MUST run `scripts/docker-prep.sh` before Docker builds +- **Deployment**: ALWAYS use `../learning_ai_devops_tools/deploy-notes.sh` + +## First Steps for AI Agents + +1. **Read REPO_CONTEXT.md** (this file) - Understand the repository structure +2. **Read AGENTS.md** - Complete coding conventions and instructions +3. **Check companion repos** - learning_ai_common_plat and learning_ai_devops_tools +4. **Understand the stack** - Fastify backend, Next.js web, React Native mobile +5. **Follow deployment patterns** - Use docker-prep.sh and deployment scripts \ No newline at end of file diff --git a/backend/vitest.config.ts b/backend/vitest.config.ts index 8dcf0ea..94bb201 100644 --- a/backend/vitest.config.ts +++ b/backend/vitest.config.ts @@ -10,8 +10,15 @@ export default defineConfig({ // without Docker still get green locally. exclude: ['**/node_modules/**', 'src/**/*.cosmos.test.ts'], passWithNoTests: true, + // Some backend integration-style route tests build Fastify apps and import + // shared common-platform packages. On cold CI/dev hosts those first tests can + // legitimately spend more than Vitest's 5s default in transform/route setup + // before the actual assertion path runs. Keep the limit bounded, but high + // enough to avoid false negative launch-gate failures. + testTimeout: 15_000, + hookTimeout: 15_000, env: { ALLOW_ANONYMOUS_DEV: 'true', - }, + } }, });