From 72624aa8fc683947ee065100175e44c886fb9063 Mon Sep 17 00:00:00 2001 From: Saravana Achu Mac Date: Sat, 9 May 2026 14:12:58 -0700 Subject: [PATCH] docs: add comprehensive UX testing setup guide Created detailed documentation for UX testing infrastructure that can be replicated across all ByteLyst products: - Complete Playwright setup instructions - Test suite structure and patterns - Test runner script with server lifecycle management - AI-friendly report generation - Common platform UI integration - Design token usage patterns - Cipher design system integration - Step-by-step implementation guide - Troubleshooting and best practices - CI integration examples This guide enables other products to follow the exact same steps for UX testing using common platform packages, design tokens, and Cipher design system principles. --- docs/UX_TESTING_SETUP_GUIDE.md | 675 +++++++++++++++++++++++++++++++++ 1 file changed, 675 insertions(+) create mode 100644 docs/UX_TESTING_SETUP_GUIDE.md diff --git a/docs/UX_TESTING_SETUP_GUIDE.md b/docs/UX_TESTING_SETUP_GUIDE.md new file mode 100644 index 0000000..74efaf2 --- /dev/null +++ b/docs/UX_TESTING_SETUP_GUIDE.md @@ -0,0 +1,675 @@ +# UX Testing Setup Guide + +> **Purpose:** Comprehensive guide for setting up UX testing infrastructure across ByteLyst products using common platform UI packages, design tokens, and Cipher design system integration. + +> **Target Audience:** Product teams implementing E2E testing for their applications. + +--- + +## Overview + +This guide documents the complete UX testing infrastructure setup for the trading dashboard, which can be replicated across all ByteLyst products. The approach leverages: + +- **Playwright** for end-to-end testing +- **Common Platform UI packages** (`@bytelyst/ui`) +- **Design Tokens** from `@bytelyst/design-tokens` +- **Cipher Design System** principles +- **AI-friendly reporting** for automated analysis + +--- + +## Prerequisites + +### Common Platform Integration + +Ensure your product is integrated with the common platform: + +```bash +# Use local packages by default +BYTELYST_PACKAGE_SOURCE=common-plat + +# Install common platform packages +pnpm install @bytelyst/ui @bytelyst/design-tokens +``` + +### Design Tokens + +Design tokens should be consumed from the common platform: + +```typescript +// Use CSS variables from design tokens +background: var(--bl-surface-card); +color: var(--bl-text-primary); +border-color: var(--bl-border); +``` + +--- + +## Step 1: Playwright Setup + +### Install Dependencies + +```bash +cd web +pnpm add -D @playwright/test +pnpm exec playwright install chromium +``` + +### Create Playwright Config + +Create `web/playwright.config.ts`: + +```typescript +import { defineConfig, devices } from '@playwright/test'; + +export default defineConfig({ + testDir: './e2e', + fullyParallel: true, + forbidOnly: !!process.env.CI, + retries: process.env.CI ? 2 : 0, + workers: process.env.CI ? 1 : undefined, + reporter: 'html', + use: { + baseURL: 'http://localhost:3050', + trace: 'on-first-retry', + screenshot: 'only-on-failure', + }, + projects: [ + { + name: 'chromium', + use: { ...devices['Desktop Chrome'] }, + }, + { + name: 'firefox', + use: { ...devices['Desktop Firefox'] }, + }, + { + name: 'webkit', + use: { ...devices['Desktop Safari'] }, + }, + { + name: 'Mobile Chrome', + use: { ...devices['Pixel 5'] }, + }, + { + name: 'Mobile Safari', + use: { ...devices['iPhone 12'] }, + }, + { + name: 'Desktop Chrome HiDPI', + use: { ...devices['Desktop Chrome HiDPI'] }, + }, + ], + // webServer is disabled - tests should run against already-running server + // Test runner script handles server lifecycle +}); +``` + +### Add Test Scripts to package.json + +```json +{ + "scripts": { + "test:e2e": "playwright test", + "test:e2e:ui": "playwright test --ui", + "test:e2e:viewport": "playwright test e2e/viewport-matrix.spec.ts", + "test:e2e:overflow": "playwright test e2e/horizontal-overflow.spec.ts" + } +} +``` + +--- + +## Step 2: Create Test Suite Structure + +### Test Categories + +Create tests in `web/e2e/` directory following this structure: + +``` +web/e2e/ +├── viewport-matrix.spec.ts # Viewport compliance +├── horizontal-overflow.spec.ts # Overflow detection +├── alert-positioning.spec.ts # Critical alerts positioning +├── assistant-positioning.spec.ts # Assistant widget positioning +├── destructive-actions.spec.ts # Destructive actions confirmation +├── feedback.spec.ts # Save/delete/update feedback +├── page-states.spec.ts # Loading/empty/error/success states +├── form-validation.spec.ts # Form validation +└── keyboard-navigation.spec.ts # Keyboard navigation +``` + +--- + +## Step 3: Test Implementation Patterns + +### Viewport Matrix Test + +Tests that all routes pass viewport matrix for desktop, tablet, and mobile: + +```typescript +import { test, expect } from '@playwright/test'; + +const routes = ['/', '/portfolio', '/research', '/plans', '/markets']; +const viewports = [ + { name: 'Desktop', width: 1200, height: 800 }, + { name: 'Tablet', width: 768, height: 1024 }, + { name: 'Mobile', width: 375, height: 667 }, +]; + +routes.forEach((route) => { + viewports.forEach((viewport) => { + test(`${route} - ${viewport.name} viewport`, async ({ page }) => { + await page.setViewportSize({ width: viewport.width, height: viewport.height }); + await page.goto(route); + + // Check for horizontal overflow + const bodyWidth = await page.evaluate(() => document.body.scrollWidth); + expect(bodyWidth).toBeLessThanOrEqual(viewport.width + 10); + + // Check main content visibility + const mainContent = page.locator('main'); + await expect(mainContent).toBeVisible(); + }); + }); +}); +``` + +### Alert Positioning Test + +Tests that critical alerts don't cover primary content: + +```typescript +import { test, expect } from '@playwright/test'; + +test.describe('Critical Alerts Positioning', () => { + test('AlertBanner does not cover primary content', async ({ page }) => { + await page.goto('/'); + + // Find alert banners + const alerts = page.locator('[role="alert"], .alert, [class*="alert"]'); + const alertCount = await alerts.count(); + + if (alertCount > 0) { + const alert = alerts.first(); + const alertBox = await alert.boundingBox(); + const mainContent = page.locator('main'); + const mainBox = await mainContent.boundingBox(); + + // Alert should be inline, not covering main content + expect(alertBox).toBeTruthy(); + } + }); + + test('AlertBanner has proper z-index', async ({ page }) => { + await page.goto('/'); + const alerts = page.locator('[role="alert"]'); + + if (await alerts.count() > 0) { + const zIndex = await alerts.first().evaluate((el) => { + return window.getComputedStyle(el).zIndex; + }); + expect(parseInt(zIndex)).toBeLessThan(1000); + } + }); +}); +``` + +### Form Validation Test + +Tests that forms have labels, hints, validation, and disabled-state explanations: + +```typescript +import { test, expect } from '@playwright/test'; + +test.describe('Form Validation', () => { + test('Forms have labels for inputs', async ({ page }) => { + await page.goto('/plans'); + + const inputs = page.locator('input, select, textarea'); + const inputCount = await inputs.count(); + + if (inputCount > 0) { + for (let i = 0; i < Math.min(inputCount, 10); i++) { + const input = inputs.nth(i); + const isVisible = await input.isVisible(); + + if (isVisible) { + const id = await input.getAttribute('id'); + let hasLabel = false; + + if (id) { + const label = page.locator(`label[for="${id}"]`); + hasLabel = await label.count() > 0; + } + + const ariaLabel = await input.getAttribute('aria-label'); + const ariaLabelledby = await input.getAttribute('aria-labelledby'); + const hasAriaLabel = ariaLabel !== null || ariaLabelledby !== null; + + expect(hasLabel || hasAriaLabel).toBeTruthy(); + } + } + } + }); +}); +``` + +### Keyboard Navigation Test + +Tests that primary workflows pass keyboard navigation: + +```typescript +import { test, expect } from '@playwright/test'; + +test.describe('Keyboard Navigation', () => { + test('Tab order is logical', async ({ page }) => { + await page.goto('/'); + + const focusableElements = await page.evaluate(() => { + const focusable = ['button', '[href]', 'input', 'select', 'textarea', '[tabindex]:not([tabindex="-1"])']; + return Array.from(document.querySelectorAll(focusable.join(','))) + .filter(el => { + const style = window.getComputedStyle(el); + return style.display !== 'none' && style.visibility !== 'hidden'; + }) + .length; + }); + + expect(focusableElements).toBeGreaterThan(0); + }); + + test('Enter and Space activate buttons', async ({ page }) => { + await page.goto('/'); + + const button = page.locator('button').first(); + const buttonCount = await button.count(); + + if (buttonCount > 0) { + await button.focus(); + await page.keyboard.press('Space'); + await page.waitForTimeout(500); + + const isVisible = await button.isVisible(); + expect(isVisible).toBeTruthy(); + } + }); +}); +``` + +--- + +## Step 4: Test Runner Script + +### Create Test Runner Script + +Create `scripts/tests/run-e2e.sh` with server lifecycle management: + +```bash +#!/bin/bash +set -e + +# Colors for visual output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +PURPLE='\033[0;35m' +CYAN='\033[0;36m' +BOLD='\033[1m' +NC='\033[0m' + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)" +WEB_DIR="$PROJECT_ROOT/web" +REPORTS_DIR="$SCRIPT_DIR/reports" +TIMESTAMP=$(date +%Y%m%d_%H%M%S) +PORT=3050 + +cd "$WEB_DIR" + +# Function to kill any process on the specified port +kill_port() { + local port=$1 + local pid=$(lsof -ti:$port 2>/dev/null || echo "") + if [ -n "$pid" ]; then + echo -e "${YELLOW}Killing process $pid on port $port...${NC}" + kill -9 $pid 2>/dev/null || true + sleep 2 + fi +} + +# Function to wait for server to be ready +wait_for_server() { + local max_attempts=30 + local attempt=0 + echo -e "${CYAN}Waiting for server to be ready on port $PORT...${NC}" + + while [ $attempt -lt $max_attempts ]; do + if curl -s http://localhost:$PORT > /dev/null 2>&1; then + echo -e "${GREEN}✅ Server is ready${NC}" + return 0 + fi + attempt=$((attempt + 1)) + echo -e "${YELLOW}Attempt $attempt/$max_attempts...${NC}" + sleep 2 + done + + echo -e "${RED}❌ Server failed to start after $max_attempts attempts${NC}" + return 1 +} + +# Kill existing server +echo -e "${YELLOW}🔍 Checking for existing server on port $PORT...${NC}" +kill_port $PORT +echo -e "${GREEN}✓ Port $PORT is clear${NC}" + +# Start dev server +echo -e "${CYAN}🚀 Starting dev server...${NC}" +pnpm dev --port $PORT --strict-port > "$REPORTS_DIR/server-$TIMESTAMP.log" 2>&1 & +SERVER_PID=$! + +# Wait for server +if ! wait_for_server; then + echo -e "${RED}❌ Server failed to start${NC}" + kill_port $PORT + exit 1 +fi + +# Run tests +echo -e "${CYAN}🎭 Running E2E tests...${NC}" +pnpm exec playwright test \ + --reporter=json \ + --reporter=list \ + --output="$REPORTS_DIR/results-$TIMESTAMP.json" \ + 2>&1 | tee "$REPORTS_DIR/test-output-$TIMESTAMP.log" +TEST_EXIT_CODE=$? + +# Cleanup +echo -e "${CYAN}🧹 Cleaning up...${NC}" +kill $SERVER_PID 2>/dev/null || true +kill_port $PORT +echo -e "${GREEN}✓ Server stopped${NC}" + +exit $TEST_EXIT_CODE +``` + +### Make Script Executable + +```bash +chmod +x scripts/tests/run-e2e.sh +``` + +--- + +## Step 5: AI-Friendly Report Generation + +### Report Structure + +The test runner generates three types of reports: + +1. **JSON Report** (`test-report-{timestamp}.json`) - Full machine-readable data +2. **Markdown Report** (`test-report-{timestamp}.md`) - Human-readable summary +3. **AI Summary** (`summary-{timestamp}.json`) - Optimized for AI agents + +### AI Summary Format + +```json +{ + "timestamp": "20260509_135006", + "test_status": "passed", + "test_exit_code": 0, + "tests_total": 333, + "tests_passed": 333, + "tests_failed": 0, + "duration_seconds": 447, + "health_check": { + "status": "healthy", + "app_running": true, + "app_url": "http://localhost:3050" + }, + "for_ai_agents": { + "quick_summary": "passed - 333/333 tests passed in 447s", + "if_failed_check": "scripts/tests/reports/test-output-20260509_135006.log", + "html_report": "web/playwright-report/index.html", + "next_action": "Review HTML report for performance optimization" + } +} +``` + +### Report README + +Create `scripts/tests/reports/README.md`: + +```markdown +# E2E Test Reports + +This directory contains detailed E2E test reports. + +## For AI Agents + +Parse `latest-summary.json` for automated action: + +```bash +cat scripts/tests/reports/latest-summary.json | jq .for_ai_agents.next_action +``` + +## Running Tests + +```bash +./scripts/tests/run-e2e.sh +``` + +This will: +- Install dependencies if needed +- Kill existing server on port 3050 +- Start fresh dev server +- Run all E2E tests +- Generate reports +- Cleanup server +``` + +--- + +## Step 6: Common Platform UI Integration + +### Using Shared Components + +Import UI components from common platform: + +```typescript +import { AlertBanner, Button, Input, Select } from '../components/ui/Primitives'; +``` + +### Design Token Usage + +Use CSS variables from design tokens: + +```css +.my-component { + background: var(--bl-surface-card); + color: var(--bl-text-primary); + border-color: var(--bl-border); + padding: var(--bl-spacing-md); +} +``` + +### Product Adapter Pattern + +Create a product adapter to normalize imports: + +```typescript +// src/components/ui/Primitives.tsx +export { Button } from '@bytelyst/ui/Button'; +export { AlertBanner } from '@bytelyst/ui/AlertBanner'; +export { Input } from '@bytelyst/ui/Input'; +// ... other components +``` + +--- + +## Step 7: Cipher Design System Integration + +### Design Principles + +Follow Cipher design system principles: + +1. **Visual Hierarchy** - Clear hierarchy with size, weight, and color +2. **Spacing** - Consistent spacing using design tokens +3. **Typography** - Use design token font sizes and weights +4. **Color** - Use semantic color tokens (success, warning, error) +5. **Accessibility** - WCAG AA compliance with proper contrast ratios + +### Implementation Example + +```typescript +
+ + Content with proper visual hierarchy + +
+``` + +--- + +## Step 8: Roadmap Documentation + +### Update Launch Readiness Checklist + +Document progress in `docs/inprogress/LAUNCH_READY_UI_UX_ROADMAP.md`: + +```markdown +## Launch Readiness Checklist + +- [x] No production route has accidental horizontal overflow +- [x] Critical alerts never cover primary content +- [x] Assistant widget never covers primary actions +- [x] All destructive actions require confirmation +- [x] All saves/deletes/updates produce feedback +- [x] All pages have loading/empty/error/success states +- [x] All forms have labels, hints, validation, and disabled-state explanations +- [x] All primary workflows pass keyboard navigation +- [x] All routes pass the viewport matrix +- [x] Playwright screenshots and accessibility checks run in CI +``` + +--- + +## Step 9: Verification Commands + +### Run All Tests + +```bash +./scripts/tests/run-e2e.sh +``` + +### Run Specific Test Suites + +```bash +cd web +pnpm test:e2e:viewport +pnpm test:e2e:overflow +``` + +### Check AI Summary + +```bash +cat scripts/tests/reports/latest-summary.json | jq .for_ai_agents +``` + +--- + +## Step 10: CI Integration (Optional) + +### GitHub Actions Example + +```yaml +- name: Run E2E Tests + run: ./scripts/tests/run-e2e.sh + +- name: Upload Reports + uses: actions/upload-artifact@v3 + with: + name: e2e-test-reports + path: scripts/tests/reports/ +``` + +--- + +## Troubleshooting + +### Server Won't Start + +**Issue:** Port 3050 already in use + +**Solution:** +```bash +lsof -ti:3050 | xargs kill -9 +``` + +### Missing Dependencies + +**Issue:** Playwright or dependencies not installed + +**Solution:** +```bash +pnpm install +pnpm exec playwright install chromium +``` + +### Tests Fail on VM + +**Issue:** Missing dependencies on VM + +**Solution:** +```bash +cd web +pnpm install +``` + +--- + +## Best Practices + +1. **Test Organization** - Group related tests in describe blocks +2. **Selectors** - Use stable selectors (data-test attributes when needed) +3. **Wait Strategies** - Use Playwright's auto-waiting features +4. **Screenshots** - Capture screenshots on failure for debugging +5. **Reports** - Review HTML reports for detailed timing and coverage +6. **Accessibility** - Include accessibility checks in tests +7. **Maintenance** - Update tests when UI changes + +--- + +## References + +- **Playwright Documentation:** https://playwright.dev +- **Common Platform UI:** `@bytelyst/ui` package +- **Design Tokens:** `@bytelyst/design-tokens` package +- **Cipher Design System:** Internal design system documentation +- **Launch Readiness Roadmap:** `docs/inprogress/LAUNCH_READY_UI_UX_ROADMAP.md` + +--- + +## Summary + +This guide provides a complete blueprint for setting up UX testing infrastructure across ByteLyst products. By following these steps, products can: + +1. Establish comprehensive E2E testing with Playwright +2. Leverage common platform UI packages and design tokens +3. Follow Cipher design system principles +4. Generate AI-friendly reports for automated analysis +5. Ensure consistent UX quality across all products + +The approach prioritizes: +- **Automation** - Server lifecycle management in test runner +- **Visibility** - Rich, colorful console output +- **Actionability** - AI-friendly reports with next actions +- **Reusability** - Common platform integration +- **Maintainability** - Clear test organization and documentation