learning_ai_invt_trdg/docs/CHAT_BOT_CAPABILITY_AUDIT.md

Chat / Bot Capability Audit

Reviewed on 2026-05-07 against the current main branch.

This audit is separate from the completed UI migration roadmap. It assesses the functional maturity of:

• trading bot/runtime
• realtime socket integration
• Trade Plans automation
• reconciliation / operational tooling
• chat / AI assistant capabilities

Executive Summary

The app has real bot and chat functionality today. It is not a mock product.

Current maturity estimate:

Subsystem	Completion	Notes
Trading bot/runtime	80-85%	Real execution, state recovery, order lifecycle, and user-scoped broker credentials exist.
Realtime web integration	85-90%	Socket-driven state and health flows are implemented and tested.
Trade Plans automation	85%	Buy/exit automation, portfolio linkage, lifecycle toasts, and setup history are in place.
Reconciliation / ops	75-80%	Strong operational depth exists, but manual-review and quarantine paths are still prominent.
Chat / AI assistant	80-85%	Real assistant now handles profile generation, runtime explanations, guided next actions, route-aware quick links, trade-plan recommendations, reconciliation follow-up, and recent-trade review, but it is not yet a full end-to-end corrective automation copilot.
Overall product maturity	75-80%	The product is functional and substantial, but not yet “quiet”, fully unified, or deeply assistant-driven.

What Exists Today

1. Trading runtime is real

Evidence:

• backend/src/index.ts
• backend/src/services/TradeExecutor.ts
• backend/src/services/apiServer.ts

Implemented behavior includes:

• authenticated trading APIs
• live trading loop
• manual trader/profile routing
• order and position lifecycle tracking
• user-scoped Alpaca credential usage
• health tracking and loop telemetry

Conclusion:

• this is not a toy bot shell
• execution/runtime infrastructure is one of the stronger parts of the app

2. Realtime web state is real

Evidence:

• web/src/hooks/useWebSocket.ts
• web/src/hooks/useWebSocket.dom.test.tsx

Implemented behavior includes:

• authenticated socket connection
• runtime symbol/order/history/alert updates
• health and reconciliation metrics in app state
• tested event application behavior

Conclusion:

• realtime state propagation is substantially implemented
• this is a strong, working foundation for trading/ops UX

3. Trade Plans are a real workflow

Evidence:

• web/src/views/SimpleView.tsx
• web/src/views/SimpleView.test.ts
• backend/src/index.ts

Implemented behavior includes:

• short-term buy plans
• profit-exit attachment to existing holdings
• concurrent same-symbol setup support
• long-term conversion / resume exit management
• portfolio linkage
• setup-level runtime events
• setup activity history and timeline

Conclusion:

• Trade Plans are feature-complete enough for real use
• remaining work is mostly refinement, not existence

4. Reconciliation and ops tooling are deep

Evidence:

• backend/src/services/reconciliationService.ts
• backend/src/services/healthTracker.ts
• web/src/tabs/ReconciliationAuditPanel.tsx
• web/src/tabs/AdminTab.tsx

Implemented behavior includes:

• reconciliation cycles
• parity / backfill / coverage / repair services
• health metrics and watchdog signals
• admin reconciliation audit UI
• operational event surfaces

Conclusion:

• the platform has real operational depth
• the remaining issue is not missing machinery; it is reducing how often users/admins need it

5. Chat assistant is real, but narrower

Evidence:

• web/src/components/ChatControl.tsx
• web/src/components/ChatControl.dom.test.tsx
• backend/src/services/apiServer.ts
• backend/src/services/aiClient.ts

Implemented behavior includes:

• authenticated /api/chat
• LLM provider fallback chain
• deterministic local fallback if AI output fails
• profile create/update/explain intent
• live copilot explanation intents for:
  • current holding
  • waiting/no-trade-yet signal state
  • main blocker
  • reconciliation summary
• guided nextActions in assistant responses
• route-aware quickLinks into:
  • Portfolio
  • Trade Plans
  • Settings/Admin Panel
• trade-plan recommendation intent for live holdings
• reconciliation follow-up recommendation intent
• recent-trade review intent for closed trade performance
• profile-tuning recommendation flow with a reusable apply path
• draft editing and apply flow in the UI
• quick actions for common profile prompts

Conclusion:

• this is now a real runtime copilot, not only a profile generator
• the remaining gap is deeper corrective/remediation automation and richer shared reasoning

Major Gaps

1. Chat is not yet a full corrective execution / reconciliation copilot

Current state:

• chat can generate or modify profiles
• chat can now explain:
  • a live holding
  • why a trade is waiting
  • the main blocker
  • reconciliation state
• chat can now recommend:
  • concrete safe next actions
  • route-aware in-app follow-up destinations
  • a profile change draft when tuning is the most likely next step
  • a Trade Plan follow-up for a live holding
  • a reconciliation follow-up path
  • a recent-trade review summary
• chat is not yet deeply integrated into:
  • direct Trade Plan creation/editing mutations from assistant reasoning
  • reconciliation repair actions beyond navigation/guidance
  • deeper post-trade journaling / coaching
  • unified explanation logic shared across all product surfaces

Impact:

• the product has strong underlying runtime data, but the assistant does not yet exploit it fully

Priority:

• highest, but substantially improved from the previous audit

2. Explanation surfaces are still shallow

Evidence:

• web/src/lib/StrategyExplanationService.ts

Current state:

• explanation logic is heuristic and limited
• comments in code explicitly note simplified behavior

Impact:

• users can see state, but not always the most useful “why” behind it

Priority:

• high

3. Reconciliation maturity is strong, but still noisy

Current state:

• there are many reconciliation and quarantine paths
• this is good operationally, but it usually means the user/admin experience is still noisy

Impact:

• reliability is protected
• clarity and quietness can still improve

Priority:

• high

4. Error surfaces are still uneven

Examples:

• user-facing flows still rely on thrown messages in several places
• some UI/runtime areas still log directly with console.error

Impact:

• technically functional
• not fully product-polished

Priority:

• medium

5. Chat lacks deeper action verbs

Current state:

• assistant handles create_profile, update_profile, explain
• it does not clearly own richer actions like:
  • explain current position
  • explain why no trade fired
  • inspect blocked exit
  • summarize reconciliation anomalies
  • propose fix without leaving chat

Impact:

• user expectation of “AI trading assistant” is only partially met

Priority:

• highest

Recommended Next Implementation Order

Phase A: Make chat a real copilot

Status: partially complete

1. Add assistant intents for:
   • explain current holding
   • explain why a symbol/profile did not enter
   • explain why an exit is blocked
   • summarize reconciliation issues
   • suggest safe next actions
2. Feed live runtime context into /api/chat:
   • holdings
   • recent orders/history
   • order failures
   • recent operational events
   • reconciliation health summary
   • profile signal/execution context
   • deeper canonical lifecycle / trade-plan linkage
3. Return structured action types, not only free text:
   • explain_position
   • explain_waiting
   • explain_blocker
   • summarize_reconciliation
   • recommend_profile_change
   • corrective action flows beyond profile updates

Phase B: Improve explanation quality outside chat

1. Replace heuristic-only explanation paths with lifecycle-aware reasoning
2. Reuse the same explanation contracts in:
   • Portfolio
   • Trade Plans
   • Overview
   • Chat

Phase C: Quiet the ops layer

1. Reduce quarantine/manual-review frequency
2. Improve user-facing messages for blocked trades/exits
3. Add clearer reconciliation state summaries for admins and end users

Phase D: Polish error UX

1. unify user-safe error formatting
2. reduce raw thrown-message exposure
3. reduce console-noise paths in stable workflows

Recommended Success Criteria

The chat/bot experience should be considered “substantially complete” when:

• chat can explain active positions and blocked actions using live runtime context
• chat can explain “why nothing happened” for a profile or symbol
• Trade Plans, Portfolio, and Chat share consistent lifecycle reasoning
• reconciliation remains available but becomes rarer and quieter
• user-facing errors are product-grade instead of raw internal messages

Bottom Line

The app already has real bot functionality and a real assistant.

The biggest remaining gap is not “build chat from scratch.” It is:

• turning the current assistant from a profile generator into a true trading copilot

That is the highest-value next product move because the runtime, socket state, Trade Plans, and reconciliation foundations are already in place.