bytelyst/learning_ai_invt_trdg
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(chat): add capability audit

Browse Source
This commit is contained in:
root 2026-05-07 06:48:29 +00:00
parent 69324be784
commit dfe876d49b
1 changed files with 293 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

293
docs/CHAT_BOT_CAPABILITY_AUDIT.md Normal file
View File

@ -0,0 +1,293 @@
# Chat / Bot Capability Audit
Reviewed on `2026-05-07` against the current `main` branch.
This audit is separate from the completed UI migration roadmap. It assesses the functional maturity of:
- trading bot/runtime
- realtime socket integration
- Trade Plans automation
- reconciliation / operational tooling
- chat / AI assistant capabilities
## Executive Summary
The app has real bot and chat functionality today. It is not a mock product.
Current maturity estimate:
| Subsystem | Completion | Notes |
|---|---:|---|
| Trading bot/runtime | `80-85%` | Real execution, state recovery, order lifecycle, and user-scoped broker credentials exist. |
| Realtime web integration | `85-90%` | Socket-driven state and health flows are implemented and tested. |
| Trade Plans automation | `85%` | Buy/exit automation, portfolio linkage, lifecycle toasts, and setup history are in place. |
| Reconciliation / ops | `75-80%` | Strong operational depth exists, but manual-review and quarantine paths are still prominent. |
| Chat / AI assistant | `60-70%` | Real assistant exists, but it is still mostly profile-generation/config assistance, not a full trading copilot. |
| Overall product maturity | `75-80%` | The product is functional and substantial, but not yet “quiet”, fully unified, or deeply assistant-driven. |
## What Exists Today
### 1. Trading runtime is real
Evidence:
- [backend/src/index.ts](/opt/bytelyst/trading/learning_ai_invt_trdg/backend/src/index.ts)
- [backend/src/services/TradeExecutor.ts](/opt/bytelyst/trading/learning_ai_invt_trdg/backend/src/services/TradeExecutor.ts)
- [backend/src/services/apiServer.ts](/opt/bytelyst/trading/learning_ai_invt_trdg/backend/src/services/apiServer.ts)
Implemented behavior includes:
- authenticated trading APIs
- live trading loop
- manual trader/profile routing
- order and position lifecycle tracking
- user-scoped Alpaca credential usage
- health tracking and loop telemetry
Conclusion:
- this is not a toy bot shell
- execution/runtime infrastructure is one of the stronger parts of the app
### 2. Realtime web state is real
Evidence:
- [web/src/hooks/useWebSocket.ts](/opt/bytelyst/trading/learning_ai_invt_trdg/web/src/hooks/useWebSocket.ts)
- [web/src/hooks/useWebSocket.dom.test.tsx](/opt/bytelyst/trading/learning_ai_invt_trdg/web/src/hooks/useWebSocket.dom.test.tsx)
Implemented behavior includes:
- authenticated socket connection
- runtime symbol/order/history/alert updates
- health and reconciliation metrics in app state
- tested event application behavior
Conclusion:
- realtime state propagation is substantially implemented
- this is a strong, working foundation for trading/ops UX
### 3. Trade Plans are a real workflow
Evidence:
- [web/src/views/SimpleView.tsx](/opt/bytelyst/trading/learning_ai_invt_trdg/web/src/views/SimpleView.tsx)
- [web/src/views/SimpleView.test.ts](/opt/bytelyst/trading/learning_ai_invt_trdg/web/src/views/SimpleView.test.ts)
- [backend/src/index.ts](/opt/bytelyst/trading/learning_ai_invt_trdg/backend/src/index.ts)
Implemented behavior includes:
- short-term buy plans
- profit-exit attachment to existing holdings
- concurrent same-symbol setup support
- long-term conversion / resume exit management
- portfolio linkage
- setup-level runtime events
- setup activity history and timeline
Conclusion:
- Trade Plans are feature-complete enough for real use
- remaining work is mostly refinement, not existence
### 4. Reconciliation and ops tooling are deep
Evidence:
- [backend/src/services/reconciliationService.ts](/opt/bytelyst/trading/learning_ai_invt_trdg/backend/src/services/reconciliationService.ts)
- [backend/src/services/healthTracker.ts](/opt/bytelyst/trading/learning_ai_invt_trdg/backend/src/services/healthTracker.ts)
- [web/src/tabs/ReconciliationAuditPanel.tsx](/opt/bytelyst/trading/learning_ai_invt_trdg/web/src/tabs/ReconciliationAuditPanel.tsx)
- [web/src/tabs/AdminTab.tsx](/opt/bytelyst/trading/learning_ai_invt_trdg/web/src/tabs/AdminTab.tsx)
Implemented behavior includes:
- reconciliation cycles
- parity / backfill / coverage / repair services
- health metrics and watchdog signals
- admin reconciliation audit UI
- operational event surfaces
Conclusion:
- the platform has real operational depth
- the remaining issue is not missing machinery; it is reducing how often users/admins need it
### 5. Chat assistant is real, but narrower
Evidence:
- [web/src/components/ChatControl.tsx](/opt/bytelyst/trading/learning_ai_invt_trdg/web/src/components/ChatControl.tsx)
- [web/src/components/ChatControl.dom.test.tsx](/opt/bytelyst/trading/learning_ai_invt_trdg/web/src/components/ChatControl.dom.test.tsx)
- [backend/src/services/apiServer.ts](/opt/bytelyst/trading/learning_ai_invt_trdg/backend/src/services/apiServer.ts:2626)
- [backend/src/services/aiClient.ts](/opt/bytelyst/trading/learning_ai_invt_trdg/backend/src/services/aiClient.ts)
Implemented behavior includes:
- authenticated `/api/chat`
- LLM provider fallback chain
- deterministic local fallback if AI output fails
- profile create/update/explain intent
- draft editing and apply flow in the UI
- quick actions for common profile prompts
Conclusion:
- this is a real assistant workflow
- but it is still mostly “profile configuration assistant”, not a broad product copilot
## Major Gaps
### 1. Chat is not yet a full portfolio / execution / reconciliation copilot
Current state:
- chat can generate or modify profiles
- chat is not yet deeply integrated into:
- portfolio explanation
- trade-fire reasoning
- exit-block diagnosis
- reconciliation guidance
- post-trade review
Impact:
- the product has strong underlying runtime data, but the assistant does not yet exploit it fully
Priority:
- highest
### 2. Explanation surfaces are still shallow
Evidence:
- [web/src/lib/StrategyExplanationService.ts](/opt/bytelyst/trading/learning_ai_invt_trdg/web/src/lib/StrategyExplanationService.ts)
Current state:
- explanation logic is heuristic and limited
- comments in code explicitly note simplified behavior
Impact:
- users can see state, but not always the most useful “why” behind it
Priority:
- high
### 3. Reconciliation maturity is strong, but still noisy
Current state:
- there are many reconciliation and quarantine paths
- this is good operationally, but it usually means the user/admin experience is still noisy
Impact:
- reliability is protected
- clarity and quietness can still improve
Priority:
- high
### 4. Error surfaces are still uneven
Examples:
- user-facing flows still rely on thrown messages in several places
- some UI/runtime areas still log directly with `console.error`
Impact:
- technically functional
- not fully product-polished
Priority:
- medium
### 5. Chat lacks deeper action verbs
Current state:
- assistant handles `create_profile`, `update_profile`, `explain`
- it does not clearly own richer actions like:
- explain current position
- explain why no trade fired
- inspect blocked exit
- summarize reconciliation anomalies
- propose fix without leaving chat
Impact:
- user expectation of “AI trading assistant” is only partially met
Priority:
- highest
## Recommended Next Implementation Order
### Phase A: Make chat a real copilot
1. Add assistant intents for:
- explain current holding
- explain why a symbol/profile did not enter
- explain why an exit is blocked
- summarize reconciliation issues
- suggest safe next actions
2. Feed live runtime context into `/api/chat`:
- holdings
- active profiles
- recent orders
- recent operational events
- reconciliation health summary
3. Return structured action types, not only free text:
- `explain_position`
- `explain_blocker`
- `summarize_reconciliation`
- `recommend_profile_change`
### Phase B: Improve explanation quality outside chat
1. Replace heuristic-only explanation paths with lifecycle-aware reasoning
2. Reuse the same explanation contracts in:
- Portfolio
- Trade Plans
- Overview
- Chat
### Phase C: Quiet the ops layer
1. Reduce quarantine/manual-review frequency
2. Improve user-facing messages for blocked trades/exits
3. Add clearer reconciliation state summaries for admins and end users
### Phase D: Polish error UX
1. unify user-safe error formatting
2. reduce raw thrown-message exposure
3. reduce console-noise paths in stable workflows
## Recommended Success Criteria
The chat/bot experience should be considered “substantially complete” when:
- chat can explain active positions and blocked actions using live runtime context
- chat can explain “why nothing happened” for a profile or symbol
- Trade Plans, Portfolio, and Chat share consistent lifecycle reasoning
- reconciliation remains available but becomes rarer and quieter
- user-facing errors are product-grade instead of raw internal messages
## Bottom Line
The app already has real bot functionality and a real assistant.
The biggest remaining gap is not “build chat from scratch.” It is:
- turning the current assistant from a profile generator into a true trading copilot
That is the highest-value next product move because the runtime, socket state, Trade Plans, and reconciliation foundations are already in place.