5.1 KiB
5.1 KiB
Phase 8 Enterprise Validation Summary
1. Completion Confirmation for Phases 1–7
Each phase below has been validated, the enforcement mechanisms are operational, and failure modes have been documented for operators.
Phase 1 — Tenant Isolation
- Validation criteria: WebSocket broadcasts partitioned by authenticated user, Supabase RLS policies on orders and trade_history, and filtered runtime states tied to user_id.
- Evidence: RLS policies, tenant metrics, and failure scenarios covering cross-tenant attempts documented in runbooks.
Phase 2 — Restart Durability
- Validation criteria: Startup routine reloads active profiles, exchange open orders/positions, and rebuilds lifecycle/capital maps deterministically.
- Evidence: Log traces of profile reload, ledger rebuild metrics, and restart-recovery runbook coverage.
Phase 3 — Capital Ledger
- Validation criteria: Ledger invariant
allocated - reserved_for_orders - reserved_for_positions + realized_pnl >= 0holds, and ledger rebuilds on restart while supporting entry, fill, cancel, and exit adjustments. - Evidence: Ledger RPCs, capital-invariant metrics, reconciliation ledger sync, and capital-invariant runbook handling violations.
Phase 4 — Transactional Lifecycle
- Validation criteria: ENTRY/EXIT persistence occurs inside single DB transactions with idempotent constraints, ensuring lifecycle data only appears after exchange confirmation.
- Evidence: fn_persist_entry_lifecycle, UNIQUE constraints, lifecycle metrics, and lifecycle-incident runbook.
Phase 5 — Reconciliation
- Validation criteria: Loop compares full DB/exchange sets, obtains profile locks, routes discrepancies through lifecycle handlers, and resets mismatch metrics automatically.
- Evidence: reconciliation lock table, mismatch metrics, reconciliation runbook, and failure scenarios for missing/excess orders.
Phase 6 — Distributed Safety
- Validation criteria: Row-based entry locks with TTL and owner tokens combined with deterministic clientOrderId prevent duplicate entries and survive multi-instance deployments.
- Evidence: entry_locks table, lock metrics (contention and latency), and lock-contention runbook.
Phase 7 — Observability & Health
- Validation criteria: /metrics and /internal/health track trading, monitor, reconciliation loops alongside lock, capital, and exchange health, permitting rapid detection of failures.
- Evidence: health endpoint fields, Prometheus metrics, structured logs, and loop-health runbook.
2. Final Checklist Mapping
| Requirement | Enforcement | Detection | Recovery |
|---|---|---|---|
| Tenant isolation | Supabase RLS, WebSocket filters | RLS audit logs, tenant breach alerts | Reject broadcast, block execution, notify compliance |
| Capital invariant | Ledger RPCs + watcher | capitalInvariantViolations, gauge dips |
Reconciliation rebuild, trading loop pause for profile |
| Lifecycle atomicity | Entry/exit RPC transactions + UNIQUE constraints | DB transaction failures, reconciliation mismatch | RPC retry, lifecycle handler repair |
| Exchange truth | Exchange-first ordering before persistence | Reconciliation mismatch count, missing metrics | Lifecycle handler corrections |
| Entry exclusivity | Row-based locks + deterministic clientOrderId | lockContentionCount, lock TTL expiration |
Wait TTL, reattempt, escalate via lock-contention runbook |
| Reconciliation parity | Lock + handler routing | reconciliationMismatchCount, missing metrics |
Handler-guided correction, metrics reset |
| Observability coverage | Prometheus counters + health endpoint | /internal/health flags, metric spikes |
Alerts trigger runbooks, loops restart if needed |
3. System Guarantees
- Capital never goes negative for active profiles thanks to ledger invariant enforcement and capital watchdog metrics.
- Only one ENTRY per profile/symbol proceeds at a time due to distributed row locks and deterministic clientOrderId.
- Lifecycle persistence is atomic, always triggered after confirmed exchange events, and idempotent on retries.
- Database state converges to exchange truth through reconciliation and structured health metrics, providing eventual consistency.
- Observability ensures loops, locks, and capital invariants are monitored and documented for operators.
4. Explicit Non-Guarantees
- Does not guarantee immediate dashboard synchronization; reconciliation is eventual.
- Does not guarantee tolerance for prolonged Supabase outages without ops intervention—trading pauses until Supabase is writable.
- Does not permit ad-hoc manual ledger or lifecycle edits; such edits require formal runbook approval.
5. Future Change Safety
- Any future agent must review docs/invariants.md and these runbooks before touching lifecycle RPCs, ledger services, reconciliation logic, or locking mechanisms.
- Changes must preserve the phase-by-phase guarantees and maintain all health/metrics fields referenced herein.
- Operators should verify that metrics and health indicators remain functional after updates, updating runbooks if detection/recovery steps change.