60 lines
5.4 KiB
Markdown
60 lines
5.4 KiB
Markdown
|
|
Bytelyst Trading Platform Invariants
|
|
|
|
## Capital Ledger Invariant
|
|
- **Invariant name:** Capital non-negativity
|
|
- **Description:** For every profile, allocated - reserved_for_orders - reserved_for_positions + realized_pnl >= 0 at all times.
|
|
- **Enforced by:** Capital ledger RPCs, ledger schema constraints, and reconciliation handlers that re-sync reservations.
|
|
- **Detection:** Capital invariant metric increments and health endpoint reports violation when ledger calculation dips below zero; observability logs critical entries.
|
|
- **Recovery:** Capital watchdog logs the violation, trading loop halts new entries for that profile, reconciliation replays state, and operators follow docs/runbooks/invariant-violation.md.
|
|
|
|
## Single ENTRY Lock Invariant
|
|
- **Invariant name:** One ENTRY per profile/symbol
|
|
- **Description:** At most one ENTRY signal can progress per (profile_id, symbol) at a time, preventing double submissions.
|
|
- **Enforced by:** Row-based entry_locks and deterministic clientOrderId, with lock TTL enforcement in the lock service code.
|
|
- **Detection:** Lock contention counter increments and health endpoint flags stale locks when TTL expires without release.
|
|
- **Recovery:** Contention metrics surface to Prometheus; the losing worker retries after TTL and operators consult docs/runbooks/lock-timeout.md if contention persists.
|
|
|
|
## Duplicate Exchange Order Invariant
|
|
- **Invariant name:** No duplicate exchange order
|
|
- **Description:** A single trade_id can only produce one exchange order; retries never issue another order to the exchange.
|
|
- **Enforced by:** Deterministic clientOrderId strategy and transactional lifecycle RPCs that guard on matching trade_id/order_id.
|
|
- **Detection:** Exchange connectors translate duplicate-order errors into existing lifecycle lookups; observability logs capture the repeat attempt.
|
|
- **Recovery:** Retried persistence fetches existing order_id; reconciliation ensures DB state matches exchange; operators are guided by docs/runbooks/reconciliation.md if duplicates surface.
|
|
|
|
## Lifecycle Atomicity Invariant
|
|
- **Invariant name:** Lifecycle persistence atomicity
|
|
- **Description:** ENTRY and EXIT lifecycle rows, order rows, positions, and history slices are inserted or rolled back as a single atomic operation.
|
|
- **Enforced by:** Supabase fn_persist_entry_lifecycle and exit RPC transactions with UNIQUE constraints (trade_lifecycle(profile_id, trade_id), orders(order_id)).
|
|
- **Detection:** DB transaction failure logs surface and rollback leaves no partial data; reconciliation detects orphan orders or missing lifecycle slices.
|
|
- **Recovery:** Retry replays through RPCs; reconciliation invokes lifecycle handlers to rebuild missing artifacts; runbooks refer to docs/runbooks/lifecycle-incident.md.
|
|
|
|
## Exchange-as-Source-of-Truth Invariant
|
|
- **Invariant name:** Exchange truth first
|
|
- **Description:** No lifecycle row exists or is updated until the exchange confirms the order or fill status.
|
|
- **Enforced by:** Entry/exit flows that place exchange orders before calling persistence RPCs plus reconciliation that corrects DB drift.
|
|
- **Detection:** Reconciliation mismatch counters increment when DB differs from exchange; health endpoint flags missing trades.
|
|
- **Recovery:** Lifecycle handlers repair DB state based on exchange data; trading loop refrains from acting on stale state; ops follow docs/runbooks/reconciliation.md.
|
|
|
|
## Idempotent Retry Invariant
|
|
- **Invariant name:** Idempotent lifecycle retries
|
|
- **Description:** Retrying the same lifecycle persistence request never creates duplicates nor mutates unintended rows.
|
|
- **Enforced by:** RPC idempotency keys, UNIQUE(profile_id, trade_id), ON CONFLICT DO NOTHING on child inserts, and deterministic clientOrderId.
|
|
- **Detection:** RPC responses include existing lifecycle references; duplicate insertion attempts are logged without causing errors.
|
|
- **Recovery:** Retry safely returns the existing lifecycle; metrics note idempotency hits, and operators refer to docs/runbooks/lifecycle-incident.md only if insert counts grow unusually.
|
|
|
|
## Lock TTL Safety Invariant
|
|
- **Invariant name:** Lock TTL safety
|
|
- **Description:** Distributed locks expire automatically if a worker crashes, preventing deadlocks.
|
|
- **Enforced by:** Lock acquisition RPCs that set expires_at = now() + TTL and lock release RPCs requiring owner authentication.
|
|
- **Detection:** Lock contention metrics, TTL expiration audit logs, and /internal/health lockContentionCount track stuck entries.
|
|
- **Recovery:** TTL expiry frees the lock automatically, reconciliation/self-healing loops resume; operators only intervene when contention spikes, per docs/runbooks/lock-timeout.md.
|
|
|
|
## Invariants Enforcement Summary
|
|
- DB-enforced invariants: lifecycle atomicity (UNIQUE constraints), capital ledger constraints, lock row TTL persistence.
|
|
- Code-enforced invariants: entry lock acquisition, deterministic clientOrderId, reconciliation routing through lifecycle handlers.
|
|
- Observability-enforced invariants: capital invariant metric, lock contention counters, reconciliation mismatch counts that trigger alerts when thresholds are crossed.
|
|
|
|
## Rules future agents MUST NOT break
|
|
Future agents must never touch lifecycle RPCs, ledger services, reconciliation logic, or locking mechanisms without confirming that the invariants listed above remain intact. Breaking capital, lock, lifecycle, or exchange-truth invariants without operator consent is forbidden.
|