# Durable Event Bus & Worker Runtime Roadmap

> **Purpose:** Replace in-process eventing and scattered background execution
> with a durable cross-service event and worker backbone.
>
> **Primary Surfaces:** `packages/events/`, `packages/queue/`, `services/platform-service/`
>
> **Status:** Planned
>
> **Estimated Effort:** 3-4 weeks

---

## Why This Is Missing

`packages/events/src/memory.ts` is an in-process event bus. That is useful for local
dispatch inside one process, but it is not enough for:

- cross-service subscriptions
- replay
- dead-letter handling
- durable delivery
- delayed fan-out

The new `@bytelyst/queue` package improves durable background work, but the eventing
layer is still incomplete.

---

## Recommended Stack

### Best Long-Term Industry Standard

- **Redis Streams** or **NATS JetStream** for durable event delivery
- `@bytelyst/queue` or BullMQ for work execution
- OpenTelemetry for trace correlation

### Repo-Fit Option

- Add a durable adapter to `@bytelyst/events`
- Use Redis-backed delivery first
- Keep current memory bus as test/dev adapter

### Recommendation

Use `@bytelyst/events` as the interface, but add a durable Redis or NATS adapter.
Do not let direct in-memory emitters remain the production default for critical flows.

### Default Decision For This Repo

- Keep `@bytelyst/events` as the public interface
- Add Redis-backed durability first
- Keep memory adapters for tests only
- Standardize worker bootstrap across services before adding more background features

---

## Dependencies

- None for the core substrate
- Required by Agent Runtime, Delivery side effects, and future SCIM sync jobs

---

## V1 Non-Goals

- Kafka-scale event streaming
- Exactly-once delivery guarantees
- Cross-region event replication

---

## Phase 1 - Event Abstraction

- [ ] Extend `@bytelyst/events` to support pluggable backends
- [ ] Keep `memory` for tests
- [ ] Add `redis-streams` or `jetstream` adapter
- [ ] Add consumer groups, ack, retry, and dead-letter support
- [ ] Add correlation and causation IDs

---

## Phase 2 - Worker Runtime

- [ ] Standardize worker bootstrap pattern
- [ ] Add handler registration, concurrency controls, leases, and health endpoints
- [ ] Add poison-message and dead-letter inspection
- [ ] Add scheduling and delayed dispatch

---

## Phase 3 - Service Migration

- [ ] Move delivery subscribers onto durable events
- [ ] Move auth side effects off fire-and-forget local emitters
- [ ] Move MCP/A2A transitions onto durable events where appropriate
- [ ] Add observability for event lag and failure rate

---

## Tech Stack Options

| Option          | Pros                                            | Cons                           | Fit                 |
| --------------- | ----------------------------------------------- | ------------------------------ | ------------------- |
| NATS JetStream  | Strong event semantics, lightweight             | New infra and integration work | Excellent long-term |
| Redis Streams   | Familiar, easy to adopt with BullMQ-style stack | Less specialized than NATS     | Best pragmatic path |
| Kafka           | Powerful at scale                               | Heavy operational footprint    | Overkill now        |
| Memory bus only | Simple                                          | Not durable                    | Dev/test only       |

---

## Risks

- In-process events hide failures and block cross-service reliability
- Durable queues without durable events still leave side effects fragile
- Multiple custom worker patterns will drift without a standard runtime
