learning_ai_common_plat/docs/ecosystem/ECOSYSTEM_CONTRACT_TEST_STRATEGY.md

Ecosystem Contract Test Strategy

Status: Draft Owner: learning_ai_common_plat Purpose: Define how shared artifact, runtime, approval, and event contracts are tested across repos without relying on manual interpretation.

---

1. Problem

Shared docs are not enough.

If multiple repos implement the same contract differently, the ecosystem will drift even if each repo passes its own local tests.

We need contract tests that verify:

• producers emit the expected shape
• consumers accept the expected shape
• changes to shared contracts are intentional and visible

---

2. Strategy

Use a three-part model:

1. canonical fixtures
   • shared JSON examples owned in learning_ai_common_plat
2. producer conformance tests
   • repo-specific tests that emit contract-shaped outputs
3. consumer compatibility tests
   • repo-specific tests that ingest canonical fixtures

---

3. Initial Contract Test Targets

Artifact contract

• transcript artifact fixture
• note artifact fixture
• memory artifact fixture
• agent-output artifact fixture

Runtime contract

• session fixture
• task fixture
• todo fixture
• dispatch fixture
• approval checkpoint fixture

Approval contract

• approval request fixture
• approval decision fixture
• degraded-device policy case

Event contract

• transcript created event
• artifact linked event
• plan generated event
• agent run completed event

---

4. Recommended Location

Canonical fixtures should eventually live in:

• learning_ai_common_plat/docs/ecosystem/fixtures/ for human-readable examples
• learning_ai_common_plat/packages/... or services/... for executable validation once implementation begins

---

5. Required Repo Behavior

Every consuming or producing repo should:

1. import or mirror the canonical fixture set
2. validate against the canonical shape
3. fail clearly when contract drift occurs
4. reference the shared contract doc version in tests or fixtures

---

6. Current Contract Test Scope

• artifact schema fixture validation
• event envelope fixture validation
• runtime session and task fixture validation
• approval record fixture validation
• one producer and one consumer test for each of the first three golden flows

Current executable anchors:

• learning_ai_common_plat/packages/events/src/ecosystem.test.ts
• learning_ai_common_plat/packages/events/src/agent-runtime.test.ts
• learning_ai_common_plat/services/platform-service/src/modules/audit/routes.test.ts
• learning_ai_common_plat/services/platform-service/src/modules/agent-runtime/routes.test.ts
• learning_ai_common_plat/services/cowork-service/src/modules/agent-runtime/routes.test.ts
• learning_ai_common_plat/dashboards/admin-web/src/__tests__/agent-runtime.test.ts
• learning_ai_common_plat/dashboards/admin-web/src/__tests__/timeline.test.ts
• learning_ai_flowmonk/backend/src/modules/agent-runtime/routes.test.ts
• learning_ai_trails/backend/src/lib/ecosystem-phase3.test.ts

---

7. Implementation Checklist

• choose canonical fixture storage location
• define JSON fixture naming convention
• define validation approach per contract
• define how repos import or copy fixtures
• define CI/manual verification expectations

Current choices:

• canonical executable fixtures live in learning_ai_common_plat/packages/events
• human-readable planning docs remain in docs/ecosystem/
• producer and consumer repos validate against @bytelyst/events instead of copying schema definitions
• focused route or library tests are acceptable completion evidence when full end-to-end automation is not yet available

Commits:

• 41fa2cd drafted the initial version