# AI Governance & Evaluation Roadmap

> **Purpose:** Centralize evals, policy enforcement, safety review, release gates,
> and regression tracking for prompts, agents, and model behavior.
>
> **Primary Surfaces:** `services/platform-service/`, `services/extraction-service/`,
> `services/mcp-server/`
>
> **Status:** Planned
>
> **Estimated Effort:** 3-5 weeks

---

## Why This Is Missing

The repo has useful pieces:

- extraction evals
- telemetry and diagnostics
- flags and experiments

What it does not have is a central AI governance surface that answers:

- which prompts are approved
- which eval suite a release passed
- what policies apply to a class of agents
- what changed after a model or prompt update

---

## Recommended Stack

- `platform-service` governance modules
- OpenTelemetry for trace-linked evidence
- Promptfoo or a similar eval harness for offline regression
- policy layer using code-first rules first, with optional Cedar or OPA later

### Default Decision For This Repo

- Use Promptfoo-style offline evals first
- Keep policy enforcement in platform code before introducing a separate policy runtime
- Require governance links to agent and prompt versions rather than free-floating eval runs

---

## Dependencies

- Agent Registry & Prompt Versioning
- Agent Runtime & Orchestration

---

## V1 Non-Goals

- Universal automated red-teaming
- Full legal/compliance policy engine
- Replacing feature flags or experiments

---

## Phase 1 - Eval Registry

- [ ] Create modules:
  - [ ] `ai-evals`
  - [ ] `ai-policies`
  - [ ] `ai-releases`
- [ ] Add entities:
  - [ ] benchmark set
  - [ ] eval run
  - [ ] eval result
  - [ ] policy decision
  - [ ] release gate

---

## Phase 2 - Policy Engine

- [ ] Add policy checks for:
  - [ ] allowed models
  - [ ] max temperature
  - [ ] blocked tools
  - [ ] required human review
  - [ ] tenant-specific restrictions
- [ ] Add release gates based on eval thresholds
- [ ] Add regression detection on prompt or model changes

---

## Phase 3 - Operational Governance

- [ ] Link agent and prompt versions to eval runs
- [ ] Add incident-driven rollback recommendations
- [ ] Add policy override audit logs
- [ ] Add dashboards for pass rate, drift, and blocked releases

---

## Tech Stack Options

| Option                        | Pros                       | Cons                     | Fit                 |
| ----------------------------- | -------------------------- | ------------------------ | ------------------- |
| Promptfoo + platform registry | Good current ecosystem fit | Need custom service glue | Best near-term      |
| Custom eval runner only       | Full control               | Reinvents too much       | Weak starting point |
| OPA/Cedar-backed governance   | Strong policy model        | More complexity          | Good phase 2+       |

---

## Risks

- Shipping prompts without eval gating causes avoidable regressions
- Governance only in docs will drift from runtime
- No policy audit trail creates enterprise trust problems
