237481247e
Rename agent-queue/docs/gigafactory/ to docs/GIGAFACTORY/ and update every reference (README, system-overview code-map, and all phase job specs). Add an index README that lists the docs and points to the companion docs in learning_ai_common_plat. Docs-only; no behavior change. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
11 KiB
11 KiB
| engine | cwd | yolo | lock | timeout |
|---|---|---|---|---|
| devin | /Users/sd9235/code/mygh/learning_ai_common_plat | true | common-plat-phase3 | 10h |
ROLE: Senior full-stack engineer. Implement PHASE 3 of the Agent Gigafactory END-TO-END
in learning_ai_common_plat, SEQUENTIALLY, over a long unattended run: smart routing
(tunable weights + preemption), DAG job decomposition, per-product budgets, and the
tracker-web fleet control plane. Work SLICE BY SLICE; each slice is self-contained,
fully tested, and pushed before the next begins. This is an overnight run — favor
correctness, small verifiable steps, and never leaving main/PR in a broken state.
================================================================================ PREREQUISITE (the operator guarantees this before starting): Phase 2 is COMPLETE and merged to origin/main — fleet foundation, atomic claim, scheduler/router core, artifacts, enrollment+tokens, feature-flags/shadow, the in-process tracker->module wiring, and the two-factory demo are ALL on main. You branch off CURRENT origin/main.
GLOBAL GUARDRAILS (unattended danger mode — obey strictly):
- Branch: feat/gigafactory-phase3 off CURRENT origin/main. ONE long-lived branch; ONE commit per slice (conventional commits). Push after EVERY slice. Open ONE PR after Slice 1 and keep pushing to it. DO NOT MERGE anything. DO NOT touch origin/main.
- Tests are SACRED: never delete, weaken, skip, or
.skip/.onlya test to go green. If you cannot make a slice pass honestly, see the FAILURE PROTOCOL below. - A slice is "done" only when its VERIFY GATE is fully green. Never start slice N+1 with slice N red.
- Reserved / DO NOT TOUCH: the agent-queue repo (different repo), unrelated services (cowork-service, extraction-service), packages/* internals (consume, don't edit), and any backup/* or dependabot/* branches. Stay in services/platform-service + dashboards/tracker-web.
- Conventions: ESM
.jsimport specifiers; noany; no console.log (use app.log/req.log, and the tracker-web logger/telemetry pattern); every Cosmos doc carriesproductId; reuse @bytelyst/* packages and existing module patterns (types.ts -> repository.ts -> routes.ts). Do NOT hardcode colors/URLs/secrets. - CHECKPOINTING: maintain docs/GIGAFACTORY/gigafactory-phase3-progress.md on the branch. After each slice, record: slice name, status (DONE/WIP/FAILED), commit sha, verify-gate result, and any follow-ups. Commit it WITH the slice. If you resume after an interruption, read it first and continue from the first not-DONE slice.
FAILURE PROTOCOL (per slice): attempt the verify gate up to 3 times, fixing the ROOT
cause each time (not the test). If still red after 3 honest attempts: commit the WIP with
message wip(<scope>): <slice> — BLOCKED: <one-line reason>, mark it FAILED in
progress.md with the exact failing output, and MOVE ON to the next slice that does NOT
depend on it (dependencies noted per slice). Never thrash; never fake green.
READ FIRST:
- ../learning_ai_devops_tools/agent-queue/docs/GIGAFACTORY/GIGAFACTORY_ROADMAP.md — §7 (scoring; Phase-3 = tunable weights + preemption), §5/§6 (DAG/deps), §11/§13 (budgets), §14 Phase-3 checklist + Exit criteria, §16 Definition-of-Done.
- services/platform-service/src/modules/fleet/{scheduler,coordinator,repository,routes, types}.ts — the engine you extend (read the existing claim/lease/fence/selectJob).
- dashboards/tracker-web/ — match its App-Router structure (src/app, src/app/api), data-fetching/auth pattern, @bytelyst/ui + design-tokens usage, vitest + Playwright (e2e/, playwright.config.ts) setup. The existing fleet HTTP API you consume: POST/GET /fleet/jobs, GET /fleet/jobs/:id, PATCH /fleet/jobs/:id, POST /fleet/claim, lease renew/release, POST /fleet/factories/heartbeat|enroll, token rotate/revoke, GET /fleet/jobs/:id/runs, GET /fleet/jobs/:id/events, artifacts upload/list/get/delete.
SLICE 1 — Tunable scoring weights + preemption (backend; depends on: nothing)
Extend the PURE scheduler (scheduler.ts) without breaking §7 Phase-2 behavior:
- Weights become configurable per-product/per-request (passed in; fixed defaults preserved so existing tests stay green). Add a small typed FleetWeightConfig resolver (defaults -> optional product override). NO env reads inside the pure module.
- Preemption: a
selectWithPreemption(candidates, runningJobs, factory, ctx, weights?)that, when a CRITICAL job cannot be placed and only lower-priority jobs are running, returns a preemption decision { evict: jobId, reason, breakdown } — PURE, no I/O. - Wire preemption into the coordinator behind a flag (FLEET_PREEMPTION, default OFF; OFF = byte-for-byte current behavior). Eviction must checkpoint + requeue the victim via the EXISTING fenced-requeue path (bump leaseEpoch; the zombie's late report is fenced). TESTS: weight override changes ranking; defaults reproduce all prior picks; preemption evicts only a strictly-lower-priority running job, never an equal/higher; victim is requeued with checkpoint + bumped epoch and its stale report is fenced; flag OFF = no preemption. VERIFY GATE: pnpm --filter @lysnrai/platform-service exec vitest run src/modules/fleet && build && (pnpm build && pnpm test).
SLICE 2 — DAG job decomposition (backend; depends on: nothing; independent of S1)
Parent/child jobs with dependency-gated execution (§5/§6):
- types: a job may declare children (subtasks) and dependsOn[] (sibling/child ids). Reuse existing kind ('leaf'|...) + parentId; add child submission + a DAG edge model. Cycle detection at submit (extend the existing submit-time cycle check).
- coordinator: a parent is not claimable until its children reach a terminal state (or its declared deps are satisfied); completing the last child unblocks the parent. claimNextJob only returns deps-satisfied jobs (extend the existing predicate). Fan-out: submitting a parent with children atomically creates the children.
- routes (additive): POST /fleet/jobs/:id/children (submit children), GET /fleet/jobs/:id/dag (return the subtree + per-node stage). productId-scoped. TESTS: parent blocked until children done; last child completion unblocks parent; cycle at submit -> rejected; capability/priority still respected per node; DAG endpoint returns the correct subtree; all prior fleet tests green. VERIFY GATE as in Slice 1 (+ items unaffected).
SLICE 3 — Per-product budgets + pause/resume (backend; depends on: nothing)
Cost ceilings that pause routing (§11/§13):
- A FleetBudgetDoc per productId (ceilingUsd, window, spentUsd, status active|paused). Spend accrues from job run cost (reuse run/insights cost if present; else estimate from budget.usd at completion). Container partitioned by /productId.
- Enforcement in claimNextJob: if the product's budget is paused or the next job would exceed the ceiling, that product's jobs are NOT claimed (other products unaffected). Behind FLEET_BUDGETS (default OFF = unchanged).
- routes (additive): GET/PUT /fleet/budgets/:productId, POST /fleet/budgets/:productId/pause, POST /fleet/budgets/:productId/resume. TESTS: under ceiling -> claims proceed; crossing ceiling -> that product pauses, others still claim; manual pause blocks claims; resume restores; flag OFF = no enforcement; spend accounting is monotonic + idempotent per run. VERIFY GATE as above.
SLICE 4 — tracker-web Fleet Control Plane UI (frontend; depends on: S1-S3 endpoints, but build defensively — feature-detect/degrade if an endpoint is absent)
A new /fleet section in dashboards/tracker-web (App Router), matching existing patterns:
- Typed fleet API client (src/lib or src/app/api proxy as the repo does it) wrapping the fleet endpoints with auth token injection (reuse the existing auth/client pattern).
- Pages/components (use @bytelyst/ui + --*-tokens; every interactive element has an
aria-label or visible label):
- Fleet map: factories (id, caps, health, load, lease state) as live cards.
- Job table: filter by product/stage/priority; submit-job modal; row -> job detail.
- Job detail: stage timeline from /events, runs from /runs, artifacts list, a SHIP action (PATCH stage), and the DAG subtree (from /dag) when present.
- Budget panel: per-product ceiling + spent + pause/resume controls.
- Live updates via polling (simple, robust) unless an SSE/stream endpoint exists.
TESTS: vitest component/unit tests for the client + key components (render, actions call
the right endpoint, error/empty/degraded states); Playwright e2e for the core flow
(see fleet map -> open a job -> ship; pause a budget -> resume). VERIFY GATE:
the tracker-web
verifyscript (typecheck + lint + test + e2e) green — run exactly what its package.json defines (e.g. pnpm --filter run verify, or the documented equivalent). Do not weaken its lint/e2e config.
SLICE 5 — Docs + roadmap + Phase-3 exit criteria (depends on: S1-S4 outcomes)
- Update ../learning_ai_devops_tools/agent-queue/docs/GIGAFACTORY/GIGAFACTORY_ROADMAP.md §14 Phase-3 checkboxes for every box you actually completed, with a one-line note + the flag names (FLEET_PREEMPTION/FLEET_BUDGETS) and which are default-OFF. Tick the Phase-3 Exit-criteria line ONLY if its conditions are genuinely met; otherwise note the exact remaining %. (This is a docs edit in the OTHER repo — make it as a separate small commit/PR in learning_ai_devops_tools, OR include the roadmap delta as a patch file under docs/ in THIS branch and note it for the operator — do NOT entangle the two repos' git history. Prefer the patch-file note if a clean cross-repo PR isn't trivial.)
- Update dashboards/tracker-web/README + a short docs/GIGAFACTORY/FLEET_CONTROL_PLANE.md (how to use the new UI, the flags, the endpoints consumed).
- Finalize docs/GIGAFACTORY/gigafactory-phase3-progress.md with the end-state of every slice.
FINAL OUTPUT — print ONE consolidated report in EXACTLY this format:
Implementation Report — Phase 3 (overnight)
Branch & PR
Per-slice results
| slice | status (DONE/WIP/FAILED) | commit | verify gate | notes |