Implements roadmap §6 (profiles) and §5 deps on the bash runner, backward-compatible
(jobs without profile/deps behave exactly as before).
Profiles (§6):
- profile_get / profile_persona / fm_eff helpers + PROFILES_DIR (AGENT_QUEUE_PROFILES
override). A job's `profile:` inherits verify (<- default-verify), capabilities,
engine-class, prefers-engine, allowed-scope, review-policy when the job omits them;
job fields always override (precedence job > profile > default). Resolution runs via
fm_eff inside the capability gate and resolve_engine, so inherited caps/engine-class
take effect before launch.
- persona injection: the profile's persona block is prepended to the stripped body
fed to the engine (job .md unchanged on disk; nothing secret logged).
- allowed-scope guardrail (WARN-ONLY): scope_check logs a non-blocking WARNING +
records scope_warning= for changed paths outside the globs; path_in_scope is a
pure, unit-testable matcher (`dir/**` = subtree).
deps / DAG, single host (§5):
- deps reference other jobs by idempotency-key. dep_satisfied: shipped/ (hard) or
shipped/+testing/ (deps-mode: soft). deps_unmet drives a block-with-reason skip in
inbox selection (never launched/failed); cmd_status surfaces "blocked (waiting on
<keys>)". deps_would_cycle rejects cyclic submits on `add`.
- _drain_pending: `--once` drains past dep-blocked jobs (idle can't satisfy them)
while still waiting on retry/recovery backoff timers.
Meta now records effective (inherited) capabilities/engine-class/prefers-engine/
review-policy/allowed-scope so `status` reflects resolved config.
Read-only from meta: tokens or cost + attempts + line deltas + duration; recognizes the new retries_exhausted result. agent-queue.sh stays the source of truth.
Implements the single-host bash equivalents of roadmap §25 (durability/crash
recovery) and §26 (execution insights), plus §11 retry/dead-letter stand-in.
Resilience (A1-A4):
- recover_orphans + `recover` command: building/ jobs with a dead worker (dead
pid, pidstart reuse-guard) are moved back to inbox/ with attempts incremented,
on `run` startup and each loop. Idempotent (folder location is the guard).
- WIP checkpointing: for a git cwd, _wip_start creates/checks out aq/wip/<job>
and _wip_checkpoint commits changes on every exit path via an EXIT/INT/TERM
trap; never commits to main/current branch; non-git cwd skipped. RESUME: a
relaunch whose aq/wip/<job> exists checks it out first (continue from
checkpoint). wip_base persisted in a write-once sidecar.
- retry policy (now functional): retry { max, backoff, on } requeues failures
whose class (timeout|verify_failed|crash) is in `on`, honoring backoff via
next_eligible (selection skips until eligible), up to max attempts; exhaustion
-> failed/ result=retries_exhausted with the WIP branch + full log preserved.
- state integrity: all meta writes stay append-only; attempts/next_eligible/wip_*
are re-derivable; recovery is crash-safe.
Insights (B1-B6):
- per-run metrics into meta: duration_s, exit, result, attempts, and (git cwd)
files_changed/lines_added/lines_deleted from numstat wip_base..HEAD.
- parse_usage(engine, log) adapter: generic AQ_USAGE line + Claude/Codex token
heuristics; Devin/Copilot TODO; usage_estimated flag; never fabricates numbers.
- status insights sub-line; new `insights [job]` command (per-job metrics or a
recent table + per-engine token/cost/success/duration rollup).
- privacy: only metrics are recorded, never prompt content or secrets.
Backward-compatible: legacy .md and non-git cwd behave exactly as before.
New §24 + feature-catalog row:
- two delegation modes: atomic (leaf bug/feature/task) vs composite (roadmap/epic)
- introduce job kind (leaf|composite); composite routes to a planner/orchestrator
that fans out child leaf jobs as a DAG across factories/agents/profiles
- parentId hierarchy + rollup semantics (status/budget/verify/phase-gates) +
idempotent re-run (skip shipped children)
- source-of-truth/sync discipline (one record referenced by many; one-way echo)
- HYBRID decision recorded: model kind/parentId/rollup in the fleet layer now,
keep shared tracker ITEM_TYPES unchanged (label kind:roadmap), promote to a
first-class epic type later via additive migration once proven
- phasing: leaf-only P1-P2; manual composite P3; auto-decomposition planner P3->P5
Adds (never weakens existing) cases, each in its own temp AGENT_QUEUE_ROOT using
the no-op engine stub:
- backward-compat: legacy engine/cwd/yolo-only .md still lands in review/.
- priority: with --max 1, a critical job queued after a low job runs first
(order-recording stub).
- capability mismatch: has:definitely-not-installed -> failed/
result=capability_mismatch, asserting the agent was never launched.
- engine-class: agentic-coder + no engine, DEVIN_BIN stubbed -> review/.
- idempotency: same key+body twice -> 1 inbox file; same key+changed body in
inbox -> superseded; same key+different body after drain -> rejected.
Inbox counts use find (not a globbing ls) so set -e/pipefail tolerate an empty inbox.
Implements Gigafactory Phase 1 - Slice 1 in the bash runner (backward-compatible;
a legacy engine/cwd/yolo-only .md behaves exactly as before):
- Parse all new §5 manifest keys via fm_get with safe defaults; record them in
<job>.meta and surface priority/profile/capabilities/tracker-item in `status`.
Only priority, capabilities, engine-class and idempotency-key are functional
this slice; the rest (profile, prefers, budget, deps, deps-mode, retry,
review-policy, artifacts, tracker-item) are stored but inert.
- priority ordering: inbox_sorted picks critical>high>medium>low, ties by oldest;
per-lock serialization preserved.
- capability grammar + match: detect_capabilities advertises os/engine/node/has
tokens; caps_match honors key, key:value, key<op>version and os:any. A job whose
declared capabilities the host cannot satisfy is moved to failed/ with
result=capability_mismatch and the agent is never launched.
- engine-class resolution: explicit engine wins; else engine-class picks the first
available engine honoring prefers-engine (agentic-coder->devin,claude,codex;
chat-coder->copilot). No available engine -> result=no_engine. Adds copilot to
the engine driver + COPILOT_BIN.
- idempotency-key dedupe on add: same key+body -> no-op; same key+different body
supersedes an inbox prior, else is rejected with a clear error.
No change to queue/ data or the run/ship lifecycle. macOS + Linux safe.
- docs/GIGAFACTORY_ROADMAP.md: distributed multi-machine fleet vision
(factory x tool x profile routing) as a checklist-driven, phased
implementation roadmap (Phase 0-5) with acceptance criteria, verify
gates, and a 100% Definition-of-Done rubric
- committed path: coordinator as a platform-service module + control
plane on tracker-web, reached via a thin tracker adapter first; bash
runner survives as the offline edge factory agent
- README: add vision/roadmap pointer
Turn dash into a menu-driven control panel (single mjs script):
- numbered, arrow/j-k/1-9 selectable JOBS list (review/testing/failed/inbox)
- single-key actions wired to agent-queue.sh (single source of truth):
p promote, s ship, x reject, u requeue (reject/requeue confirm y/n)
- enter/l opens a live log viewer; r starts a detached run loop, S stops it
- run-loop pid indicator, transient action flashes, ? help overlay
- non-TTY falls back to the read-only live view
- README: dash command + interactive key table
- requeue <job>: move a failed job back to inbox/ and drop stale meta/body so
it re-runs cleanly
- clean [--keep N]: archive finished jobs' logs+meta beyond the newest N
(default 50) into queue/.archive/<ts>/; running jobs + .md records untouched
- document both in usage + bytelyst-cli subcommand list
Record pidstart (ps lstart) at launch and verify it in all liveness checks
(_meta_active, status, stop) via _pid_alive, so a recycled pid can never be
mistaken for our worker. Falls back to plain liveness when no start time recorded.
cmd_run now checks daemon.pid liveness up front: if a run loop is alive it exits
with an error (protecting the single-launcher invariant locking depends on); a
stale daemon.pid (dead pid) is cleared and the run proceeds.
Replace live_workers with reservation-aware active_workers + shared _meta_active:
a job counts toward --max the moment its meta is written (before the worker is
backgrounded), so --max can never be exceeded. A <30s guard prevents a meta
orphaned mid-launch from pinning a slot. busy_keys now shares _meta_active.
Mark a running worker '⚠ stalled' when its log has not changed for more than
AGENT_QUEUE_STALL_MIN minutes (default 10), using log mtime as the freshness
signal. Implemented in both the bash status table and the Node dashboard.
Honor 'timeout: 45m' (90s|45m|2h|1d) by wrapping the agent in timeout/gtimeout
when available (hard process-tree kill), else a portable bash watchdog. On expiry
the job moves doing->failed with result=timeout and a TIMED OUT log line.
Serialize jobs by lock key (frontmatter 'lock:' override, default cwd) via the
single run-loop's pre-launch eligibility check; the oldest non-busy job is picked
regardless of --max. Adds a flock-based worker guard where flock exists (Linux);
macOS relies on the single-daemon model. Records lock= in job meta.
