Merge PR #4: Phase 1 Slice 4 — tracker adapter (task <-> job round-trip)
Closes Phase 1. selftest 53/53 (46 regression + 7 new); one-way echo sends metrics only (no prompt/secrets — asserted); token env-sourced; Items API contract (GET/PATCH status/POST comments, bearer + X-Product-Id).
This commit is contained in:
saravanakumardb1
commit
2ad5c6dee5
@ -317,6 +317,67 @@ agent-queue.sh insights # recent-jobs table + per-engine rollup
|
||||
> numbers are never fabricated. The per-engine rollup marks totals that include any
|
||||
> estimated value with `*`.
|
||||
|
||||
## Tracker integration (§10)
|
||||
|
||||
Closes the task ↔ job round-trip against the platform-service **items API**: a
|
||||
tracker Item can become a job, and a job's outcome echoes back to the Item.
|
||||
|
||||
```bash
|
||||
agent-queue.sh from-tracker <ITEM_ID> # pull an Item -> materialize a job in inbox/
|
||||
agent-queue.sh to-tracker <job> # echo the job's current outcome to its Item
|
||||
```
|
||||
|
||||
All HTTP goes through one curl wrapper (`tracker_api`); there are no other network
|
||||
calls. Real use needs **platform-service running and a bearer token**.
|
||||
|
||||
### Config (env)
|
||||
|
||||
| Var | Default | Meaning |
|
||||
| --- | ------- | ------- |
|
||||
| `AQ_TRACKER_API` | `http://localhost:4003` | base URL of the items API (routes live under `/api`) |
|
||||
| `AQ_TRACKER_TOKEN` | _(none)_ | bearer token — **required** for real calls; never hardcode |
|
||||
| `AQ_PRODUCT_ID` | _(none)_ | productId (sent as `X-Product-Id`; every Item has one) |
|
||||
| `AQ_TRACKER_CWD` | `$PWD` | cwd a tracker-derived job runs in (Items carry no cwd) |
|
||||
| `AQ_TRACKER_AUTO` | `0` | `1` = auto-echo on each transition (default OFF — echo is manual) |
|
||||
| `AQ_TRACKER_STATUS_INPROGRESS` / `_DONE` / `_FAILED` | `in_progress` / `done` / `wont_fix` | Item status per bucket (the API has no blocked/failed status) |
|
||||
| `AQ_TRACKER_API_CMD` | _(none)_ | test seam: a stub that replaces the curl HTTP entirely (selftest uses it) |
|
||||
|
||||
### `from-tracker` — Item → job
|
||||
|
||||
`GET /api/items/<id>`, then maps fields to job frontmatter:
|
||||
|
||||
| Item | Job |
|
||||
| ---- | --- |
|
||||
| `title` + `description` | job body (verbatim instruction markdown) |
|
||||
| `id` | `tracker-item: <id>` and `idempotency-key: tracker-<id>` (stable) |
|
||||
| `priority` | `priority:` (label overrides; else Item priority; else `medium`) |
|
||||
| label `engine-class:<x>` | `engine-class: <x>` |
|
||||
| label `profile:<x>` | `profile: <x>` |
|
||||
| label `priority:<x>` | `priority: <x>` |
|
||||
| label `cap:<token>` | a `capabilities: [...]` entry |
|
||||
|
||||
Idempotent on the derived `idempotency-key` (Slice 1 dedupe) — pulling the same
|
||||
Item twice never enqueues a duplicate.
|
||||
|
||||
### `to-tracker` — job → Item (one-way echo, §24.5)
|
||||
|
||||
Only if the job's meta has a `tracker-item`. Maps the job's stage/result to an Item
|
||||
status and `PATCH /api/items/<id>/status`, then `POST /api/items/<id>/comments`
|
||||
with a **metrics-only** summary (result, attempts, duration, tokens/cost, +/- lines —
|
||||
**never prompt content or secrets**):
|
||||
|
||||
| job result/stage | Item status |
|
||||
| ---------------- | ----------- |
|
||||
| building / review / testing / recovered | `in_progress` |
|
||||
| shipped | `done` |
|
||||
| failed / timeout / verify_failed / retries_exhausted / capability_mismatch / no_engine / rejected | `wont_fix` (override via `AQ_TRACKER_STATUS_FAILED`) |
|
||||
|
||||
Idempotent via `tracker_echoed` in the meta (re-echoing an unchanged outcome is a
|
||||
no-op). The echo is **one-way** (child → tracker) and **never authoritative for
|
||||
execution**: an echo failure is logged and the job continues unchanged. With
|
||||
`AQ_TRACKER_AUTO=1` the worker echoes automatically on each transition; otherwise
|
||||
echo is manual. `status` / `insights` surface the `tracker-item` and last echoed status.
|
||||
|
||||
## Config (env overrides)
|
||||
|
||||
| Var | Default | Meaning |
|
||||
|
||||
@ -66,6 +66,25 @@ CLAUDE_BIN="${CLAUDE_BIN:-$(command -v claude || echo claude)}"
|
||||
CODEX_BIN="${CODEX_BIN:-$(command -v codex || echo codex)}"
|
||||
COPILOT_BIN="${COPILOT_BIN:-$(command -v copilot || echo copilot)}"
|
||||
|
||||
# ── Tracker integration (§10) — task <-> job round-trip via the items API ──
|
||||
# Base URL of the platform-service items API; the items routes live under /api.
|
||||
AQ_TRACKER_API="${AQ_TRACKER_API:-http://localhost:4003}"
|
||||
# Bearer token (required for real calls; never hardcode). productId stamps Items.
|
||||
AQ_TRACKER_TOKEN="${AQ_TRACKER_TOKEN:-}"
|
||||
AQ_PRODUCT_ID="${AQ_PRODUCT_ID:-}"
|
||||
# cwd a tracker-derived job runs in (Items carry no cwd); defaults to the invoking dir.
|
||||
AQ_TRACKER_CWD="${AQ_TRACKER_CWD:-$PWD}"
|
||||
# Auto-echo job outcomes back to the tracker on each transition (opt-in, default OFF).
|
||||
AQ_TRACKER_AUTO="${AQ_TRACKER_AUTO:-0}"
|
||||
# Item status the API uses for each bucket (the items API has no blocked/failed
|
||||
# status, so failures map to wont_fix by default — all overridable).
|
||||
AQ_TRACKER_STATUS_INPROGRESS="${AQ_TRACKER_STATUS_INPROGRESS:-in_progress}"
|
||||
AQ_TRACKER_STATUS_DONE="${AQ_TRACKER_STATUS_DONE:-done}"
|
||||
AQ_TRACKER_STATUS_FAILED="${AQ_TRACKER_STATUS_FAILED:-wont_fix}"
|
||||
# Test seam: a stub script that replaces the real curl HTTP (see selftest.sh).
|
||||
AQ_TRACKER_API_CMD="${AQ_TRACKER_API_CMD:-}"
|
||||
CURL_BIN="${CURL_BIN:-$(command -v curl || echo curl)}"
|
||||
|
||||
# ── Colors ──────────────────────────────────────────────────────────
|
||||
if [[ -t 1 ]]; then
|
||||
C_RESET=$'\033[0m'; C_DIM=$'\033[2m'; C_BOLD=$'\033[1m'
|
||||
@ -613,6 +632,7 @@ run_worker() {
|
||||
} >> "$logf"
|
||||
mv "$doing_file" "$FAILED/" 2>/dev/null
|
||||
{ echo "result=capability_mismatch"; echo "ended=$(date +%s)"; } >> "$metaf"
|
||||
_auto_echo "$job"
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
@ -628,6 +648,7 @@ run_worker() {
|
||||
} >> "$logf"
|
||||
mv "$doing_file" "$FAILED/" 2>/dev/null
|
||||
{ echo "result=no_engine"; echo "ended=$(date +%s)"; } >> "$metaf"
|
||||
_auto_echo "$job"
|
||||
return 0
|
||||
fi
|
||||
|
||||
@ -642,6 +663,7 @@ run_worker() {
|
||||
echo "FATAL: cwd does not exist: $cwd" >> "$logf"
|
||||
mv "$doing_file" "$FAILED/" 2>/dev/null
|
||||
echo "result=failed" >> "$metaf"; echo "ended=$(date +%s)" >> "$metaf"
|
||||
_auto_echo "$job"
|
||||
return 1
|
||||
fi
|
||||
|
||||
@ -773,6 +795,9 @@ run_worker() {
|
||||
echo "FAILED (rc=$rc): $(date)" >> "$logf"
|
||||
_finish_failure "$job" "$doing_file" "$metaf" "$logf" "crash" "$rc" "$started"
|
||||
fi
|
||||
|
||||
# Opt-in echo of the resting/terminal outcome back to the tracker (§10).
|
||||
_auto_echo "$job"
|
||||
}
|
||||
|
||||
# ── Resilience & insights helpers (Phase 1 — single-host §25/§26) ────
|
||||
@ -1029,6 +1054,222 @@ _insights_line() {
|
||||
}
|
||||
|
||||
# ── Commands ────────────────────────────────────────────────────────
|
||||
# ── Tracker integration (§10) — task <-> job round-trip ─────────────
|
||||
#
|
||||
# tracker_api <METHOD> <PATH> [JSON] -> emits the response body, then a final
|
||||
# line with the HTTP status code. ALL HTTP goes through here (curl only). Tests
|
||||
# replace it wholesale via AQ_TRACKER_API_CMD so no live service is needed.
|
||||
tracker_api() {
|
||||
local method=$1 path=$2 body=${3:-}
|
||||
if [[ -n "$AQ_TRACKER_API_CMD" ]]; then
|
||||
"$AQ_TRACKER_API_CMD" "$method" "$path" "$body"
|
||||
return $?
|
||||
fi
|
||||
local url="${AQ_TRACKER_API}${path}"
|
||||
local -a args=(-sS -m "${AQ_TRACKER_TIMEOUT:-30}" -X "$method"
|
||||
-H "Content-Type: application/json" -w '\n%{http_code}')
|
||||
[[ -n "$AQ_TRACKER_TOKEN" ]] && args+=(-H "Authorization: Bearer $AQ_TRACKER_TOKEN")
|
||||
[[ -n "$AQ_PRODUCT_ID" ]] && args+=(-H "X-Product-Id: $AQ_PRODUCT_ID")
|
||||
[[ -n "$body" ]] && args+=(--data "$body")
|
||||
local out rc
|
||||
out=$("$CURL_BIN" "${args[@]}" "$url" 2>/dev/null); rc=$?
|
||||
if [[ $rc -ne 0 ]]; then printf '%s\n000\n' "$out"; else printf '%s\n' "$out"; fi
|
||||
}
|
||||
|
||||
# _api_call <METHOD> <PATH> [JSON] -> sets globals API_BODY + API_CODE.
|
||||
_api_call() {
|
||||
local out; out=$(tracker_api "$@")
|
||||
API_CODE=$(printf '%s' "$out" | tail -n1)
|
||||
API_BODY=$(printf '%s' "$out" | sed '$d')
|
||||
}
|
||||
|
||||
# _json_str <key> (reads JSON on stdin) -> the top-level string value, unescaped.
|
||||
# Pure awk (POSIX) — mac + linux safe, no jq dependency.
|
||||
_json_str() {
|
||||
awk -v key="$1" '
|
||||
{ json = json $0 }
|
||||
END {
|
||||
pat = "\"" key "\""
|
||||
i = index(json, pat); if (i == 0) exit
|
||||
rest = substr(json, i + length(pat))
|
||||
sub(/^[ \t]*:[ \t]*/, "", rest)
|
||||
if (substr(rest, 1, 1) != "\"") exit
|
||||
rest = substr(rest, 2); n = length(rest)
|
||||
for (j = 1; j <= n; j++) {
|
||||
c = substr(rest, j, 1)
|
||||
if (c == "\\") { j++; e = substr(rest, j, 1)
|
||||
if (e == "n") out = out "\n"; else if (e == "t") out = out "\t"
|
||||
else if (e == "r") out = out "\r"; else out = out e }
|
||||
else if (c == "\"") break
|
||||
else out = out c
|
||||
}
|
||||
printf "%s", out
|
||||
}'
|
||||
}
|
||||
|
||||
# _json_labels (reads JSON on stdin) -> one labels[] entry per line.
|
||||
_json_labels() {
|
||||
awk '
|
||||
{ json = json $0 }
|
||||
END {
|
||||
i = index(json, "\"labels\""); if (i == 0) exit
|
||||
rest = substr(json, i); lb = index(rest, "["); rb = index(rest, "]")
|
||||
if (lb == 0 || rb == 0 || rb < lb) exit
|
||||
arr = substr(rest, lb + 1, rb - lb - 1)
|
||||
while (match(arr, /"([^"\\]|\\.)*"/)) {
|
||||
tok = substr(arr, RSTART + 1, RLENGTH - 2)
|
||||
gsub(/\\"/, "\"", tok); print tok
|
||||
arr = substr(arr, RSTART + RLENGTH)
|
||||
}
|
||||
}'
|
||||
}
|
||||
|
||||
# _json_escape <text> -> the text as a JSON string body (no surrounding quotes).
|
||||
_json_escape() {
|
||||
printf '%s' "$1" | awk '
|
||||
{ line = $0; gsub(/\\/, "\\\\", line); gsub(/"/, "\\\"", line); gsub(/\t/, "\\t", line)
|
||||
out = out (NR > 1 ? "\\n" : "") line }
|
||||
END { printf "%s", out }'
|
||||
}
|
||||
|
||||
# _tracker_status_for <result> -> the Item status for a job result/stage.
|
||||
_tracker_status_for() {
|
||||
case "$1" in
|
||||
shipped) printf '%s' "$AQ_TRACKER_STATUS_DONE";;
|
||||
failed|timeout|verify_failed|retries_exhausted|capability_mismatch|no_engine|rejected)
|
||||
printf '%s' "$AQ_TRACKER_STATUS_FAILED";;
|
||||
*) printf '%s' "$AQ_TRACKER_STATUS_INPROGRESS";;
|
||||
esac
|
||||
}
|
||||
|
||||
# _tracker_note <job> <metaf> <result> <status> -> a metrics-only summary line.
|
||||
# NEVER includes prompt/body content or secrets — only run metrics (§24.5/§26).
|
||||
_tracker_note() {
|
||||
local jn=$1 metaf=$2 result=$3 status=$4 s attempts dur ti to cost la ld
|
||||
attempts=$(_meta_val "$metaf" attempts); dur=$(_meta_val "$metaf" duration_s)
|
||||
ti=$(_meta_val "$metaf" tokens_in); to=$(_meta_val "$metaf" tokens_out)
|
||||
cost=$(_meta_val "$metaf" cost_usd)
|
||||
la=$(_meta_val "$metaf" lines_added); ld=$(_meta_val "$metaf" lines_deleted)
|
||||
s="agent-queue: job ${jn} -> ${result:-$status} (status=${status})."
|
||||
[[ -n "$attempts" ]] && s+=" attempts=${attempts}."
|
||||
[[ -n "$dur" ]] && s+=" duration=${dur}s."
|
||||
[[ -n "$ti$to" ]] && s+=" tokens=${ti:-0}/${to:-0}."
|
||||
[[ -n "$cost" ]] && s+=" cost_usd=${cost}."
|
||||
[[ -n "$la$ld" ]] && s+=" diff=+${la:-0}/-${ld:-0}."
|
||||
printf '%s' "$s"
|
||||
}
|
||||
|
||||
# from-tracker <ITEM_ID> — pull a tracker Item and materialize a job in inbox/.
|
||||
# Idempotent on the derived key `tracker-<ITEM_ID>` (Slice 1 dedupe).
|
||||
cmd_from_tracker() {
|
||||
ensure_dirs
|
||||
local item_id="${1:-}"
|
||||
[[ -n "$item_id" ]] || die "usage: from-tracker <ITEM_ID>"
|
||||
_api_call GET "/api/items/$item_id"
|
||||
case "$API_CODE" in 2*) :;; *) die "from-tracker: items API returned HTTP ${API_CODE:-error} for item $item_id";; esac
|
||||
|
||||
local title desc iprio
|
||||
title=$(printf '%s' "$API_BODY" | _json_str title)
|
||||
desc=$(printf '%s' "$API_BODY" | _json_str description)
|
||||
iprio=$(printf '%s' "$API_BODY" | _json_str priority)
|
||||
[[ -n "$title$desc" ]] || die "from-tracker: item $item_id has no title/description (HTTP $API_CODE)"
|
||||
|
||||
# labels carry optional manifest hints: engine-class:, profile:, priority:, cap:
|
||||
local engine_class="" profile="" prio="" caps_list="" label
|
||||
while IFS= read -r label; do
|
||||
[[ -n "$label" ]] || continue
|
||||
case "$label" in
|
||||
engine-class:*) engine_class=${label#engine-class:};;
|
||||
profile:*) profile=${label#profile:};;
|
||||
priority:*) prio=${label#priority:};;
|
||||
cap:*) caps_list+="${label#cap:}, ";;
|
||||
esac
|
||||
done < <(printf '%s' "$API_BODY" | _json_labels)
|
||||
prio=${prio:-${iprio:-medium}}
|
||||
case "$prio" in critical|high|medium|low) :;; *) prio=medium;; esac
|
||||
|
||||
# Materialize into a .md file (so cmd_add/the queue recognize it). mktemp -d is
|
||||
# the portable way to get a unique path with a fixed (.md) basename on mac+linux.
|
||||
local safe_id tmpdir tmp
|
||||
safe_id=$(printf '%s' "$item_id" | tr -c 'A-Za-z0-9._-' '_')
|
||||
tmpdir=$(mktemp -d "${TMPDIR:-/tmp}/aq-fromtracker.XXXXXX")
|
||||
tmp="$tmpdir/tracker-$safe_id.md"
|
||||
{
|
||||
echo "---"
|
||||
echo "cwd: $AQ_TRACKER_CWD"
|
||||
echo "yolo: true"
|
||||
echo "priority: $prio"
|
||||
[[ -n "$engine_class" ]] && echo "engine-class: $engine_class"
|
||||
[[ -n "$profile" ]] && echo "profile: $profile"
|
||||
[[ -n "$caps_list" ]] && echo "capabilities: [${caps_list%, }]"
|
||||
echo "tracker-item: $item_id"
|
||||
echo "idempotency-key: tracker-$item_id"
|
||||
echo "---"
|
||||
echo
|
||||
[[ -n "$title" ]] && { echo "# $title"; echo; }
|
||||
printf '%s\n' "$desc"
|
||||
} > "$tmp"
|
||||
|
||||
cmd_add "$tmp"
|
||||
rm -rf "$tmpdir"
|
||||
local created; created=$(grep -lE "^tracker-item:[[:space:]]*${item_id}[[:space:]]*\$" "$INBOX"/*.md 2>/dev/null | head -1)
|
||||
if [[ -n "$created" ]]; then
|
||||
log "from-tracker: item $item_id -> $C_BOLD$(basename "$created")$C_RESET"
|
||||
else
|
||||
log "from-tracker: item $item_id already queued elsewhere (deduped)"
|
||||
fi
|
||||
}
|
||||
|
||||
# to-tracker <job> — one-way echo of a job's CURRENT outcome to its tracker Item
|
||||
# (child -> tracker, §24.5). Idempotent via meta `tracker_echoed`; never fatal.
|
||||
cmd_to_tracker() {
|
||||
ensure_dirs
|
||||
local job="${1:-}"
|
||||
[[ -n "$job" ]] || die "usage: to-tracker <job>"
|
||||
local metaf="$STATE/$job.meta"
|
||||
[[ -f "$metaf" ]] || metaf=$(ls -1t "$STATE"/*"$job"*.meta 2>/dev/null | head -1)
|
||||
[[ -f "$metaf" ]] || die "to-tracker: no meta for job '$job'"
|
||||
local jn; jn=$(basename "$metaf"); jn=${jn%.meta}
|
||||
|
||||
local item_id; item_id=$(_meta_val "$metaf" tracker_item)
|
||||
if [[ -z "$item_id" ]]; then
|
||||
log "to-tracker: $jn has no tracker-item — nothing to echo"
|
||||
return 0
|
||||
fi
|
||||
local result status last
|
||||
result=$(_meta_val "$metaf" result)
|
||||
status=$(_tracker_status_for "$result")
|
||||
last=$(_meta_val "$metaf" tracker_echoed)
|
||||
if [[ "$last" == "$status" ]]; then
|
||||
log "to-tracker: $jn already echoed status=$status to $item_id (no-op)"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# 1) status transition
|
||||
_api_call PATCH "/api/items/$item_id/status" "{\"status\":\"$status\"}"
|
||||
case "$API_CODE" in
|
||||
2*) :;;
|
||||
*) err "to-tracker: status PATCH for $item_id failed (HTTP ${API_CODE:-error}) — non-fatal; job state unchanged"; return 0;;
|
||||
esac
|
||||
# 2) metrics-only comment (never prompt content / secrets)
|
||||
local note; note=$(_tracker_note "$jn" "$metaf" "$result" "$status")
|
||||
_api_call POST "/api/items/$item_id/comments" "{\"body\":\"$(_json_escape "$note")\"}"
|
||||
case "$API_CODE" in
|
||||
2*) :;;
|
||||
*) err "to-tracker: comment for $item_id failed (HTTP ${API_CODE:-error}) — non-fatal";;
|
||||
esac
|
||||
# 3) record echoed status (idempotency)
|
||||
{ echo "tracker_echoed=$status"; echo "tracker_echoed_at=$(date +%s)"; } >> "$metaf"
|
||||
log "to-tracker: echoed $jn -> item $item_id (status=$status)"
|
||||
}
|
||||
|
||||
# _auto_echo <job> — opt-in (AQ_TRACKER_AUTO=1) best-effort echo on a transition.
|
||||
# Never blocks or fails the job: the tracker is downstream, not authoritative.
|
||||
_auto_echo() {
|
||||
[[ "$AQ_TRACKER_AUTO" == 1 ]] || return 0
|
||||
cmd_to_tracker "$1" >/dev/null 2>&1 || true
|
||||
}
|
||||
|
||||
cmd_init() { ensure_dirs; log "queue initialized at $C_BOLD$QUEUE_ROOT$C_RESET"; }
|
||||
|
||||
cmd_add() {
|
||||
@ -1215,6 +1456,7 @@ cmd_run() {
|
||||
run_worker "$doing_file" &
|
||||
{ echo "pid=$!"; echo "pidstart=$(_pidstart "$!")"; } >> "$STATE/$job.meta"
|
||||
log "▶ launching $C_BOLD$job$C_RESET (engine=$w_eng, lock=$w_key)"
|
||||
_auto_echo "$job" # building -> in_progress (opt-in)
|
||||
sleep 1
|
||||
running=$(active_workers)
|
||||
done
|
||||
@ -1269,10 +1511,12 @@ cmd_status() {
|
||||
local m_prio m_prof m_caps m_trk extra=""
|
||||
m_prio=$(grep '^priority=' "$f" | cut -d= -f2-); m_prof=$(grep '^profile=' "$f" | cut -d= -f2-)
|
||||
m_caps=$(grep '^capabilities=' "$f" | cut -d= -f2-); m_trk=$(grep '^tracker_item=' "$f" | cut -d= -f2-)
|
||||
local m_echo; m_echo=$(grep '^tracker_echoed=' "$f" | tail -1 | cut -d= -f2-)
|
||||
[[ -n "$m_prio" ]] && extra+="prio=$m_prio "
|
||||
[[ -n "$m_prof" ]] && extra+="profile=$m_prof "
|
||||
[[ -n "$m_caps" ]] && extra+="caps=$m_caps "
|
||||
[[ -n "$m_trk" ]] && extra+="tracker=$m_trk "
|
||||
[[ -n "$m_echo" ]] && extra+="echoed=$m_echo "
|
||||
[[ -n "$extra" ]] && printf ' %s%s%s\n' "$C_DIM" "$extra" "$C_RESET"
|
||||
printf ' %s%s%s\n' "$C_DIM" "$(_insights_line "$f")" "$C_RESET"
|
||||
done
|
||||
@ -1315,7 +1559,7 @@ cmd_insights() {
|
||||
for k in engine result attempts started ended duration_s exit verify_exit \
|
||||
model tokens_in tokens_out tokens_cached cost_usd turns tool_calls usage_estimated \
|
||||
files_changed lines_added lines_deleted wip_branch wip_base wip_commit \
|
||||
next_eligible retry_class recovered; do
|
||||
next_eligible retry_class recovered tracker_item tracker_echoed tracker_echoed_at; do
|
||||
val=$(_meta_val "$f" "$k")
|
||||
[[ -n "$val" ]] && printf ' %-15s %s\n' "$k" "$val"
|
||||
done
|
||||
@ -1428,6 +1672,7 @@ cmd_ship() {
|
||||
mv "$f" "$SHIPPED/$base"
|
||||
[[ -f "$STATE/$name.meta" ]] && echo "result=shipped" >> "$STATE/$name.meta"
|
||||
log "shipped $C_BOLD$base$C_RESET (testing → shipped)"
|
||||
_auto_echo "$name"
|
||||
}
|
||||
|
||||
# promote <job> — advance one stage forward: review → testing → shipped.
|
||||
@ -1447,6 +1692,7 @@ cmd_promote() {
|
||||
mv "$f" "$dest/$base"
|
||||
[[ -f "$STATE/$name.meta" ]] && echo "result=$result" >> "$STATE/$name.meta"
|
||||
log "promoted $C_BOLD$base$C_RESET ($from → $result)"
|
||||
_auto_echo "$name"
|
||||
}
|
||||
|
||||
# reject <job> — move a review/testing job to failed/ (manual gate rejection).
|
||||
@ -1516,6 +1762,8 @@ ${C_BOLD}COMMANDS${C_RESET}
|
||||
watch [interval] live status (default 2s, bash)
|
||||
insights [job] per-job metrics, or recent table + per-engine rollup
|
||||
recover reclaim orphaned building/ jobs (dead worker) -> inbox
|
||||
from-tracker <ITEM_ID> pull a tracker Item -> materialize a job in inbox/ (§10)
|
||||
to-tracker <job> echo a job's outcome to its tracker Item (one-way)
|
||||
dash [--interval N] richer live Node dashboard (recent shipped/failed too)
|
||||
stop kill running workers + the run loop
|
||||
logs <job> [-f] print (or follow) a job's log
|
||||
@ -1565,6 +1813,12 @@ ${C_BOLD}ENV${C_RESET}
|
||||
AGENT_QUEUE_ROOT (=$QUEUE_ROOT) AGENT_QUEUE_MAX (=$MAX_CONCURRENCY)
|
||||
AGENT_QUEUE_ENGINE (=$DEFAULT_ENGINE) AGENT_QUEUE_VERIFY (default verify cmd)
|
||||
DEVIN_BIN / CLAUDE_BIN / CODEX_BIN / COPILOT_BIN
|
||||
|
||||
${C_BOLD}TRACKER${C_RESET} (§10 — from-tracker / to-tracker; real use needs platform-service + a token)
|
||||
AQ_TRACKER_API (=$AQ_TRACKER_API) AQ_TRACKER_TOKEN (bearer) AQ_PRODUCT_ID
|
||||
AQ_TRACKER_AUTO=1 to auto-echo outcomes on each transition (default OFF)
|
||||
AQ_TRACKER_CWD (cwd for tracker-derived jobs) AQ_TRACKER_API_CMD (test stub seam)
|
||||
label hints on an Item: engine-class:<x> profile:<x> priority:<x> cap:<token>
|
||||
EOF
|
||||
}
|
||||
|
||||
@ -1578,6 +1832,8 @@ main() {
|
||||
watch) cmd_watch "$@";;
|
||||
insights) cmd_insights "$@";;
|
||||
recover) cmd_recover "$@";;
|
||||
from-tracker) cmd_from_tracker "$@";;
|
||||
to-tracker) cmd_to_tracker "$@";;
|
||||
dash|dashboard) cmd_dash "$@";;
|
||||
stop) cmd_stop "$@";;
|
||||
logs) cmd_logs "$@";;
|
||||
|
||||
@ -11,7 +11,7 @@
|
||||
| Phase | Theme | Status | % | Gate |
|
||||
| ----- | ----- | ------ | - | ---- |
|
||||
| **0** | Baseline (today) | ✅ shipped | 100% | `selftest.sh` green |
|
||||
| **1** | Manifest + profiles + capabilities + tracker adapter (single host) | ◐ in progress | 80% | adapter e2e + selftest |
|
||||
| **1** | Manifest + profiles + capabilities + tracker adapter (single host) | ◐ in progress | 95% | adapter e2e + selftest |
|
||||
| **2** | Coordinator as platform-service module + Cosmos + multi-factory leasing | ☐ not started | 0% | fleet e2e + module tests |
|
||||
| **3** | Fleet control plane in tracker-web + DAG deps + budgets + scoring router | ☐ not started | 0% | web e2e + router tests |
|
||||
| **4** | Message bus + autoscaling + cross-OS capability marketplace | ☐ not started | 0% | load/chaos suite |
|
||||
@ -253,10 +253,10 @@ Three transports were evaluated. **Decision: platform-service-native coordinator
|
||||
**Layering:** tracker = *WHAT/WHY* (plan, intake, prioritize, roadmap, votes) · gigafactory = *HOW* (execute) · platform-service = shared brain · agent-queue runner = offline edge. Grounded in the real `tracker-service` model (`Item`: `type` bug/feature/**task**, `status` open/in_progress/done/closed/wont_fix, priority, labels, assignee, `source` incl. **auto_detected**, votes, comments, public roadmap) and the `tracker-web` `/api/tracker/[...path]` proxy pattern.
|
||||
|
||||
### Phase 1 — Adapter (no new infra)
|
||||
- [ ] **task → job**: a tracker `Item` of `type: task` (e.g. `assignee: @agent` or label `agent:run`) is exported to a job `.md` (manifest mapped: title/description → body, priority → priority, labels → capabilities/profile hints).
|
||||
- [ ] **job → tracker**: lifecycle events post back as **status updates + comments** — `building` → status `in_progress` + comment "started on factory X"; `shipped` → `done` + comment with commit SHAs / PR link / verify results; `failed` → comment with reason (status stays `in_progress` for human triage).
|
||||
- [ ] Idempotency: re-running the adapter for the same item doesn't create duplicate jobs (idempotency-key = item id + content hash).
|
||||
- [ ] Adapter is a thin script/CLI (`aq from-tracker ITEM-789`) + optional poller.
|
||||
- [x] **task → job**: a tracker `Item` of `type: task` (e.g. `assignee: @agent` or label `agent:run`) is exported to a job `.md` (manifest mapped: title/description → body, priority → priority, labels → capabilities/profile hints). *(P1-S4: `aq from-tracker`; labels `engine-class:`/`profile:`/`priority:`/`cap:` → frontmatter.)*
|
||||
- [x] **job → tracker**: lifecycle events post back as **status updates + comments** — `building` → status `in_progress` + comment "started on factory X"; `shipped` → `done` + comment with commit SHAs / PR link / verify results; `failed` → comment with reason (status stays `in_progress` for human triage). *(P1-S4: `aq to-tracker` PATCHes status + posts a metrics-only comment; one-way echo §24.5; never fatal. The items API has no blocked/failed status, so failures map to `wont_fix` by default — override via `AQ_TRACKER_STATUS_FAILED`.)*
|
||||
- [x] Idempotency: re-running the adapter for the same item doesn't create duplicate jobs (idempotency-key = item id + content hash). *(P1-S4: derived `idempotency-key: tracker-<id>` reuses Slice 1 dedupe; `to-tracker` is idempotent via `tracker_echoed`.)*
|
||||
- [x] Adapter is a thin script/CLI (`aq from-tracker ITEM-789`) + optional poller. *(P1-S4: `from-tracker`/`to-tracker` + opt-in `AQ_TRACKER_AUTO` auto-echo; a standalone poller is deferred.)*
|
||||
- **Acceptance:** filing a tracker task, marking it `agent:run`, results in a queued job; on ship, the item flips to `done` with a SHA comment.
|
||||
- **Verify gate:** adapter e2e against a tracker-service test instance (or mock); round-trip assertion.
|
||||
|
||||
@ -347,7 +347,9 @@ Each phase: **Goal → checklist → Exit criteria**. Don't start a phase until
|
||||
>
|
||||
> **Slice progress — P1-S3 (resilience & insights, single host):** crash recovery (`recover_orphans` + `aq recover`), git WIP checkpoint/resume (`aq/wip/<job>`), functional `retry` policy (backoff + `retries_exhausted`), and execution insights (`parse_usage`, per-run metrics in meta, `aq insights`, `status`/`dash` insights) are **done** — see §11/§25/§26.
|
||||
>
|
||||
> **Slice progress — P1-S2 (profiles + deps/DAG, single host):** the `profiles/` catalog + resolution (`fm_eff` inheritance with job>profile>default precedence, persona injection), the warn-only `allowed-scope` guardrail (`scope_check`/`path_in_scope`), and single-host `deps` (block-with-reason in selection, `status` surfacing, submit-time cycle detection) are **done** — see §5/§6. The tracker adapter and `budget.wall` remain **for later slices**.
|
||||
> **Slice progress — P1-S2 (profiles + deps/DAG, single host):** the `profiles/` catalog + resolution (`fm_eff` inheritance with job>profile>default precedence, persona injection), the warn-only `allowed-scope` guardrail (`scope_check`/`path_in_scope`), and single-host `deps` (block-with-reason in selection, `status` surfacing, submit-time cycle detection) are **done** — see §5/§6.
|
||||
>
|
||||
> **Slice progress — P1-S4 (tracker adapter, single host):** the task ↔ job round-trip is **done** (§10) — `aq from-tracker` materializes a job from a tracker Item (idempotent on `tracker-<id>`, label→manifest mapping), `aq to-tracker` echoes status + a metrics-only comment one-way (idempotent via `tracker_echoed`, never fatal), and opt-in `AQ_TRACKER_AUTO` auto-echoes on transitions. All HTTP is curl-only through one wrapper (test seam `AQ_TRACKER_API_CMD`). **This closes the Phase-1 §14 tracker-adapter item.** Remaining P1 extras: `budget.wall` (P1-S3 left it) and Node-`dash` surfacing of the new fields.
|
||||
|
||||
- [x] Extend `agent-queue.sh` frontmatter parsing for all new manifest fields (§5), defaulted + backward-compatible. *(P1-S1)*
|
||||
- [x] Add `profiles/` directory + profile resolution (persona injection, default verify/caps/scope) (§6). *(P1-S2)*
|
||||
@ -356,9 +358,9 @@ Each phase: **Goal → checklist → Exit criteria**. Don't start a phase until
|
||||
- [x] `deps` (DAG) blocking on a single host; `idempotency-key` dedupe on `add`. *(P1-S1 idempotency dedupe + P1-S2 `deps` blocking/cycle detection.)*
|
||||
- [ ] `retry` with backoff into `failed`/requeue; `budget.wall` enforced (extends `timeout`). *(P1-S3: `retry` with backoff + `retries_exhausted` DONE; `budget.wall` still pending.)*
|
||||
- [x] `allowed-scope` guardrail (warn-only this phase) + post-run diff report. *(P1-S2: `scope_check` WARN-only + `scope_warning=`.)*
|
||||
- [ ] **Tracker adapter** `aq from-tracker <ITEM>` + `aq to-tracker` event poster (§10 P1).
|
||||
- [ ] Dashboard shows profile + priority + capability tags + tracker-item link. *(P1-S1: `status` shows priority/profile/caps/tracker-item; Node `dash` surfacing pending.)*
|
||||
- [ ] Update `selftest.sh` with: manifest parse fixtures, profile resolution, priority order, dep-block, idempotency, adapter round-trip (mock). *(P1-S1 manifest/priority/idempotency + P1-S2 profile resolution/persona/scope/dep-block/cycle + P1-S3 resilience/insights; tracker adapter round-trip still pending.)*
|
||||
- [x] **Tracker adapter** `aq from-tracker <ITEM>` + `aq to-tracker` event poster (§10 P1). *(P1-S4: curl-only `tracker_api`; from-tracker materializes a job (idempotent), to-tracker echoes status+metrics one-way; opt-in `AQ_TRACKER_AUTO`. A standalone background poller is deferred to P2.)*
|
||||
- [ ] Dashboard shows profile + priority + capability tags + tracker-item link. *(P1-S1: `status` shows priority/profile/caps/tracker-item; P1-S4: status/insights also show last echoed tracker status; Node `dash` surfacing pending.)*
|
||||
- [x] Update `selftest.sh` with: manifest parse fixtures, profile resolution, priority order, dep-block, idempotency, adapter round-trip (mock). *(P1-S1 manifest/priority/idempotency + P1-S2 profile/persona/scope/dep-block/cycle + P1-S3 resilience/insights + P1-S4 tracker from/to round-trip via stub.)*
|
||||
- [x] Update README + this doc's progress table. *(P1-S1)*
|
||||
- **Exit criteria:** all boxes ✅; `selftest.sh` green; a tracker task → executed → tracker `done` with SHA comment, fully on one host; no regression to Phase-0 `.md` files.
|
||||
|
||||
|
||||
@ -585,4 +585,107 @@ else
|
||||
fi
|
||||
unset AGENT_QUEUE_PROFILES
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Phase 1 — Slice 4 cases (tracker adapter §10). No live service: a stub
|
||||
# replaces tracker_api via AQ_TRACKER_API_CMD, records calls, returns canned JSON.
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
trkstub="$tmp/trk-stub.sh"
|
||||
cat > "$trkstub" <<'STUBEOF'
|
||||
#!/usr/bin/env bash
|
||||
# tracker API stub: records "<method> <path> :: <body>" and returns canned output
|
||||
# (body line + HTTP code line), keyed off the method.
|
||||
[ -n "${AQ_STUB_CALLS:-}" ] && printf '%s %s :: %s\n' "$1" "$2" "$3" >> "$AQ_STUB_CALLS"
|
||||
case "$1" in
|
||||
GET) printf '%s\n%s\n' "${AQ_STUB_ITEM:-}" "${AQ_STUB_GET_CODE:-200}" ;;
|
||||
*) printf '%s\n%s\n' '{}' "${AQ_STUB_CODE:-200}" ;;
|
||||
esac
|
||||
STUBEOF
|
||||
chmod +x "$trkstub"
|
||||
export AQ_TRACKER_API_CMD="$trkstub"
|
||||
export AQ_TRACKER_CWD="$work"
|
||||
|
||||
# 26. from-tracker materializes an inbox job with the derived frontmatter + body.
|
||||
export AGENT_QUEUE_ROOT="$tmp/queue-ft1"
|
||||
"$AQ" init >/dev/null
|
||||
export AQ_STUB_CALLS="$tmp/ft1-calls.log"; : > "$AQ_STUB_CALLS"
|
||||
export AQ_STUB_ITEM='{"id":"T-1","productId":"p","type":"task","status":"open","priority":"medium","title":"Title One","description":"BODY-DESC-ALPHA","labels":[]}'
|
||||
"$AQ" from-tracker T-1 >/dev/null 2>&1
|
||||
ftf=$(find "$AGENT_QUEUE_ROOT/inbox" -maxdepth 1 -name '*.md' 2>/dev/null | head -1)
|
||||
if [ -n "$ftf" ] && grep -q '^tracker-item: T-1$' "$ftf" \
|
||||
&& grep -q '^idempotency-key: tracker-T-1$' "$ftf" && grep -q 'BODY-DESC-ALPHA' "$ftf"; then
|
||||
pass "from-tracker: materializes inbox job (tracker-item + idempotency-key + body)"
|
||||
else
|
||||
[ -n "$ftf" ] && cat "$ftf" >&2; fail "from-tracker did not create the expected inbox job"
|
||||
fi
|
||||
|
||||
# 27. from-tracker maps labels -> manifest frontmatter.
|
||||
export AGENT_QUEUE_ROOT="$tmp/queue-ft2"
|
||||
"$AQ" init >/dev/null
|
||||
export AQ_STUB_ITEM='{"id":"T-2","productId":"p","type":"task","status":"open","priority":"low","title":"Two","description":"desc two","labels":["engine-class:agentic-coder","priority:high","cap:os:mac"]}'
|
||||
"$AQ" from-tracker T-2 >/dev/null 2>&1
|
||||
ftf2=$(find "$AGENT_QUEUE_ROOT/inbox" -maxdepth 1 -name '*.md' 2>/dev/null | head -1)
|
||||
if grep -q '^engine-class: agentic-coder$' "$ftf2" && grep -q '^priority: high$' "$ftf2" \
|
||||
&& grep -q '^capabilities: \[os:mac\]$' "$ftf2"; then
|
||||
pass "from-tracker: label mapping (engine-class/priority/cap) -> frontmatter"
|
||||
else
|
||||
cat "$ftf2" >&2; fail "from-tracker label mapping incorrect"
|
||||
fi
|
||||
|
||||
# 28. from-tracker is idempotent on the derived key (no duplicate enqueue).
|
||||
"$AQ" from-tracker T-2 >/dev/null 2>&1
|
||||
n=$(find "$AGENT_QUEUE_ROOT/inbox" -maxdepth 1 -name '*.md' 2>/dev/null | wc -l | tr -d ' ')
|
||||
[ "$n" = "1" ] && pass "from-tracker: idempotent (T-2 twice -> one job)" \
|
||||
|| fail "from-tracker not idempotent (inbox=$n)"
|
||||
|
||||
# 29. to-tracker echoes a shipped outcome: PATCH status=done + metrics comment,
|
||||
# and NEVER sends the prompt body.
|
||||
export AGENT_QUEUE_ROOT="$tmp/queue-tt"
|
||||
"$AQ" init >/dev/null
|
||||
export AQ_STUB_CALLS="$tmp/tt-calls.log"; : > "$AQ_STUB_CALLS"
|
||||
printf '%s\n' 'job=jt' 'tracker_item=T-9' 'result=shipped' 'attempts=1' 'duration_s=5' \
|
||||
'tokens_in=10' 'tokens_out=3' 'cost_usd=0.001' > "$AGENT_QUEUE_ROOT/.state/jt.meta"
|
||||
printf 'SECRET-PROMPT-SENTINEL\n' > "$AGENT_QUEUE_ROOT/.state/jt.body.md"
|
||||
"$AQ" to-tracker jt >/dev/null 2>&1
|
||||
if grep -q 'PATCH /api/items/T-9/status :: {"status":"done"}' "$AQ_STUB_CALLS" \
|
||||
&& grep -q 'POST /api/items/T-9/comments' "$AQ_STUB_CALLS" \
|
||||
&& ! grep -q 'SECRET-PROMPT-SENTINEL' "$AQ_STUB_CALLS"; then
|
||||
pass "to-tracker: shipped -> PATCH status=done + metrics comment; no prompt body sent"
|
||||
else
|
||||
cat "$AQ_STUB_CALLS" >&2; fail "to-tracker echo incorrect / leaked body"
|
||||
fi
|
||||
|
||||
# 30. to-tracker is idempotent: a second call for an unchanged outcome is a no-op.
|
||||
: > "$AQ_STUB_CALLS"
|
||||
"$AQ" to-tracker jt >/dev/null 2>&1
|
||||
[ ! -s "$AQ_STUB_CALLS" ] && pass "to-tracker: idempotent (unchanged outcome -> no PATCH/comment)" \
|
||||
|| { cat "$AQ_STUB_CALLS" >&2; fail "to-tracker not idempotent (made calls on unchanged outcome)"; }
|
||||
|
||||
# 31. echo failure is non-fatal: a 500 logs an error, exits 0, leaves job state intact.
|
||||
export AGENT_QUEUE_ROOT="$tmp/queue-tt6"
|
||||
"$AQ" init >/dev/null
|
||||
printf '%s\n' 'job=jf' 'tracker_item=T-8' 'result=shipped' > "$AGENT_QUEUE_ROOT/.state/jf.meta"
|
||||
printf '%s\n' '---' 'tracker-item: T-8' '---' '' '# x' > "$AGENT_QUEUE_ROOT/shipped/jf.md"
|
||||
AQ_STUB_CODE=500 "$AQ" to-tracker jf >/dev/null 2>&1; rc=$?
|
||||
if [ "$rc" = "0" ] && [ -f "$AGENT_QUEUE_ROOT/shipped/jf.md" ] \
|
||||
&& [ -z "$(metaval "$AGENT_QUEUE_ROOT/.state/jf.meta" tracker_echoed)" ]; then
|
||||
pass "to-tracker: HTTP 500 is non-fatal (exit 0, job unchanged, not marked echoed)"
|
||||
else
|
||||
fail "to-tracker 500 was not handled non-fatally (rc=$rc)"
|
||||
fi
|
||||
|
||||
# 32. auto-echo (AQ_TRACKER_AUTO=1): a tracker-derived job run echoes automatically.
|
||||
export AGENT_QUEUE_ROOT="$tmp/queue-auto"
|
||||
"$AQ" init >/dev/null
|
||||
export AQ_STUB_CALLS="$tmp/auto-calls.log"; : > "$AQ_STUB_CALLS"
|
||||
printf '%s\n' '---' 'engine: devin' "cwd: $work" 'yolo: true' 'tracker-item: T-7' '---' '' '# auto task' \
|
||||
> "$AGENT_QUEUE_ROOT/inbox/autojob.md"
|
||||
AQ_TRACKER_AUTO=1 DEVIN_BIN="$stub" "$AQ" run --once >/dev/null 2>&1
|
||||
if grep -q 'PATCH /api/items/T-7/status' "$AQ_STUB_CALLS" 2>/dev/null \
|
||||
&& ls "$AGENT_QUEUE_ROOT"/review/autojob.md >/dev/null 2>&1; then
|
||||
pass "auto-echo: AQ_TRACKER_AUTO=1 echoes a transition during run (job still reaches review/)"
|
||||
else
|
||||
cat "$AQ_STUB_CALLS" 2>/dev/null >&2; fail "auto-echo did not fire / job did not complete"
|
||||
fi
|
||||
unset AQ_TRACKER_API_CMD AQ_TRACKER_CWD AQ_STUB_CALLS AQ_STUB_ITEM AQ_STUB_CODE AQ_STUB_GET_CODE
|
||||
|
||||
echo "self-test PASS"
|
||||
|
||||
Loading…
Reference in New Issue
Block a user