From 3224199894ba24c6de6f095d7d1a4f84b3ef0978 Mon Sep 17 00:00:00 2001 From: saravanakumardb1 Date: Thu, 28 May 2026 18:05:55 -0700 Subject: [PATCH] feat(gitea): reproducible Actions runner registration + harden runner config - add scripts/gitea/register-runner.sh (idempotent register, host/docker modes, capacity arg, admin-API registration token, --force re-register) - GITEA_VM_SETUP.md Step 11: runner install/register, host-vs-docker tradeoffs, token externalization (env_file), concurrency (capacity), token rotation, end-to-end CI verification - document runner registration + secrets in persist/ephemeral table Live runner hardened separately: capacity 1->2, GITEA_NPM_TOKEN moved from inline config.yaml to chmod-600 runner.env via env_file. --- docs/runbooks/GITEA_VM_SETUP.md | 144 +++++++++++++++++++++++++++++-- scripts/gitea/register-runner.sh | 106 +++++++++++++++++++++++ 2 files changed, 241 insertions(+), 9 deletions(-) create mode 100755 scripts/gitea/register-runner.sh diff --git a/docs/runbooks/GITEA_VM_SETUP.md b/docs/runbooks/GITEA_VM_SETUP.md index 1b2996ea..7bbb2612 100644 --- a/docs/runbooks/GITEA_VM_SETUP.md +++ b/docs/runbooks/GITEA_VM_SETUP.md @@ -1,6 +1,6 @@ # Gitea Cloud VM Setup — Runbook -> **Status:** Active runbook · **Last verified:** 2026-05-27 +> **Status:** Active runbook · **Last verified:** 2026-05-28 > **Use this when:** You have provisioned a cloud VM (Azure / wherever), Gitea > is installed and running on `:3300`, repos are cloned, and you need to wire > the npm registry end-to-end with your laptop. @@ -335,6 +335,129 @@ source "$HOME/code/mygh/learning_ai_common_plat/scripts/switch-network.sh" --- +## Step 11 — Gitea Actions runner (CI) + +The npm registry (Steps 1–10) is independent of CI. To run the `docker-lint` +job (and the rest of each repo's `.gitea/workflows/*.yml`) you need an +**Actions runner** registered against Gitea. This section makes that +reproducible — the original runner was registered by hand. + +### 11.1 — Enable Actions on the instance + +In `app.ini` (inside the Gitea container/conf dir): + +```ini +[actions] +ENABLED = true +``` + +Then `sudo docker restart gitea`. Confirm with +`curl -fsS http://localhost:3300/api/v1/version` and that the repo Settings → +Actions toggle is available. + +### 11.2 — Install and register the runner + +```bash +# macOS host runner (laptop) — install +brew install act_runner + +# Register reproducibly (fetches a registration token via the admin API, +# registers with the agreed labels + capacity). Host mode is the default. +GITEA_ADMIN_USER=gitea-admin GITEA_ADMIN_PASS='' \ + bash scripts/gitea/register-runner.sh --name bytelyst-mac --capacity 2 + +# Containerized runner (better isolation; requires Docker on the host): +GITEA_ADMIN_USER=gitea-admin GITEA_ADMIN_PASS='' \ + bash scripts/gitea/register-runner.sh --mode docker --capacity 2 +``` + +`register-runner.sh` is idempotent: if a runner is already registered it +prints the current identity and exits. Pass `--force` to re-register (this +invalidates the old runner row in Gitea). + +### 11.3 — Host mode vs. containerized mode + +| | **Host mode** (`ubuntu-latest:host`) | **Docker mode** (`ubuntu-latest:docker://…`) | +| --------------- | ------------------------------------ | ---------------------------------------------- | +| Isolation | None — jobs run directly on macOS | Each job in a fresh container | +| Speed | Fast (no image pull) | Slower first run (pulls `catthehacker/ubuntu`) | +| Reproducibility | Depends on host toolchain | Pinned image, matches GitHub closely | +| Best for | Single-operator laptop / corp proxy | Shared/VM runners, untrusted PRs | + +We run **host mode** on the laptop because the corp proxy + Docker-in-Docker is +fragile, and the jobs are trusted (own repos). Prefer **docker mode** on the VM. + +### 11.4 — Secrets: never inline the token + +The npm token must **not** live inline in `config.yaml`. Externalise it into a +gitignored `runner.env` referenced by `runner.env_file`: + +```yaml +# /opt/homebrew/etc/act_runner/config.yaml +runner: + capacity: 2 # parallel jobs + env_file: '/opt/homebrew/etc/act_runner/runner.env' + envs: + # GITEA_NPM_TOKEN is loaded from env_file — never inline here. + NODE_ENV: test +``` + +```bash +# /opt/homebrew/etc/act_runner/runner.env (chmod 600, never committed) +GITEA_NPM_TOKEN= +``` + +After editing config, reload the daemon: + +```bash +brew services restart act_runner # or, if brew name mismatches: +launchctl kickstart -k gui/$(id -u)/homebrew.mxcl.act_runner +tail -f /opt/homebrew/var/log/act_runner.log # expect "declare successfully" +``` + +### 11.5 — Concurrency + +`runner.capacity` controls parallel jobs on one runner. With `capacity: 1` +the lightweight `docker-lint` job queues behind slow backend/web/mobile/E2E +jobs (observed: ~13 min wait). `capacity: 2` lets `docker-lint` run alongside +one heavy job. For more parallelism, register additional runners rather than +pushing capacity high on a single laptop. + +### 11.6 — Runner token rotation + +The **registration token** (used once at register time) is separate from the +**npm token** (used by jobs). To rotate: + +```bash +# Registration token — just re-register; old token is single-use anyway: +bash scripts/gitea/register-runner.sh --force --name bytelyst-mac + +# npm token — rotate via the existing helper, then update runner.env: +bash scripts/gitea/token.sh rotate +TOKEN=$(cat ~/.gitea_npm_token) +printf 'GITEA_NPM_TOKEN=%s\n' "$TOKEN" > /opt/homebrew/etc/act_runner/runner.env +chmod 600 /opt/homebrew/etc/act_runner/runner.env +brew services restart act_runner +``` + +### 11.7 — Verify CI end-to-end + +Push any repo that has a `docker-lint` job, then: + +```bash +PAT=$(cat ~/.gitea_c5_pat) +R=learning_ai_clock +RID=$(curl -s -H "Authorization: token $PAT" \ + "http://localhost:3300/api/v1/repos/learning_ai_user/$R/actions/runs?limit=1" \ + | python3 -c "import json,sys; print(json.load(sys.stdin)['workflow_runs'][0]['id'])") +curl -s -H "Authorization: token $PAT" \ + "http://localhost:3300/api/v1/repos/learning_ai_user/$R/actions/runs/$RID/jobs" \ + | python3 -c "import json,sys; [print(j['status'], j.get('conclusion'), j['name']) for j in json.load(sys.stdin)['jobs']]" +# Expect: completed success Docker lint — gitea-doctor + docker-doctor +``` + +--- + ## Troubleshooting ### Doctor reports `STALE TOKEN: env GITEA_NPM_TOKEN ≠ file` @@ -381,14 +504,16 @@ sudo docker start gitea ## What's persistent vs. ephemeral -| Item | Where | Survives VM reboot? | Survives VM rebuild? | -| ------------------------- | ------------------------------------------------ | ------------------- | -------------------------- | -| Gitea database | `/var/lib/gitea/data/gitea.db` | ✅ | ❌ (snapshot the disk) | -| Published packages | `/var/lib/gitea/data/packages/` | ✅ | ❌ (re-publish via Step 7) | -| Admin/npm users | inside Gitea DB | ✅ | ❌ (re-run Steps 1-2) | -| NPM tokens | inside Gitea DB + your `~/.gitea_npm_token_home` | ✅ | ❌ (re-run Step 3) | -| `~/.gitea_vm_host` | your laptop | ✅ | n/a | -| `~/.gitea_npm_token_home` | your laptop | ✅ | n/a | +| Item | Where | Survives VM reboot? | Survives VM rebuild? | +| --------------------------- | ------------------------------------------------ | ------------------- | -------------------------------- | +| Gitea database | `/var/lib/gitea/data/gitea.db` | ✅ | ❌ (snapshot the disk) | +| Published packages | `/var/lib/gitea/data/packages/` | ✅ | ❌ (re-publish via Step 7) | +| Admin/npm users | inside Gitea DB | ✅ | ❌ (re-run Steps 1-2) | +| NPM tokens | inside Gitea DB + your `~/.gitea_npm_token_home` | ✅ | ❌ (re-run Step 3) | +| `~/.gitea_vm_host` | your laptop | ✅ | n/a | +| `~/.gitea_npm_token_home` | your laptop | ✅ | n/a | +| Actions runner registration | Gitea DB + `.runner` file | ✅ | ❌ (re-run `register-runner.sh`) | +| Runner secrets | `act_runner/runner.env` (chmod 600) | ✅ | ❌ (recreate from token) | For VM rebuilds: snapshot `/var/lib/gitea` to Azure Disk Snapshot weekly, restore on rebuild. Avoids re-running Steps 1-7. @@ -399,6 +524,7 @@ restore on rebuild. Avoids re-running Steps 1-7. - `scripts/gitea/doctor.sh` — pre-flight validation (run before every deploy) - `scripts/gitea/token.sh` — token rotation helper +- `scripts/gitea/register-runner.sh` — reproducible Actions runner registration (Step 11) - `scripts/gitea/bootstrap-vm.sh` — automates Steps 1-3 on a fresh VM - `scripts/switch-network.sh` — exports `GITEA_NPM_*` env vars per network - `docker-build-optimization-roadmap.md` (in `learning_ai_devops_tools/docs/`) — diff --git a/scripts/gitea/register-runner.sh b/scripts/gitea/register-runner.sh new file mode 100755 index 00000000..2be640e0 --- /dev/null +++ b/scripts/gitea/register-runner.sh @@ -0,0 +1,106 @@ +#!/usr/bin/env bash +# +# register-runner.sh — Reproducibly register (or re-register) a Gitea Actions +# runner against the local/VM Gitea instance. +# +# Why this exists: the runner was originally registered by hand, which is not +# reproducible for the next operator. This script captures the canonical flow: +# fetch a registration token from Gitea, register act_runner with the agreed +# labels + capacity, and externalise the npm token into a gitignored env_file +# (never inline in config.yaml). +# +# Usage: +# GITEA_ADMIN_USER=gitea-admin GITEA_ADMIN_PASS=... bash register-runner.sh +# bash register-runner.sh --instance http://localhost:3300 --name bytelyst-mac +# bash register-runner.sh --mode docker # containerized labels +# bash register-runner.sh --capacity 2 # parallel jobs +# +# Idempotent: if a runner is already registered (.runner exists), it prints the +# current registration and exits unless --force is passed. +set -euo pipefail + +# ── defaults ─────────────────────────────────────────────────────────────── +INSTANCE="${GITEA_INSTANCE:-http://localhost:3300}" +RUNNER_NAME="${RUNNER_NAME:-$(hostname -s)}" +CAPACITY="${RUNNER_CAPACITY:-2}" +MODE="host" # host | docker +FORCE=0 +CONFIG_DIR="${ACT_RUNNER_CONFIG_DIR:-/opt/homebrew/etc/act_runner}" +RUNNER_FILE="${ACT_RUNNER_FILE:-/opt/homebrew/var/lib/act_runner/.runner}" +ENV_FILE="${ACT_RUNNER_ENV_FILE:-$CONFIG_DIR/runner.env}" + +# Docker image used when MODE=docker (mirrors GitHub's ubuntu-latest closely). +DOCKER_IMAGE="${RUNNER_DOCKER_IMAGE:-catthehacker/ubuntu:act-latest}" + +while [ $# -gt 0 ]; do + case "$1" in + --instance) INSTANCE="$2"; shift 2 ;; + --name) RUNNER_NAME="$2"; shift 2 ;; + --capacity) CAPACITY="$2"; shift 2 ;; + --mode) MODE="$2"; shift 2 ;; + --force) FORCE=1; shift ;; + -h|--help) grep '^#' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; + *) echo "unknown arg: $1" >&2; exit 2 ;; + esac +done + +command -v act_runner >/dev/null 2>&1 || { + echo "✗ act_runner not on PATH. Install: brew install act_runner" >&2 + exit 1 +} + +# ── labels per mode ────────────────────────────────────────────────────────── +if [ "$MODE" = "docker" ]; then + LABELS="ubuntu-latest:docker://$DOCKER_IMAGE,ubuntu-22.04:docker://$DOCKER_IMAGE,self-hosted:host" +else + LABELS="ubuntu-latest:host,macos-latest:host,macos-15:host,self-hosted:host" +fi + +echo "── Gitea runner registration ──" +echo " instance : $INSTANCE" +echo " name : $RUNNER_NAME" +echo " mode : $MODE" +echo " capacity : $CAPACITY" +echo " labels : $LABELS" + +# ── already registered? ────────────────────────────────────────────────────── +if [ -f "$RUNNER_FILE" ] && [ "$FORCE" -ne 1 ]; then + echo "" + echo "✓ runner already registered ($RUNNER_FILE). Current identity:" + python3 - "$RUNNER_FILE" <<'PY' 2>/dev/null || cat "$RUNNER_FILE" +import json, sys +d = json.load(open(sys.argv[1])) +for k in ("id", "uuid", "name", "address", "labels", "ephemeral"): + print(f" {k}: {d.get(k)}") +PY + echo "" + echo " Pass --force to re-register (this invalidates the old runner)." + exit 0 +fi + +# ── fetch a registration token ─────────────────────────────────────────────── +# Admin-scoped: GET /admin/runners/registration-token (Gitea >= 1.20). +: "${GITEA_ADMIN_USER:?set GITEA_ADMIN_USER}" +: "${GITEA_ADMIN_PASS:?set GITEA_ADMIN_PASS}" + +REG_TOKEN=$(curl -fsS -u "$GITEA_ADMIN_USER:$GITEA_ADMIN_PASS" \ + "$INSTANCE/api/v1/admin/runners/registration-token" \ + | python3 -c "import json,sys; print(json.load(sys.stdin)['token'])") +[ -n "$REG_TOKEN" ] || { echo "✗ could not fetch registration token" >&2; exit 1; } +echo "✓ registration token obtained" + +# ── register ──────────────────────────────────────────────────────────────── +[ "$FORCE" -eq 1 ] && rm -f "$RUNNER_FILE" +act_runner register \ + --no-interactive \ + --instance "$INSTANCE" \ + --token "$REG_TOKEN" \ + --name "$RUNNER_NAME" \ + --labels "$LABELS" \ + --config "$CONFIG_DIR/config.yaml" +echo "✓ runner registered" + +echo "" +echo "Next: externalise secrets into $ENV_FILE (see GITEA_VM_SETUP.md §11)," +echo "set runner.capacity: $CAPACITY in config.yaml, then:" +echo " brew services restart act_runner"