# Hostinger VM — GitHub Actions Self-Hosted Runner Setup

> ⚠️ **PLAN B — not the recommended path.**
> The canonical path uses **Gitea Actions** (which you already have configured in `.gitea/workflows/` across all repos). See [`HOSTINGER_GITEA_ACT_RUNNER_SETUP.md`](./HOSTINGER_GITEA_ACT_RUNNER_SETUP.md) for the recommended setup, and [`GITEA_PACKAGES_PUBLISH_WORKFLOW.md`](./GITEA_PACKAGES_PUBLISH_WORKFLOW.md) for the publish pipeline.
>
> Why this doc exists: it documents a fully valid alternative — a self-hosted GitHub Actions runner on the Hostinger VM. Keep this if you ever decide to make GitHub Actions the canonical CI driver (e.g., to take advantage of the GitHub Actions ecosystem). For now, Gitea Actions is simpler because:
>
> - Workflows already exist in `.gitea/workflows/` for all 20+ repos.
> - No GitHub Organization migration needed (avoids Vercel/Netlify pricing concerns).
> - Both Gitea instances (corp Mac local + Hostinger) can publish independently from the same git tag, giving byte-identical tarballs without any sync script.
>
> **Delegation prompt for the Codex agent running on the Hostinger VM.**
> Read top-to-bottom before executing. Stop and ask the human if any pre-flight check fails or any deliverable is unclear.

---

## 1. Goal

Set up a GitHub Actions self-hosted runner on the Hostinger VM that can:

1. Receive workflow triggers from **all 20+ `@bytelyst/*` repos** (see §8 for the org-vs-repo registration decision).
2. Build `@bytelyst/*` npm packages from tagged releases.
3. Publish them to **the local Gitea instance on this VM** (`http://localhost:3300/api/packages/bytelyst/npm/`).
4. Upload the same tarballs as **GitHub Release assets** so a corp-network Mac can sync them into its own local Gitea (via the separate `bytelyst-sync` script described in a follow-up prompt).

Self-hosted on Hostinger beats GitHub-hosted runners because:

- No GitHub Actions minute cap.
- Gitea is on `localhost` from this VM → zero-latency publish, no public TLS needed.
- VM is always on; runner is reachable indefinitely.

### Multi-repo registration decision

**Decide before Step 2** whether this runner serves a single repo or all 20+:

| Approach                             | Registration scope        | Use when                                                                                                       |
| ------------------------------------ | ------------------------- | -------------------------------------------------------------------------------------------------------------- |
| **Repo-level** (default in this doc) | Single repo URL           | You're validating the runner first, or you only have 1–2 repos publishing packages                             |
| **Org-level** (recommended at scale) | A GitHub Organization URL | You've migrated 20+ repos under one org — see §8 for migration steps. One registration, all org repos eligible |

The install steps are identical — only the `--url` flag in `config.sh` (Step 3) changes.

---

## 2. Pre-flight checks (run first, do not skip)

```bash
# 1. Confirm Linux VM
hostname && uname -a       # Expected: Linux

# 2. Confirm Gitea is running locally
curl -s -o /dev/null -w "%{http_code}\n" http://localhost:3300/   # Expected: 200 or 302

# 3. Confirm Gitea npm registry endpoint reachable
curl -s -o /dev/null -w "%{http_code}\n" http://localhost:3300/api/packages/bytelyst/npm/    # Expected: 200 or 401

# 4. Confirm Node 20 and pnpm 9 installable
node --version 2>/dev/null || echo "Node not installed (will install in Step 5)"
pnpm --version 2>/dev/null || echo "pnpm not installed (will install in Step 5)"

# 5. Confirm gh CLI exists
gh --version 2>/dev/null || echo "gh CLI not installed (will install in Step 5)"

# 6a. Detect architecture (used for runner tarball selection in Step 3)
ARCH=$(uname -m)
case "$ARCH" in
  x86_64)  export RUNNER_ARCH="linux-x64";;
  aarch64) export RUNNER_ARCH="linux-arm64";;
  *) echo "Unsupported arch: $ARCH — STOP and report"; ;;
esac
echo "Will install runner for: $RUNNER_ARCH"

# 6b. Disk free
df -h /     # Need ~5 GB headroom

# 7. Confirm no existing runner
ls -la ~/actions-runner 2>/dev/null && echo "Runner dir exists — STOP and confirm with human" || echo "No existing runner"

# 8. Confirm github.com reachable (both API and download CDN)
curl -s -o /dev/null -w "api.github.com: %{http_code}\n" https://api.github.com/
curl -s -o /dev/null -w "objects.githubusercontent.com: %{http_code}\n" -L \
  https://objects.githubusercontent.com/   # runner tarball download host
# Expected: 200 / 403 (403 from CDN root is normal; what matters is non-network-error)

# 9. Confirm the Gitea token file exists somewhere on this VM
sudo find /home /root -maxdepth 3 -name ".gitea_npm_token" 2>/dev/null | head -5
# Expected: at least one path. Note the owning user — needed in Step 6.

# 10. Confirm gh CLI is auth'd as saravanakumardb1 (needed for registration token)
gh auth status 2>&1 | grep -E "Logged in to|saravanakumardb1" | head -5
# If not logged in as saravanakumardb1, run: gh auth login (and pick saravanakumardb1)
```

If any check fails or surprises you, **stop and report back** before proceeding.

---

## 3. What you'll create

| Item                       | Path/Identifier                                                                        |
| -------------------------- | -------------------------------------------------------------------------------------- |
| Dedicated Linux user       | `gha-runner`                                                                           |
| Runner installation        | `/home/gha-runner/actions-runner/`                                                     |
| systemd service            | `actions.runner.saravanakumardb1-learning_ai_common_plat.hostinger-bytelyst-1.service` |
| Runner labels              | `self-hosted, linux, x64, hostinger, bytelyst`                                         |
| Gitea publish token (copy) | `/home/gha-runner/.gitea_npm_token` (mode 600)                                         |
| Smoke-test workflow        | `.github/workflows/runner-smoke.yml` (commit on a branch)                              |
| E2E test workflow          | `.github/workflows/runner-e2e-publish.yml` (commit on a branch)                        |

---

## 4. Installation

### Step 1 — Create the dedicated runner user (idempotent)

```bash
if ! getent passwd gha-runner >/dev/null; then
  sudo useradd -m -s /bin/bash gha-runner
  echo "Created gha-runner user"
else
  echo "gha-runner user already exists — skipping useradd"
fi

# Add to docker group only if docker is on this host
if getent group docker >/dev/null; then
  sudo usermod -aG docker gha-runner
  echo "Added gha-runner to docker group"
fi

id gha-runner
```

### Step 2 — Get a GitHub runner registration token (one-time, ~1h TTL)

**Preferred (via `gh` CLI auth'd as `saravanakumardb1`):**

```bash
gh api -X POST /repos/saravanakumardb1/learning_ai_common_plat/actions/runners/registration-token --jq .token
```

**Alternative (browser):** Open `https://github.com/saravanakumardb1/learning_ai_common_plat/settings/actions/runners/new?arch=x64&os=linux` and copy the token from the `./config.sh` command shown.

Hold the token in shell memory only:

```bash
read -s RUNNER_TOKEN   # paste, press Enter (no echo)
```

### Step 3 — Download, verify, configure the runner

```bash
# Query the latest runner version (don't hardcode)
LATEST=$(gh api /repos/actions/runner/releases/latest --jq '.tag_name' | sed 's/^v//')
echo "Latest runner version: $LATEST"
# As of writing this doc: 2.319.1. If LATEST is wildly different, STOP and confirm with human.

sudo -iu gha-runner bash <<EOF
mkdir -p ~/actions-runner && cd ~/actions-runner
RUNNER_VERSION="$LATEST"
RUNNER_ARCH="${RUNNER_ARCH}"   # from pre-flight 5a
TARBALL="actions-runner-\${RUNNER_ARCH}-\${RUNNER_VERSION}.tar.gz"

# Download tarball
curl -fSL -o "\$TARBALL" \
  "https://github.com/actions/runner/releases/download/v\${RUNNER_VERSION}/\$TARBALL"

# Download checksum manifest and verify (GitHub publishes SHA256 alongside each release)
EXPECTED_SHA=\$(gh api /repos/actions/runner/releases/tags/v\${RUNNER_VERSION} \
  --jq ".body" | grep -oE "\b[0-9a-f]{64}\s+\$TARBALL\b" | awk '{print \$1}')
ACTUAL_SHA=\$(sha256sum "\$TARBALL" | awk '{print \$1}')

if [ "\$EXPECTED_SHA" != "\$ACTUAL_SHA" ]; then
  echo "FAIL: SHA mismatch"
  echo "  Expected: \$EXPECTED_SHA"
  echo "  Actual:   \$ACTUAL_SHA"
  exit 1
fi
echo "PASS: tarball SHA verified"

tar xzf "./\$TARBALL"
EOF
```

Note: if `gh api` parsing of the SHA from the release body fails (GitHub sometimes changes release-note formatting), fall back to the official hashes page:
`https://github.com/actions/runner/releases/tag/v<version>`. If you can't verify the SHA, STOP and report — don't run unverified binaries.

Register the runner. **Choose the URL based on your scope decision (§1):**

```bash
# OPTION A — repo-level (default during validation)
REGISTRATION_URL="https://github.com/saravanakumardb1/learning_ai_common_plat"

# OPTION B — org-level (once you've migrated to an org per §8)
# REGISTRATION_URL="https://github.com/<your-org-name>"

sudo -u gha-runner -E -i bash -c "
cd ~/actions-runner && \
./config.sh \
  --url $REGISTRATION_URL \
  --token $RUNNER_TOKEN \
  --name hostinger-bytelyst-1 \
  --labels self-hosted,linux,x64,hostinger,bytelyst \
  --work _work \
  --replace \
  --unattended
"

unset RUNNER_TOKEN
```

Verify in GitHub UI: runner should show "Idle" green under `Settings → Actions → Runners`.

### Step 4 — Install as a systemd service

```bash
sudo bash -c "
cd /home/gha-runner/actions-runner && \
./svc.sh install gha-runner && \
./svc.sh start
"

SVC_NAME='actions.runner.saravanakumardb1-learning_ai_common_plat.hostinger-bytelyst-1.service'
sudo systemctl status "$SVC_NAME"
sudo journalctl -u "$SVC_NAME" -n 30 --no-pager
# Expected: "active (running)" + "Listening for Jobs"
```

### Step 5 — Install Node 20, pnpm 9, gh CLI system-wide

```bash
# Node 20 via Nodesource
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo bash -
sudo apt-get install -y nodejs

# gh CLI (if not already)
sudo apt-get install -y gh   # or follow https://cli.github.com/manual/installation if 'gh' isn't in the apt repos

# pnpm
sudo npm install -g pnpm@9

# Verify all reachable from the runner user
sudo -u gha-runner bash -c 'node --version && pnpm --version && gh --version'
```

### Step 6 — Give the runner access to the local Gitea publish token

```bash
# Identify the source token file (from pre-flight check #9)
SRC_TOKEN=/home/<original-user>/.gitea_npm_token

# Copy and lock down
sudo cp "$SRC_TOKEN" /home/gha-runner/.gitea_npm_token
sudo chown gha-runner:gha-runner /home/gha-runner/.gitea_npm_token
sudo chmod 600 /home/gha-runner/.gitea_npm_token

# Verify
sudo -u gha-runner bash -c 'wc -c < ~/.gitea_npm_token && stat -c "%a %U:%G" ~/.gitea_npm_token'
# Expected: nonzero byte count, mode 600, owner gha-runner:gha-runner
```

---

## 5. Smoke test (basic — runner picks up jobs)

Create branch `runner/smoke` in `learning_ai_common_plat` with the workflow below.

```bash
cd ~/code/mygh/learning_ai_common_plat
git checkout -b runner/smoke
mkdir -p .github/workflows
# (paste workflow below into .github/workflows/runner-smoke.yml)
git add .github/workflows/runner-smoke.yml
git commit -m "ci: add self-hosted runner smoke-test workflow"
git push origin runner/smoke
```

```yaml
# .github/workflows/runner-smoke.yml
name: Runner Smoke Test
on:
  workflow_dispatch:
  push:
    branches: [runner/smoke]
jobs:
  smoke:
    runs-on: [self-hosted, linux, hostinger]
    steps:
      - run: echo "host=$(hostname) user=$(whoami) cwd=$(pwd)"
      - run: node --version && pnpm --version && gh --version
      - run: |
          echo "Gitea health:"
          curl -s -o /dev/null -w "  http://localhost:3300/  → %{http_code}\n" http://localhost:3300/
          curl -s -o /dev/null -w "  /api/packages/bytelyst/npm/ → %{http_code}\n" \
            http://localhost:3300/api/packages/bytelyst/npm/
      - run: |
          if [ -f ~/.gitea_npm_token ]; then
            echo "Gitea token present, $(wc -c < ~/.gitea_npm_token) bytes, mode $(stat -c %a ~/.gitea_npm_token)"
          else
            echo "ERROR: Gitea token missing"
            exit 1
          fi
```

Trigger from GitHub UI (Actions tab → Runner Smoke Test → Run workflow on `runner/smoke`). All steps must pass.

---

## 6. End-to-end validation (CRITICAL — proves the actual use case)

The smoke test only proves the runner can execute. The **E2E test** proves the **whole publish pipeline** works: package builds, publishes to local Gitea, uploads to GitHub Releases, and is installable on a different machine.

### E2E test workflow

Create a **temporary test package** in `learning_ai_common_plat` (will be removed after validation):

```bash
# On Hostinger or from human's machine — does NOT need to run on the runner
mkdir -p packages/_runner-e2e-test
cat > packages/_runner-e2e-test/package.json <<'EOF'
{
  "name": "@bytelyst/_runner-e2e-test",
  "version": "0.0.1",
  "description": "Throwaway package for E2E validating the Hostinger runner. Safe to delete after validation.",
  "main": "index.js",
  "files": ["index.js"]
}
EOF
echo "module.exports = { ok: true, builtAt: new Date().toISOString() };" \
  > packages/_runner-e2e-test/index.js
git checkout -b runner/e2e
git add packages/_runner-e2e-test/
git commit -m "test: add throwaway package for runner E2E validation"
git push origin runner/e2e
```

Create the E2E workflow on the same branch:

```yaml
# .github/workflows/runner-e2e-publish.yml
name: Runner E2E — publish + release

on:
  workflow_dispatch:
    inputs:
      version:
        description: 'Version to publish (semver)'
        required: true
        default: '0.0.1-e2e.1'

jobs:
  publish:
    runs-on: [self-hosted, linux, hostinger]
    permissions:
      contents: write # for GitHub Release creation
    steps:
      - uses: actions/checkout@v4

      - name: Set test version
        working-directory: packages/_runner-e2e-test
        run: |
          npm version "${{ inputs.version }}" --no-git-tag-version --allow-same-version
          cat package.json

      - name: Configure pnpm registry for Gitea
        working-directory: packages/_runner-e2e-test
        run: |
          cat > .npmrc <<NPMRC
          @bytelyst:registry=http://localhost:3300/api/packages/bytelyst/npm/
          //localhost:3300/api/packages/bytelyst/npm/:_authToken=$(cat ~/.gitea_npm_token)
          NPMRC
          echo ".npmrc written:"
          sed 's|_authToken=.*|_authToken=***|' .npmrc

      - name: Publish to local Gitea
        working-directory: packages/_runner-e2e-test
        run: pnpm publish --no-git-checks --registry http://localhost:3300/api/packages/bytelyst/npm/

      - name: Pack tarball for GitHub Release
        working-directory: packages/_runner-e2e-test
        run: |
          pnpm pack --pack-destination /tmp
          ls -la /tmp/bytelyst-_runner-e2e-test-*.tgz

      - name: Create GitHub Release with tarball
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          TAG="e2e-runner-${{ inputs.version }}"
          gh release create "$TAG" \
            /tmp/bytelyst-_runner-e2e-test-*.tgz \
            --title "Runner E2E test $TAG" \
            --notes "Throwaway release from runner E2E validation. Safe to delete." \
            --prerelease

      - name: Verify package queryable from Gitea
        run: |
          AUTH_HEADER="Authorization: Bearer $(cat ~/.gitea_npm_token)"
          curl -s -H "$AUTH_HEADER" \
            "http://localhost:3300/api/packages/bytelyst/npm/@bytelyst%2F_runner-e2e-test" \
            | head -200
          echo ""
          # Assert the version we just published is in the response
          curl -s -H "$AUTH_HEADER" \
            "http://localhost:3300/api/packages/bytelyst/npm/@bytelyst%2F_runner-e2e-test" \
            | grep -q '"${{ inputs.version }}"' || { echo "FAIL: version not found in Gitea registry"; exit 1; }
          echo "PASS: version ${{ inputs.version }} is in Gitea registry"

      - name: Verify pnpm install works from a clean directory
        run: |
          mkdir -p /tmp/runner-e2e-consumer && cd /tmp/runner-e2e-consumer
          cat > .npmrc <<NPMRC
          @bytelyst:registry=http://localhost:3300/api/packages/bytelyst/npm/
          //localhost:3300/api/packages/bytelyst/npm/:_authToken=$(cat ~/.gitea_npm_token)
          NPMRC
          cat > package.json <<JSON
          { "name": "runner-e2e-consumer", "version": "1.0.0", "dependencies": { "@bytelyst/_runner-e2e-test": "${{ inputs.version }}" } }
          JSON
          pnpm install --no-frozen-lockfile
          node -e "const m = require('@bytelyst/_runner-e2e-test'); console.log('Module loaded:', m); process.exit(m.ok ? 0 : 1);"
          echo "PASS: pnpm install + require works end-to-end"

      - name: Verify GitHub Release tarball matches Gitea tarball (byte-identical)
        run: |
          # The released tarball should be byte-identical to what we pushed to Gitea
          RELEASED=$(ls /tmp/bytelyst-_runner-e2e-test-*.tgz)
          GITEA_URL=$(curl -s -H "Authorization: Bearer $(cat ~/.gitea_npm_token)" \
            "http://localhost:3300/api/packages/bytelyst/npm/@bytelyst%2F_runner-e2e-test" \
            | grep -oE '"tarball":"[^"]*"' | head -1 | cut -d'"' -f4)
          curl -s -H "Authorization: Bearer $(cat ~/.gitea_npm_token)" -o /tmp/from-gitea.tgz "$GITEA_URL"
          SHA_RELEASED=$(sha256sum "$RELEASED" | awk '{print $1}')
          SHA_GITEA=$(sha256sum /tmp/from-gitea.tgz | awk '{print $1}')
          echo "Released:  $SHA_RELEASED"
          echo "Gitea:     $SHA_GITEA"
          [ "$SHA_RELEASED" = "$SHA_GITEA" ] || { echo "FAIL: tarball mismatch — bytes-identical guarantee broken"; exit 1; }
          echo "PASS: tarball is byte-identical between Gitea and GitHub Release"
```

### Run the E2E

```bash
# From any machine that has gh auth'd as saravanakumardb1:
gh workflow run runner-e2e-publish.yml --ref runner/e2e -f version=0.0.1-e2e.1 --repo saravanakumardb1/learning_ai_common_plat
gh run watch --repo saravanakumardb1/learning_ai_common_plat
```

### E2E pass criteria — ALL must succeed

| Check                                     | Step that proves it                                                                          |
| ----------------------------------------- | -------------------------------------------------------------------------------------------- |
| Runner picks up the job                   | "Set up job" log in GH Actions UI shows `Runner name: hostinger-bytelyst-1`                  |
| Package publishes to Gitea                | `pnpm publish` step exits 0                                                                  |
| GitHub Release created with tarball asset | `gh release view e2e-runner-0.0.1-e2e.1 --json assets` returns 1 asset                       |
| Gitea reports the version                 | `Verify package queryable from Gitea` step says `PASS: version ... is in Gitea registry`     |
| Consumer install works                    | `Verify pnpm install works` step says `PASS: pnpm install + require works end-to-end`        |
| Release tarball ≡ Gitea tarball (sha256)  | `Verify GitHub Release tarball matches Gitea tarball` says `PASS: tarball is byte-identical` |

The last check is the **key invariant** for the corp-Mac sync flow: same bytes → same SHA512 integrity hash → lockfiles portable across both Gitea instances.

### Cleanup after E2E

```bash
# Delete the test release
gh release delete e2e-runner-0.0.1-e2e.1 --yes --repo saravanakumardb1/learning_ai_common_plat

# Delete the test package from local Gitea (via API or UI under Packages → @bytelyst/_runner-e2e-test → Delete)

# Delete the test branch and package
git checkout main
git push origin --delete runner/e2e
rm -rf packages/_runner-e2e-test
git checkout main
git commit -am "test: remove runner E2E throwaway package" || true
git push origin main
```

Leave both workflow files (`runner-smoke.yml`, `runner-e2e-publish.yml`) **on `main`** — they're idempotent and provide a way to re-validate the runner anytime (just bump the version input).

---

## 7. Hardening (do before relying for prod)

### a. Workflow approval for external PRs

`Settings → Actions → General → Fork pull request workflows` → "Require approval for all outside collaborators". Repo is private so risk is contained, but keep approval-required on.

### b. systemd resource limits

```bash
sudo systemctl edit "$SVC_NAME"
```

```ini
[Service]
CPUQuota=200%
MemoryMax=4G
TasksMax=2048
```

```bash
sudo systemctl daemon-reload
sudo systemctl restart "$SVC_NAME"
```

### c. Log rotation

```bash
sudo tee /etc/logrotate.d/gha-runner > /dev/null <<'EOF'
/home/gha-runner/actions-runner/_diag/*.log {
    weekly
    rotate 4
    compress
    missingok
    notifempty
    copytruncate
}
EOF
```

### d. Update mechanism

GitHub auto-updates the runner agent unless disabled. Keep auto-update on. Pin a minimum version in workflows if you need a specific feature:

```yaml
runs-on: [self-hosted, linux, hostinger]
```

### e. Auth scope of the runner

The runner's `GITHUB_TOKEN` (provided by GitHub Actions automatically) is scoped per-workflow. Verify in `runner-e2e-publish.yml` that `permissions:` is set narrowly (we set `contents: write` only for release creation).

---

## 8. Scaling to all 20+ repos — GitHub Organization migration

A single self-hosted runner can serve all 20+ `@bytelyst/*` repos **only if** registered at **GitHub Organization level**. The personal-account path (repo-level registration) doesn't scale beyond 1–3 repos.

### Why migrate to an org

| Concern                          | Personal account today                     | Under an org                                       |
| -------------------------------- | ------------------------------------------ | -------------------------------------------------- |
| Self-hosted runner reuse         | One registration per repo                  | One registration covers all org repos              |
| Secrets management               | Per-repo (duplicated)                      | Org-level secrets inherited by all repos           |
| Visibility                       | Per-repo Actions tabs (no cross-repo view) | Org-level Actions dashboard across all repos       |
| Permissions / team collaboration | Limited                                    | Teams, code owners, etc.                           |
| Cost                             | Free                                       | Free for unlimited public + private repos          |
| Move cost                        | —                                          | ~1–2 hours total for 20 repos (mostly automatable) |

### Migration steps (do these BEFORE Step 3 if going org-level from day 1)

```bash
# 1. Create the org via the GitHub UI:
#    https://github.com/organizations/plan
#    Choose "Free" plan. Suggested name: bytelyst-platform (or whatever fits).

# 2. Transfer each repo to the org (one-time, preserves all history + issues + stars)
for repo in learning_ai_common_plat learning_ai_clock learning_ai_notes \
            learning_ai_flowmonk learning_ai_trails learning_ai_jarvis_jr \
            learning_ai_fastgap learning_ai_peakpulse learning_ai_efforise \
            learning_ai_auth_app learning_voice_ai_agent learning_multimodal_memory_agents \
            learning_ai_local_memory_gpt learning_ai_local_llms learning_ai_talk2obsidian \
            learning_ai_mac_tooling learning_ai_productivity_web learning_ai_smart_auth; do
  echo "Transferring $repo..."
  gh api -X POST "/repos/saravanakumardb1/$repo/transfer" -f new_owner="<your-org-name>"
done

# 3. Update your local clones to point to the new owner
#    (run on each machine, in each repo dir)
cd ~/code/mygh/<repo>
git remote set-url origin https://github.com/<your-org-name>/<repo>.git
```

GitHub automatically sets up redirects from the old URLs, so external links won't break immediately — but you should update CI references, README badges, and any inter-repo URL references.

### After migration

- Get a runner registration token at the **org level**:
  ```bash
  gh api -X POST /orgs/<your-org-name>/actions/runners/registration-token --jq .token
  ```
- Use the org URL in Step 3's `config.sh` (Option B above).
- The runner now picks up jobs from any repo in the org that targets `runs-on: [self-hosted, hostinger, bytelyst]`.

### Workflow propagation across 20+ repos

Once the runner is org-level, the next problem is propagating the `publish-packages.yml` workflow file to every repo that publishes packages. Two strategies:

1. **Reusable workflow** (preferred) — define `publish-packages.yml` once as a `workflow_call` reusable workflow in `learning_ai_common_plat/.github/workflows/`, then each consuming repo has a tiny stub that calls it.
2. **Per-repo copy maintained by a sync script** — follow the same pattern as the existing `sync-npmrc.sh` in `scripts/`. Less elegant but works fine for a small repo count.

Deliver as a separate follow-up prompt.

---

## 9. Monitoring + observability — how to track this runner

The GitHub Actions tab tracks runner state at three levels:

### a. Per-repo (or per-org) workflow runs

`https://github.com/<owner>/<repo>/actions` (or `/orgs/<org>/actions/` after migration) shows every workflow run with live-streaming logs. The "Set up job" step always logs:

```
Runner name: 'hostinger-bytelyst-1'
Runner group name: 'Default'
Machine name: 'hostinger-vm'
```

This is how you confirm the right runner picked up the job.

### b. Runner pool health

- **Repo level:** `Settings → Actions → Runners`
- **Org level:** `Org settings → Actions → Runners`

Shows: status (`Idle` / `Active` / `Offline`), labels, OS, last connection time. This is where you debug "is my runner alive?".

### c. Scripted monitoring via `gh` CLI

```bash
# Watch a specific run live
gh run watch --repo <owner>/<repo>

# List recent runs
gh run list --repo <owner>/<repo> --limit 10

# View finished run with full logs
gh run view <run-id> --log --repo <owner>/<repo>

# List runners + their status (admin scope required)
gh api /repos/<owner>/<repo>/actions/runners \
  --jq '.runners[] | {name, status, busy, labels: [.labels[].name]}'

# Or at org level:
gh api /orgs/<org>/actions/runners \
  --jq '.runners[] | {name, status, busy}'
```

### d. Host-side observability (on the Hostinger VM)

```bash
SVC_NAME='actions.runner.saravanakumardb1-learning_ai_common_plat.hostinger-bytelyst-1.service'

# Live tail
sudo journalctl -u "$SVC_NAME" -f

# Last 100 lines
sudo journalctl -u "$SVC_NAME" -n 100 --no-pager

# Per-run diagnostic logs
ls -la /home/gha-runner/actions-runner/_diag/

# Current systemd state
sudo systemctl status "$SVC_NAME"
```

Use host-side logs when the runner shows "Offline" in GitHub UI but the VM is reachable — typically a daemon crash, expired registration, or network blip.

---

## 10. Deliverables — report back to the human

When complete:

1. **Service status:**
   ```bash
   sudo systemctl is-active "$SVC_NAME"
   ```
2. **GitHub UI confirmation:** screenshot or text "Runner shows 'Idle' green at github.com/saravanakumardb1/learning_ai_common_plat/settings/actions/runners".
3. **Smoke test run URL** — workflow passed.
4. **E2E test run URL** — workflow passed, all 6 pass criteria green.
5. **Installed versions:**
   ```bash
   sudo -u gha-runner bash -c 'node --version; pnpm --version; gh --version; docker --version 2>/dev/null || echo "no docker"'
   ```
6. **Log paths:**
   - systemd: `journalctl -u <service-name>`
   - runner diag: `/home/gha-runner/actions-runner/_diag/`
7. **Confirmation that cleanup happened:** test release deleted, test package removed from Gitea, throwaway package deleted from repo.

---

## 11. Guardrails

- **Do not** run the runner as root.
- **Do not** persist the GitHub registration token to disk — memory only.
- **Do not** install under the `gitea` user or any other service user — keep concerns separated.
- **Do not** open inbound ports on the VM firewall — the runner is outbound-only long-poll.
- **Do not** skip the E2E test. The smoke test alone does not prove the publish pipeline works.
- **Do not** mark E2E as passed unless all 6 pass criteria succeed, especially the byte-identical tarball check.
- **Do not** leave the throwaway `@bytelyst/_runner-e2e-test` package in the Gitea registry — it pollutes the namespace.

---

## 12. Rollback

```bash
SVC_NAME='actions.runner.saravanakumardb1-learning_ai_common_plat.hostinger-bytelyst-1.service'

# Stop and uninstall systemd service
sudo bash -c "cd /home/gha-runner/actions-runner && ./svc.sh stop && ./svc.sh uninstall"

# Unregister from GitHub (need a fresh removal token)
REMOVAL_TOKEN=$(gh api -X POST /repos/saravanakumardb1/learning_ai_common_plat/actions/runners/remove-token --jq .token)
sudo -u gha-runner bash -c "cd ~/actions-runner && ./config.sh remove --token $REMOVAL_TOKEN"

# Remove the user and all its files
sudo userdel -r gha-runner
```

---

## 13. Follow-up prompts (separate tasks)

Once this runner is verified end-to-end, the next prompts to issue:

1. **`publish-packages.yml`** — the real production workflow in `learning_ai_common_plat`, modeled on the E2E template above, that triggers on `v*` tags and publishes all changed `@bytelyst/*` packages.
2. **`bytelyst-sync` script** — runs on the corp Mac; downloads GitHub Release tarballs and republishes to the corp local Gitea. Verifies sha256 against Gitea before considering sync successful.
3. **SKILL doc** at `AI.dev/SKILLS/gitea-package-sync.md` — describes the full three-system flow for future contributors.

---

## 14. Questions to ask the human BEFORE starting if anything is ambiguous

- "Are we registering this runner at **repo level** (one repo only) or **org level** (after migrating 20+ repos to a GitHub org)? See §1 and §8."
- "If org-level: what is the org name? Has the migration in §8 already happened?"
- "If repo-level: which repo am I registering for? (default: `saravanakumardb1/learning_ai_common_plat`)"
- "Is Docker required on the runner — i.e., does any planned workflow run `docker` commands? (default: no, only Gitea uses Docker)"
- "What user currently owns `~/.gitea_npm_token` on this VM? (pre-flight check #9 will tell us)"
- "Do you have a runner registration token, or should I fetch one via `gh api`?"
- "Are you OK with me creating a throwaway `@bytelyst/_runner-e2e-test` package, publishing it, and then deleting it as part of E2E validation?"

If any of these are unclear, stop and ask before installing anything.