learning_ai_common_plat/docs/devops/HOSTINGER_GITHUB_RUNNER_SETUP.md
saravanakumardb1 6bf15eae7a docs(devops): Hostinger runner prompt v2 — org migration + monitoring + hardening
Adds the missing pieces revealed during review:

§1 Multi-repo registration decision — choose repo-level vs org-level
   up-front. Default doc remains repo-level, but explicitly calls out
   org-level as the scaling path for 20+ repos.

§2 Pre-flight check additions:
  - Arch detection (x86_64 / aarch64) before downloading runner tarball
  - github.com + objects.githubusercontent.com reachability check
  - gh CLI auth status check (must be saravanakumardb1)

§4 Installation hardening:
  - Step 1 is now idempotent (getent guards on useradd/usermod)
  - Step 3 queries latest runner version via gh api (no more stale pin)
  - Step 3 includes SHA256 verification of the downloaded tarball
    against the release-notes manifest, with explicit STOP-if-mismatch
  - Step 3 has REGISTRATION_URL var with commented Option A/B for
    repo-level vs org-level scope

§5 Smoke test — added explicit git checkout/add/commit/push commands
   for creating the runner/smoke branch (was implicit before).

§8 (renamed) — comprehensive org migration guide:
  - Side-by-side table: personal account today vs under-an-org
  - Bash loop to transfer all 18 repos via gh api
  - git remote set-url commands for each local clone
  - Post-migration org-level registration token fetch
  - Workflow propagation strategies (reusable workflow vs sync script)

§9 (new) — Monitoring + observability:
  - GitHub Actions tab per-repo + per-org workflow views
  - Runner pool health (Settings → Actions → Runners) at repo + org level
  - gh CLI commands for scripted monitoring (run watch, list, view, runners)
  - Host-side journalctl + _diag/ inspection commands

§14 Questions — updated to ask about scope (repo vs org) first.

Section numbering shifted by +1 from §9 onward to make room for the
new Monitoring section.
2026-05-24 18:04:50 -07:00

29 KiB
Raw Blame History

Hostinger VM — GitHub Actions Self-Hosted Runner Setup

Delegation prompt for the Codex agent running on the Hostinger VM. Read top-to-bottom before executing. Stop and ask the human if any pre-flight check fails or any deliverable is unclear.


1. Goal

Set up a GitHub Actions self-hosted runner on the Hostinger VM that can:

  1. Receive workflow triggers from all 20+ @bytelyst/* repos (see §8 for the org-vs-repo registration decision).
  2. Build @bytelyst/* npm packages from tagged releases.
  3. Publish them to the local Gitea instance on this VM (http://localhost:3300/api/packages/bytelyst/npm/).
  4. Upload the same tarballs as GitHub Release assets so a corp-network Mac can sync them into its own local Gitea (via the separate bytelyst-sync script described in a follow-up prompt).

Self-hosted on Hostinger beats GitHub-hosted runners because:

  • No GitHub Actions minute cap.
  • Gitea is on localhost from this VM → zero-latency publish, no public TLS needed.
  • VM is always on; runner is reachable indefinitely.

Multi-repo registration decision

Decide before Step 2 whether this runner serves a single repo or all 20+:

Approach Registration scope Use when
Repo-level (default in this doc) Single repo URL You're validating the runner first, or you only have 12 repos publishing packages
Org-level (recommended at scale) A GitHub Organization URL You've migrated 20+ repos under one org — see §8 for migration steps. One registration, all org repos eligible

The install steps are identical — only the --url flag in config.sh (Step 3) changes.


2. Pre-flight checks (run first, do not skip)

# 1. Confirm Linux VM
hostname && uname -a       # Expected: Linux

# 2. Confirm Gitea is running locally
curl -s -o /dev/null -w "%{http_code}\n" http://localhost:3300/   # Expected: 200 or 302

# 3. Confirm Gitea npm registry endpoint reachable
curl -s -o /dev/null -w "%{http_code}\n" http://localhost:3300/api/packages/bytelyst/npm/    # Expected: 200 or 401

# 4. Confirm Node 20 and pnpm 9 installable
node --version 2>/dev/null || echo "Node not installed (will install in Step 5)"
pnpm --version 2>/dev/null || echo "pnpm not installed (will install in Step 5)"

# 5. Confirm gh CLI exists
gh --version 2>/dev/null || echo "gh CLI not installed (will install in Step 5)"

# 6a. Detect architecture (used for runner tarball selection in Step 3)
ARCH=$(uname -m)
case "$ARCH" in
  x86_64)  export RUNNER_ARCH="linux-x64";;
  aarch64) export RUNNER_ARCH="linux-arm64";;
  *) echo "Unsupported arch: $ARCH — STOP and report"; ;;
esac
echo "Will install runner for: $RUNNER_ARCH"

# 6b. Disk free
df -h /     # Need ~5 GB headroom

# 7. Confirm no existing runner
ls -la ~/actions-runner 2>/dev/null && echo "Runner dir exists — STOP and confirm with human" || echo "No existing runner"

# 8. Confirm github.com reachable (both API and download CDN)
curl -s -o /dev/null -w "api.github.com: %{http_code}\n" https://api.github.com/
curl -s -o /dev/null -w "objects.githubusercontent.com: %{http_code}\n" -L \
  https://objects.githubusercontent.com/   # runner tarball download host
# Expected: 200 / 403 (403 from CDN root is normal; what matters is non-network-error)

# 9. Confirm the Gitea token file exists somewhere on this VM
sudo find /home /root -maxdepth 3 -name ".gitea_npm_token" 2>/dev/null | head -5
# Expected: at least one path. Note the owning user — needed in Step 6.

# 10. Confirm gh CLI is auth'd as saravanakumardb1 (needed for registration token)
gh auth status 2>&1 | grep -E "Logged in to|saravanakumardb1" | head -5
# If not logged in as saravanakumardb1, run: gh auth login (and pick saravanakumardb1)

If any check fails or surprises you, stop and report back before proceeding.


3. What you'll create

Item Path/Identifier
Dedicated Linux user gha-runner
Runner installation /home/gha-runner/actions-runner/
systemd service actions.runner.saravanakumardb1-learning_ai_common_plat.hostinger-bytelyst-1.service
Runner labels self-hosted, linux, x64, hostinger, bytelyst
Gitea publish token (copy) /home/gha-runner/.gitea_npm_token (mode 600)
Smoke-test workflow .github/workflows/runner-smoke.yml (commit on a branch)
E2E test workflow .github/workflows/runner-e2e-publish.yml (commit on a branch)

4. Installation

Step 1 — Create the dedicated runner user (idempotent)

if ! getent passwd gha-runner >/dev/null; then
  sudo useradd -m -s /bin/bash gha-runner
  echo "Created gha-runner user"
else
  echo "gha-runner user already exists — skipping useradd"
fi

# Add to docker group only if docker is on this host
if getent group docker >/dev/null; then
  sudo usermod -aG docker gha-runner
  echo "Added gha-runner to docker group"
fi

id gha-runner

Step 2 — Get a GitHub runner registration token (one-time, ~1h TTL)

Preferred (via gh CLI auth'd as saravanakumardb1):

gh api -X POST /repos/saravanakumardb1/learning_ai_common_plat/actions/runners/registration-token --jq .token

Alternative (browser): Open https://github.com/saravanakumardb1/learning_ai_common_plat/settings/actions/runners/new?arch=x64&os=linux and copy the token from the ./config.sh command shown.

Hold the token in shell memory only:

read -s RUNNER_TOKEN   # paste, press Enter (no echo)

Step 3 — Download, verify, configure the runner

# Query the latest runner version (don't hardcode)
LATEST=$(gh api /repos/actions/runner/releases/latest --jq '.tag_name' | sed 's/^v//')
echo "Latest runner version: $LATEST"
# As of writing this doc: 2.319.1. If LATEST is wildly different, STOP and confirm with human.

sudo -iu gha-runner bash <<EOF
mkdir -p ~/actions-runner && cd ~/actions-runner
RUNNER_VERSION="$LATEST"
RUNNER_ARCH="${RUNNER_ARCH}"   # from pre-flight 5a
TARBALL="actions-runner-\${RUNNER_ARCH}-\${RUNNER_VERSION}.tar.gz"

# Download tarball
curl -fSL -o "\$TARBALL" \
  "https://github.com/actions/runner/releases/download/v\${RUNNER_VERSION}/\$TARBALL"

# Download checksum manifest and verify (GitHub publishes SHA256 alongside each release)
EXPECTED_SHA=\$(gh api /repos/actions/runner/releases/tags/v\${RUNNER_VERSION} \
  --jq ".body" | grep -oE "\b[0-9a-f]{64}\s+\$TARBALL\b" | awk '{print \$1}')
ACTUAL_SHA=\$(sha256sum "\$TARBALL" | awk '{print \$1}')

if [ "\$EXPECTED_SHA" != "\$ACTUAL_SHA" ]; then
  echo "FAIL: SHA mismatch"
  echo "  Expected: \$EXPECTED_SHA"
  echo "  Actual:   \$ACTUAL_SHA"
  exit 1
fi
echo "PASS: tarball SHA verified"

tar xzf "./\$TARBALL"
EOF

Note: if gh api parsing of the SHA from the release body fails (GitHub sometimes changes release-note formatting), fall back to the official hashes page: https://github.com/actions/runner/releases/tag/v<version>. If you can't verify the SHA, STOP and report — don't run unverified binaries.

Register the runner. Choose the URL based on your scope decision (§1):

# OPTION A — repo-level (default during validation)
REGISTRATION_URL="https://github.com/saravanakumardb1/learning_ai_common_plat"

# OPTION B — org-level (once you've migrated to an org per §8)
# REGISTRATION_URL="https://github.com/<your-org-name>"

sudo -u gha-runner -E -i bash -c "
cd ~/actions-runner && \
./config.sh \
  --url $REGISTRATION_URL \
  --token $RUNNER_TOKEN \
  --name hostinger-bytelyst-1 \
  --labels self-hosted,linux,x64,hostinger,bytelyst \
  --work _work \
  --replace \
  --unattended
"

unset RUNNER_TOKEN

Verify in GitHub UI: runner should show "Idle" green under Settings → Actions → Runners.

Step 4 — Install as a systemd service

sudo bash -c "
cd /home/gha-runner/actions-runner && \
./svc.sh install gha-runner && \
./svc.sh start
"

SVC_NAME='actions.runner.saravanakumardb1-learning_ai_common_plat.hostinger-bytelyst-1.service'
sudo systemctl status "$SVC_NAME"
sudo journalctl -u "$SVC_NAME" -n 30 --no-pager
# Expected: "active (running)" + "Listening for Jobs"

Step 5 — Install Node 20, pnpm 9, gh CLI system-wide

# Node 20 via Nodesource
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo bash -
sudo apt-get install -y nodejs

# gh CLI (if not already)
sudo apt-get install -y gh   # or follow https://cli.github.com/manual/installation if 'gh' isn't in the apt repos

# pnpm
sudo npm install -g pnpm@9

# Verify all reachable from the runner user
sudo -u gha-runner bash -c 'node --version && pnpm --version && gh --version'

Step 6 — Give the runner access to the local Gitea publish token

# Identify the source token file (from pre-flight check #9)
SRC_TOKEN=/home/<original-user>/.gitea_npm_token

# Copy and lock down
sudo cp "$SRC_TOKEN" /home/gha-runner/.gitea_npm_token
sudo chown gha-runner:gha-runner /home/gha-runner/.gitea_npm_token
sudo chmod 600 /home/gha-runner/.gitea_npm_token

# Verify
sudo -u gha-runner bash -c 'wc -c < ~/.gitea_npm_token && stat -c "%a %U:%G" ~/.gitea_npm_token'
# Expected: nonzero byte count, mode 600, owner gha-runner:gha-runner

5. Smoke test (basic — runner picks up jobs)

Create branch runner/smoke in learning_ai_common_plat with the workflow below.

cd ~/code/mygh/learning_ai_common_plat
git checkout -b runner/smoke
mkdir -p .github/workflows
# (paste workflow below into .github/workflows/runner-smoke.yml)
git add .github/workflows/runner-smoke.yml
git commit -m "ci: add self-hosted runner smoke-test workflow"
git push origin runner/smoke
# .github/workflows/runner-smoke.yml
name: Runner Smoke Test
on:
  workflow_dispatch:
  push:
    branches: [runner/smoke]
jobs:
  smoke:
    runs-on: [self-hosted, linux, hostinger]
    steps:
      - run: echo "host=$(hostname) user=$(whoami) cwd=$(pwd)"
      - run: node --version && pnpm --version && gh --version
      - run: |
          echo "Gitea health:"
          curl -s -o /dev/null -w "  http://localhost:3300/  → %{http_code}\n" http://localhost:3300/
          curl -s -o /dev/null -w "  /api/packages/bytelyst/npm/ → %{http_code}\n" \
            http://localhost:3300/api/packages/bytelyst/npm/          
      - run: |
          if [ -f ~/.gitea_npm_token ]; then
            echo "Gitea token present, $(wc -c < ~/.gitea_npm_token) bytes, mode $(stat -c %a ~/.gitea_npm_token)"
          else
            echo "ERROR: Gitea token missing"
            exit 1
          fi          

Trigger from GitHub UI (Actions tab → Runner Smoke Test → Run workflow on runner/smoke). All steps must pass.


6. End-to-end validation (CRITICAL — proves the actual use case)

The smoke test only proves the runner can execute. The E2E test proves the whole publish pipeline works: package builds, publishes to local Gitea, uploads to GitHub Releases, and is installable on a different machine.

E2E test workflow

Create a temporary test package in learning_ai_common_plat (will be removed after validation):

# On Hostinger or from human's machine — does NOT need to run on the runner
mkdir -p packages/_runner-e2e-test
cat > packages/_runner-e2e-test/package.json <<'EOF'
{
  "name": "@bytelyst/_runner-e2e-test",
  "version": "0.0.1",
  "description": "Throwaway package for E2E validating the Hostinger runner. Safe to delete after validation.",
  "main": "index.js",
  "files": ["index.js"]
}
EOF
echo "module.exports = { ok: true, builtAt: new Date().toISOString() };" \
  > packages/_runner-e2e-test/index.js
git checkout -b runner/e2e
git add packages/_runner-e2e-test/
git commit -m "test: add throwaway package for runner E2E validation"
git push origin runner/e2e

Create the E2E workflow on the same branch:

# .github/workflows/runner-e2e-publish.yml
name: Runner E2E — publish + release

on:
  workflow_dispatch:
    inputs:
      version:
        description: 'Version to publish (semver)'
        required: true
        default: '0.0.1-e2e.1'

jobs:
  publish:
    runs-on: [self-hosted, linux, hostinger]
    permissions:
      contents: write # for GitHub Release creation
    steps:
      - uses: actions/checkout@v4

      - name: Set test version
        working-directory: packages/_runner-e2e-test
        run: |
          npm version "${{ inputs.version }}" --no-git-tag-version --allow-same-version
          cat package.json          

      - name: Configure pnpm registry for Gitea
        working-directory: packages/_runner-e2e-test
        run: |
          cat > .npmrc <<NPMRC
          @bytelyst:registry=http://localhost:3300/api/packages/bytelyst/npm/
          //localhost:3300/api/packages/bytelyst/npm/:_authToken=$(cat ~/.gitea_npm_token)
          NPMRC
          echo ".npmrc written:"
          sed 's|_authToken=.*|_authToken=***|' .npmrc          

      - name: Publish to local Gitea
        working-directory: packages/_runner-e2e-test
        run: pnpm publish --no-git-checks --registry http://localhost:3300/api/packages/bytelyst/npm/

      - name: Pack tarball for GitHub Release
        working-directory: packages/_runner-e2e-test
        run: |
          pnpm pack --pack-destination /tmp
          ls -la /tmp/bytelyst-_runner-e2e-test-*.tgz          

      - name: Create GitHub Release with tarball
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          TAG="e2e-runner-${{ inputs.version }}"
          gh release create "$TAG" \
            /tmp/bytelyst-_runner-e2e-test-*.tgz \
            --title "Runner E2E test $TAG" \
            --notes "Throwaway release from runner E2E validation. Safe to delete." \
            --prerelease          

      - name: Verify package queryable from Gitea
        run: |
          AUTH_HEADER="Authorization: Bearer $(cat ~/.gitea_npm_token)"
          curl -s -H "$AUTH_HEADER" \
            "http://localhost:3300/api/packages/bytelyst/npm/@bytelyst%2F_runner-e2e-test" \
            | head -200
          echo ""
          # Assert the version we just published is in the response
          curl -s -H "$AUTH_HEADER" \
            "http://localhost:3300/api/packages/bytelyst/npm/@bytelyst%2F_runner-e2e-test" \
            | grep -q '"${{ inputs.version }}"' || { echo "FAIL: version not found in Gitea registry"; exit 1; }
          echo "PASS: version ${{ inputs.version }} is in Gitea registry"          

      - name: Verify pnpm install works from a clean directory
        run: |
          mkdir -p /tmp/runner-e2e-consumer && cd /tmp/runner-e2e-consumer
          cat > .npmrc <<NPMRC
          @bytelyst:registry=http://localhost:3300/api/packages/bytelyst/npm/
          //localhost:3300/api/packages/bytelyst/npm/:_authToken=$(cat ~/.gitea_npm_token)
          NPMRC
          cat > package.json <<JSON
          { "name": "runner-e2e-consumer", "version": "1.0.0", "dependencies": { "@bytelyst/_runner-e2e-test": "${{ inputs.version }}" } }
          JSON
          pnpm install --no-frozen-lockfile
          node -e "const m = require('@bytelyst/_runner-e2e-test'); console.log('Module loaded:', m); process.exit(m.ok ? 0 : 1);"
          echo "PASS: pnpm install + require works end-to-end"          

      - name: Verify GitHub Release tarball matches Gitea tarball (byte-identical)
        run: |
          # The released tarball should be byte-identical to what we pushed to Gitea
          RELEASED=$(ls /tmp/bytelyst-_runner-e2e-test-*.tgz)
          GITEA_URL=$(curl -s -H "Authorization: Bearer $(cat ~/.gitea_npm_token)" \
            "http://localhost:3300/api/packages/bytelyst/npm/@bytelyst%2F_runner-e2e-test" \
            | grep -oE '"tarball":"[^"]*"' | head -1 | cut -d'"' -f4)
          curl -s -H "Authorization: Bearer $(cat ~/.gitea_npm_token)" -o /tmp/from-gitea.tgz "$GITEA_URL"
          SHA_RELEASED=$(sha256sum "$RELEASED" | awk '{print $1}')
          SHA_GITEA=$(sha256sum /tmp/from-gitea.tgz | awk '{print $1}')
          echo "Released:  $SHA_RELEASED"
          echo "Gitea:     $SHA_GITEA"
          [ "$SHA_RELEASED" = "$SHA_GITEA" ] || { echo "FAIL: tarball mismatch — bytes-identical guarantee broken"; exit 1; }
          echo "PASS: tarball is byte-identical between Gitea and GitHub Release"          

Run the E2E

# From any machine that has gh auth'd as saravanakumardb1:
gh workflow run runner-e2e-publish.yml --ref runner/e2e -f version=0.0.1-e2e.1 --repo saravanakumardb1/learning_ai_common_plat
gh run watch --repo saravanakumardb1/learning_ai_common_plat

E2E pass criteria — ALL must succeed

Check Step that proves it
Runner picks up the job "Set up job" log in GH Actions UI shows Runner name: hostinger-bytelyst-1
Package publishes to Gitea pnpm publish step exits 0
GitHub Release created with tarball asset gh release view e2e-runner-0.0.1-e2e.1 --json assets returns 1 asset
Gitea reports the version Verify package queryable from Gitea step says PASS: version ... is in Gitea registry
Consumer install works Verify pnpm install works step says PASS: pnpm install + require works end-to-end
Release tarball ≡ Gitea tarball (sha256) Verify GitHub Release tarball matches Gitea tarball says PASS: tarball is byte-identical

The last check is the key invariant for the corp-Mac sync flow: same bytes → same SHA512 integrity hash → lockfiles portable across both Gitea instances.

Cleanup after E2E

# Delete the test release
gh release delete e2e-runner-0.0.1-e2e.1 --yes --repo saravanakumardb1/learning_ai_common_plat

# Delete the test package from local Gitea (via API or UI under Packages → @bytelyst/_runner-e2e-test → Delete)

# Delete the test branch and package
git checkout main
git push origin --delete runner/e2e
rm -rf packages/_runner-e2e-test
git checkout main
git commit -am "test: remove runner E2E throwaway package" || true
git push origin main

Leave both workflow files (runner-smoke.yml, runner-e2e-publish.yml) on main — they're idempotent and provide a way to re-validate the runner anytime (just bump the version input).


7. Hardening (do before relying for prod)

a. Workflow approval for external PRs

Settings → Actions → General → Fork pull request workflows → "Require approval for all outside collaborators". Repo is private so risk is contained, but keep approval-required on.

b. systemd resource limits

sudo systemctl edit "$SVC_NAME"
[Service]
CPUQuota=200%
MemoryMax=4G
TasksMax=2048
sudo systemctl daemon-reload
sudo systemctl restart "$SVC_NAME"

c. Log rotation

sudo tee /etc/logrotate.d/gha-runner > /dev/null <<'EOF'
/home/gha-runner/actions-runner/_diag/*.log {
    weekly
    rotate 4
    compress
    missingok
    notifempty
    copytruncate
}
EOF

d. Update mechanism

GitHub auto-updates the runner agent unless disabled. Keep auto-update on. Pin a minimum version in workflows if you need a specific feature:

runs-on: [self-hosted, linux, hostinger]

e. Auth scope of the runner

The runner's GITHUB_TOKEN (provided by GitHub Actions automatically) is scoped per-workflow. Verify in runner-e2e-publish.yml that permissions: is set narrowly (we set contents: write only for release creation).


8. Scaling to all 20+ repos — GitHub Organization migration

A single self-hosted runner can serve all 20+ @bytelyst/* repos only if registered at GitHub Organization level. The personal-account path (repo-level registration) doesn't scale beyond 13 repos.

Why migrate to an org

Concern Personal account today Under an org
Self-hosted runner reuse One registration per repo One registration covers all org repos
Secrets management Per-repo (duplicated) Org-level secrets inherited by all repos
Visibility Per-repo Actions tabs (no cross-repo view) Org-level Actions dashboard across all repos
Permissions / team collaboration Limited Teams, code owners, etc.
Cost Free Free for unlimited public + private repos
Move cost ~12 hours total for 20 repos (mostly automatable)

Migration steps (do these BEFORE Step 3 if going org-level from day 1)

# 1. Create the org via the GitHub UI:
#    https://github.com/organizations/plan
#    Choose "Free" plan. Suggested name: bytelyst-platform (or whatever fits).

# 2. Transfer each repo to the org (one-time, preserves all history + issues + stars)
for repo in learning_ai_common_plat learning_ai_clock learning_ai_notes \
            learning_ai_flowmonk learning_ai_trails learning_ai_jarvis_jr \
            learning_ai_fastgap learning_ai_peakpulse learning_ai_efforise \
            learning_ai_auth_app learning_voice_ai_agent learning_multimodal_memory_agents \
            learning_ai_local_memory_gpt learning_ai_local_llms learning_ai_talk2obsidian \
            learning_ai_mac_tooling learning_ai_productivity_web learning_ai_smart_auth; do
  echo "Transferring $repo..."
  gh api -X POST "/repos/saravanakumardb1/$repo/transfer" -f new_owner="<your-org-name>"
done

# 3. Update your local clones to point to the new owner
#    (run on each machine, in each repo dir)
cd ~/code/mygh/<repo>
git remote set-url origin https://github.com/<your-org-name>/<repo>.git

GitHub automatically sets up redirects from the old URLs, so external links won't break immediately — but you should update CI references, README badges, and any inter-repo URL references.

After migration

  • Get a runner registration token at the org level:
    gh api -X POST /orgs/<your-org-name>/actions/runners/registration-token --jq .token
    
  • Use the org URL in Step 3's config.sh (Option B above).
  • The runner now picks up jobs from any repo in the org that targets runs-on: [self-hosted, hostinger, bytelyst].

Workflow propagation across 20+ repos

Once the runner is org-level, the next problem is propagating the publish-packages.yml workflow file to every repo that publishes packages. Two strategies:

  1. Reusable workflow (preferred) — define publish-packages.yml once as a workflow_call reusable workflow in learning_ai_common_plat/.github/workflows/, then each consuming repo has a tiny stub that calls it.
  2. Per-repo copy maintained by a sync script — follow the same pattern as the existing sync-npmrc.sh in scripts/. Less elegant but works fine for a small repo count.

Deliver as a separate follow-up prompt.


9. Monitoring + observability — how to track this runner

The GitHub Actions tab tracks runner state at three levels:

a. Per-repo (or per-org) workflow runs

https://github.com/<owner>/<repo>/actions (or /orgs/<org>/actions/ after migration) shows every workflow run with live-streaming logs. The "Set up job" step always logs:

Runner name: 'hostinger-bytelyst-1'
Runner group name: 'Default'
Machine name: 'hostinger-vm'

This is how you confirm the right runner picked up the job.

b. Runner pool health

  • Repo level: Settings → Actions → Runners
  • Org level: Org settings → Actions → Runners

Shows: status (Idle / Active / Offline), labels, OS, last connection time. This is where you debug "is my runner alive?".

c. Scripted monitoring via gh CLI

# Watch a specific run live
gh run watch --repo <owner>/<repo>

# List recent runs
gh run list --repo <owner>/<repo> --limit 10

# View finished run with full logs
gh run view <run-id> --log --repo <owner>/<repo>

# List runners + their status (admin scope required)
gh api /repos/<owner>/<repo>/actions/runners \
  --jq '.runners[] | {name, status, busy, labels: [.labels[].name]}'

# Or at org level:
gh api /orgs/<org>/actions/runners \
  --jq '.runners[] | {name, status, busy}'

d. Host-side observability (on the Hostinger VM)

SVC_NAME='actions.runner.saravanakumardb1-learning_ai_common_plat.hostinger-bytelyst-1.service'

# Live tail
sudo journalctl -u "$SVC_NAME" -f

# Last 100 lines
sudo journalctl -u "$SVC_NAME" -n 100 --no-pager

# Per-run diagnostic logs
ls -la /home/gha-runner/actions-runner/_diag/

# Current systemd state
sudo systemctl status "$SVC_NAME"

Use host-side logs when the runner shows "Offline" in GitHub UI but the VM is reachable — typically a daemon crash, expired registration, or network blip.


10. Deliverables — report back to the human

When complete:

  1. Service status:
    sudo systemctl is-active "$SVC_NAME"
    
  2. GitHub UI confirmation: screenshot or text "Runner shows 'Idle' green at github.com/saravanakumardb1/learning_ai_common_plat/settings/actions/runners".
  3. Smoke test run URL — workflow passed.
  4. E2E test run URL — workflow passed, all 6 pass criteria green.
  5. Installed versions:
    sudo -u gha-runner bash -c 'node --version; pnpm --version; gh --version; docker --version 2>/dev/null || echo "no docker"'
    
  6. Log paths:
    • systemd: journalctl -u <service-name>
    • runner diag: /home/gha-runner/actions-runner/_diag/
  7. Confirmation that cleanup happened: test release deleted, test package removed from Gitea, throwaway package deleted from repo.

11. Guardrails

  • Do not run the runner as root.
  • Do not persist the GitHub registration token to disk — memory only.
  • Do not install under the gitea user or any other service user — keep concerns separated.
  • Do not open inbound ports on the VM firewall — the runner is outbound-only long-poll.
  • Do not skip the E2E test. The smoke test alone does not prove the publish pipeline works.
  • Do not mark E2E as passed unless all 6 pass criteria succeed, especially the byte-identical tarball check.
  • Do not leave the throwaway @bytelyst/_runner-e2e-test package in the Gitea registry — it pollutes the namespace.

12. Rollback

SVC_NAME='actions.runner.saravanakumardb1-learning_ai_common_plat.hostinger-bytelyst-1.service'

# Stop and uninstall systemd service
sudo bash -c "cd /home/gha-runner/actions-runner && ./svc.sh stop && ./svc.sh uninstall"

# Unregister from GitHub (need a fresh removal token)
REMOVAL_TOKEN=$(gh api -X POST /repos/saravanakumardb1/learning_ai_common_plat/actions/runners/remove-token --jq .token)
sudo -u gha-runner bash -c "cd ~/actions-runner && ./config.sh remove --token $REMOVAL_TOKEN"

# Remove the user and all its files
sudo userdel -r gha-runner

13. Follow-up prompts (separate tasks)

Once this runner is verified end-to-end, the next prompts to issue:

  1. publish-packages.yml — the real production workflow in learning_ai_common_plat, modeled on the E2E template above, that triggers on v* tags and publishes all changed @bytelyst/* packages.
  2. bytelyst-sync script — runs on the corp Mac; downloads GitHub Release tarballs and republishes to the corp local Gitea. Verifies sha256 against Gitea before considering sync successful.
  3. SKILL doc at AI.dev/SKILLS/gitea-package-sync.md — describes the full three-system flow for future contributors.

14. Questions to ask the human BEFORE starting if anything is ambiguous

  • "Are we registering this runner at repo level (one repo only) or org level (after migrating 20+ repos to a GitHub org)? See §1 and §8."
  • "If org-level: what is the org name? Has the migration in §8 already happened?"
  • "If repo-level: which repo am I registering for? (default: saravanakumardb1/learning_ai_common_plat)"
  • "Is Docker required on the runner — i.e., does any planned workflow run docker commands? (default: no, only Gitea uses Docker)"
  • "What user currently owns ~/.gitea_npm_token on this VM? (pre-flight check #9 will tell us)"
  • "Do you have a runner registration token, or should I fetch one via gh api?"
  • "Are you OK with me creating a throwaway @bytelyst/_runner-e2e-test package, publishing it, and then deleting it as part of E2E validation?"

If any of these are unclear, stop and ask before installing anything.