bytelyst-devops-tools/docs/operations.md

Operations Guide

Common operational paths for the team.

Use this file as the routing guide. For the exact support boundary, cross-check docs/supported-scripts.md.

---

Hostinger VM Maintenance

See docs/hostinger-vm-maintenance.md for:

• The May 2026 incident post-mortem (load avg 1305, root cause, full fix log)
• Ongoing disk/memory/load health targets
• Safe cleanup reference commands
• How to deploy admin-web when ready

Quick scripts:

# Health check (read-only, any time)
bash scripts/VMs/HostingerVM/vm-health-check.sh

# Weekly safe cleanup
bash scripts/VMs/HostingerVM/vm-cleanup.sh

# Monthly full cleanup
bash scripts/VMs/HostingerVM/vm-cleanup.sh --full

# Install automated cron schedule
bash scripts/VMs/HostingerVM/vm-cleanup.sh --install-cron

Cron setup details: scripts/VMs/HostingerVM/CRON_SETUP.md

---

Gitea Registry Backup

The local Gitea npm registry is backed up with a native gitea dump job.

Installed VM paths:

• Script: /opt/bytelyst/scripts/backup-gitea.sh
• Backup directory: /opt/bytelyst/backups/gitea
• Systemd service: bytelyst-gitea-backup.service
• Systemd timer: bytelyst-gitea-backup.timer

Versioned source files:

• scripts/gitea-backup.sh
• systemd/bytelyst-gitea-backup.service
• systemd/bytelyst-gitea-backup.timer

Useful commands:

# Run a backup immediately
sudo systemctl start bytelyst-gitea-backup.service

# Check last run and next scheduled run
sudo systemctl status bytelyst-gitea-backup.service --no-pager
systemctl list-timers --all --no-pager bytelyst-gitea-backup.timer

# List retained backup dumps
ls -lh /opt/bytelyst/backups/gitea

The timer runs daily at 03:15 UTC and the script deletes dumps older than 14 days by default.

---

1. Remove A Collaborator Interactively

Use:

./remove_user_interactive.sh

Use this when:

• you need repository pattern matching
• you want per-repository confirmation
• you want the safest default path for access removal

Requirements:

• curl
• jq
• GitHub token with the necessary repo and org permissions

2. Remove A Collaborator In A More Guided Flow

Use:

./remove_user_guided.sh

Use this when:

• you want a more opinionated step-by-step flow
• you are onboarding someone to the repo tools

3. Remove A Collaborator In A Scripted Way

Use:

./remove_user_from_repos.sh

Use this when:

• you already know the target root, user, and repo pattern
• you want a repeatable command invocation

Read first:

• README_remove_user_script.md
• docs/supported-scripts.md

4. Run Common GitHub Admin Commands

Use:

./bytelyst-cli.sh help

Typical commands:

./bytelyst-cli.sh list-public-repos --user <username>
./bytelyst-cli.sh list-private-repos --org <orgname>
./bytelyst-cli.sh check-collaborators --input github_acc_input.json

Use this when:

• you need the main shared CLI entry point
• the task fits the commands already exposed there

5. Scan Multi-Repo Git Status

Use:

./git-work-safety-tools/git_repos_status.sh
./git-work-safety-tools/multi_repo_status.sh

Use this when:

• you need visibility across many local repositories
• you want safer batch git operations

6. Safely Push Across Multiple Repositories

Use:

./git-work-safety-tools/multi_repo_safe_push.sh

Use this when:

• you are coordinating many repos
• you want a safer batch push helper instead of ad hoc shell loops

7. Scan GitHub Access Across Accounts

Use:

cp accounts.example.json accounts.json
./github_access_scripts/list_user_repos.sh <github_user>
./github_access_scripts/check_repo_access.sh <github_user> <owner/repo>

Use this when:

• you need access verification
• you need a focused check outside the main CLI

8. Generate Repo Scan Artifacts Locally

Use:

./github_repo_scanners/create_user_repo_lists.sh
./github_repo_scanners/run_contributor_json_creation.sh accounts.json

Generated outputs go to:

• github_repo_scanners/user_repos/
• github_repo_scanners/contributor_repos/

Do not commit those outputs unless they are intentionally curated fixtures.

9. Harden An Ubuntu VM

Use:

sudo bash scripts/ubuntu-vm-security-update.sh --help
sudo bash scripts/ubuntu-vm-security-update.sh --dry-run

Use this when:

• you are preparing or refreshing an Ubuntu VM
• you want unattended upgrades, UFW, and fail2ban configured

Recommended order:

1. Read scripts/README.md
2. Run --help
3. Run --dry-run
4. Run the real command with explicit reboot behavior

After a reboot triggered by the script, verify the resulting host state with:

test -f /var/run/reboot-required && echo "reboot still required" || echo "reboot clear"
sudo systemctl status unattended-upgrades fail2ban
sudo ufw status verbose
sudo tail -n 50 /var/log/vm-security-update.log
apt list --upgradable

If packages are still pending or the services are unhealthy, rerun:

sudo bash scripts/ubuntu-vm-security-update.sh

10. Operate Hermes Agent Safely

Use:

hermes --version
hermes status --all
hermes cron list
systemctl status hermes-gateway --no-pager

Read first:

• docs/hermes-setup-upgrade-roadmap.md
• docs/hermes-operations.md

Use this when:

• you are upgrading or troubleshooting Hermes
• you are checking Telegram gateway health
• you are verifying backup/watchdog cron jobs
• you are evaluating any private-only dashboard/API access pattern

Hard rule: do not expose a Hermes dashboard or Hermes API publicly unless S explicitly approves the exact hostname, auth gate, and access path.

Team Guidance

• Prefer the supported entry points in docs/tooling-status.md.
• Prefer docs/operations.md and docs/supported-scripts.md over scanning the repo root manually.
• Prefer templates such as accounts.example.json and .env.example over ad hoc local files.
• Do not commit local credentials, generated transcripts, or repo-scan outputs.