diff --git a/docs/operations.md b/docs/operations.md new file mode 100644 index 0000000..5084143 --- /dev/null +++ b/docs/operations.md @@ -0,0 +1,179 @@ +# Operations Guide + +Common operational paths for the team. + +Use this file as the routing guide. For the exact support boundary, cross-check `docs/supported-scripts.md`. + +## 1. Remove A Collaborator Interactively + +Use: + +```bash +./remove_user_interactive.sh +``` + +Use this when: + +- you need repository pattern matching +- you want per-repository confirmation +- you want the safest default path for access removal + +Requirements: + +- `curl` +- `jq` +- GitHub token with the necessary repo and org permissions + +## 2. Remove A Collaborator In A More Guided Flow + +Use: + +```bash +./remove_user_guided.sh +``` + +Use this when: + +- you want a more opinionated step-by-step flow +- you are onboarding someone to the repo tools + +## 3. Remove A Collaborator In A Scripted Way + +Use: + +```bash +./remove_user_from_repos.sh +``` + +Use this when: + +- you already know the target root, user, and repo pattern +- you want a repeatable command invocation + +Read first: + +- `README_remove_user_script.md` +- `docs/supported-scripts.md` + +## 4. Run Common GitHub Admin Commands + +Use: + +```bash +./bytelyst-cli.sh help +``` + +Typical commands: + +```bash +./bytelyst-cli.sh list-public-repos --user +./bytelyst-cli.sh list-private-repos --org +./bytelyst-cli.sh check-collaborators --input github_acc_input.json +``` + +Use this when: + +- you need the main shared CLI entry point +- the task fits the commands already exposed there + +## 5. Scan Multi-Repo Git Status + +Use: + +```bash +./git-work-safety-tools/git_repos_status.sh +./git-work-safety-tools/multi_repo_status.sh +``` + +Use this when: + +- you need visibility across many local repositories +- you want safer batch git operations + +## 6. Safely Push Across Multiple Repositories + +Use: + +```bash +./git-work-safety-tools/multi_repo_safe_push.sh +``` + +Use this when: + +- you are coordinating many repos +- you want a safer batch push helper instead of ad hoc shell loops + +## 7. Scan GitHub Access Across Accounts + +Use: + +```bash +cp accounts.example.json accounts.json +./github_access_scripts/list_user_repos.sh +./github_access_scripts/check_repo_access.sh +``` + +Use this when: + +- you need access verification +- you need a focused check outside the main CLI + +## 8. Generate Repo Scan Artifacts Locally + +Use: + +```bash +./github_repo_scanners/create_user_repo_lists.sh +./github_repo_scanners/run_contributor_json_creation.sh accounts.json +``` + +Generated outputs go to: + +- `github_repo_scanners/user_repos/` +- `github_repo_scanners/contributor_repos/` + +Do not commit those outputs unless they are intentionally curated fixtures. + +## 9. Harden An Ubuntu VM + +Use: + +```bash +sudo bash scripts/ubuntu-vm-security-update.sh --help +sudo bash scripts/ubuntu-vm-security-update.sh --dry-run +``` + +Use this when: + +- you are preparing or refreshing an Ubuntu VM +- you want unattended upgrades, UFW, and fail2ban configured + +Recommended order: + +1. Read `scripts/README.md` +2. Run `--help` +3. Run `--dry-run` +4. Run the real command with explicit reboot behavior + +After a reboot triggered by the script, verify the resulting host state with: + +```bash +test -f /var/run/reboot-required && echo "reboot still required" || echo "reboot clear" +sudo systemctl status unattended-upgrades fail2ban +sudo ufw status verbose +sudo tail -n 50 /var/log/vm-security-update.log +apt list --upgradable +``` + +If packages are still pending or the services are unhealthy, rerun: + +```bash +sudo bash scripts/ubuntu-vm-security-update.sh +``` + +## Team Guidance + +- Prefer the supported entry points in `docs/tooling-status.md`. +- Prefer `docs/operations.md` and `docs/supported-scripts.md` over scanning the repo root manually. +- Prefer templates such as `accounts.example.json` and `.env.example` over ad hoc local files. +- Do not commit local credentials, generated transcripts, or repo-scan outputs. diff --git a/docs/remove_user_interactive.md b/docs/remove_user_interactive.md index 0ba5adb..8455794 100644 --- a/docs/remove_user_interactive.md +++ b/docs/remove_user_interactive.md @@ -1,204 +1,126 @@ -# Interactive User Removal Tool +# Interactive User Removal -## Overview +Reference for [`remove_user_interactive.sh`](/opt/bytelyst/bytelyst-devops-tools/remove_user_interactive.sh). -`remove_user_interactive.sh` is a comprehensive command-line tool designed to remove users from GitHub repositories with advanced pattern matching and interactive confirmation features. The tool supports both individual user repositories and organization-wide operations. +## Purpose -## Features +Use this script to remove a GitHub user from repositories that match a name pattern. It supports both: -### đŸŽ¯ Core Functionality -- **Interactive User Input**: Guides users through token, user, and pattern collection -- **Multi-Level Repository Discovery**: Handles both root account and organization repositories -- **Wildcard Pattern Matching**: Supports complex repository patterns -- **Batch Operations**: Process multiple repositories efficiently -- **Rich Visual Feedback**: Color-coded logging with progress indicators +- interactive prompting +- command-line arguments for repeatable runs -### 🔐 Security Features -- **Token Validation**: Verifies GitHub API access before operations -- **Interactive Confirmations**: Prevents accidental bulk operations -- **Permission Checks**: Validates repository access permissions -- **Error Handling**: Comprehensive error reporting and recovery +This is the safer collaborator-removal path when you want per-repository confirmation by default. -### 🎨 User Experience -- **Progress Tracking**: Real-time progress indicators -- **Flexible Confirmation System**: Multiple response options -- **Summary Reports**: Detailed operation statistics -- **Color-Coded Output**: Enhanced readability with emojis +## Requirements + +- `bash` +- `curl` +- `jq` +- a GitHub token with enough access to inspect repositories and remove collaborators + +The script accepts a token via `-t/--token`, or it will prompt for one interactively. ## Usage -### Basic Execution +Interactive: + ```bash ./remove_user_interactive.sh ``` -### Interactive Flow -1. **Dependency Check**: Validates required tools (`curl`, `jq`) -2. **Authentication**: Collects and validates GitHub token -3. **Root User Input**: Specifies the root account/organization -4. **Target User**: Defines the user to remove -5. **Pattern Matching**: Sets repository selection criteria -6. **Repository Discovery**: Finds all matching repositories -7. **Confirmation**: Reviews and confirms the operation -8. **Batch Processing**: Removes user from selected repositories -9. **Summary Report**: Displays operation results +Non-interactive: + +```bash +./remove_user_interactive.sh \ + --user \ + --root \ + --pattern '' \ + --token "$GITHUB_TOKEN" \ + --non-interactive +``` + +Help: + +```bash +./remove_user_interactive.sh --help +``` + +## Supported Flags + +- `-u`, `--user` + Username to remove. +- `-r`, `--root` + Root GitHub user or organization used for repository discovery. +- `-p`, `--pattern` + Repository name filter. `*` is supported as a wildcard. +- `-t`, `--token` + GitHub personal access token. +- `--non-interactive` + Auto-confirms all removals after discovery. +- `-h`, `--help` + Show usage. + +## How Discovery Works + +The script: + +1. validates dependencies +2. validates the GitHub token +3. collects the root owner, target user, and repository pattern +4. fetches repositories for the root user +5. fetches organizations visible from that user and their repositories +6. filters repositories by the provided pattern +7. checks each repository for the target user as a collaborator +8. asks for confirmation before each removal unless `--non-interactive` is used + +Repository names are handled as `owner/repo`. Invalid names discovered from the API are skipped. ## Pattern Matching -The tool supports sophisticated wildcard patterns for repository selection: +Examples: -### Pattern Examples -- `*-go-api*` - Matches any repository containing "-go-api" -- `frontend-*` - Matches repositories starting with "frontend-" -- `*-service` - Matches repositories ending with "-service" -- `web-*-app` - Matches repositories with "web-" prefix and "-app" suffix -- `exact-repo-name` - Matches specific repository name +- `*` + Match all discovered repositories. +- `frontend-*` + Match repositories whose names begin with `frontend-`. +- `*api*` + Match repositories whose names contain `api`. +- `exact-repo-name` + Match that exact repository name. -### Pattern Behavior -- Patterns are case-sensitive -- Wildcards (`*`) match any sequence of characters -- Patterns apply to repository names, not full paths -- Multiple patterns can be combined using shell globbing +The script converts `*` to a regex-style wildcard internally. Matching is done against the repository name, not the full `owner/repo` path. -## Interactive Confirmation System +## Confirmation Modes -During the removal process, users have multiple response options: +During interactive execution, each repository prompt supports: -### Confirmation Options -- **`[y]es`** - Remove user from current repository -- **`[n]o`** - Skip current repository -- **`[a]ll yes`** - Remove user from all remaining repositories -- **`[s]kip all`** - Skip all remaining repositories -- **`[q]uit`** - Cancel entire operation +- `y` + Remove from the current repository. +- `n` + Skip the current repository. +- `a` + Remove from all remaining repositories. +- `s` + Skip all remaining repositories. +- `q` + Exit immediately. -### Confirmation Modes -- **`ask`** - Default mode, prompts for each repository -- **`yes_all`** - Automatically confirm all operations -- **`no_all`** - Automatically skip all operations +## Exit And Result Notes -## Repository Discovery +- `204` from the GitHub API means the collaborator removal succeeded. +- `404` usually means the user is not a collaborator on that repository, or the repository could not be resolved with the current access. +- `403` during collaborator checks usually means the token lacks sufficient access for that repository. -The tool performs comprehensive repository discovery: +At the end, the script prints counts for: -### Discovery Process -1. **Root User Repositories**: Fetches all repositories owned by the root user -2. **Organization Discovery**: Identifies organizations the root user belongs to -3. **Organization Repositories**: Fetches repositories from each organization -4. **Pattern Filtering**: Applies wildcard patterns to repository names -5. **Deduplication**: Removes duplicate repositories from the final list +- processed repositories +- successful removals +- failed removals +- skipped removals -### API Endpoints Used -- `GET /users/{user}/repos` - User repositories -- `GET /users/{user}/orgs` - User organizations -- `GET /orgs/{org}/repos` - Organization repositories -- `DELETE /repos/{owner}/{repo}/collaborators/{username}` - Remove collaborator +## Operational Guidance -## Error Handling - -The tool includes comprehensive error handling: - -### HTTP Status Codes -- **`204`** - Successful removal -- **`404`** - User not found or not a collaborator -- **`403`** - Insufficient permissions -- **`401`** - Invalid authentication - -### Error Recovery -- **Network Failures**: Retry mechanisms for API calls -- **Rate Limiting**: Respects GitHub API rate limits -- **Permission Errors**: Clear error messages and suggestions -- **Invalid Input**: Input validation and correction prompts - -## Logging and Output - -### Log Levels -- **Info** (ℹī¸): General information messages -- **Success** (✅): Successful operations -- **Warning** (⚠ī¸): Non-critical issues -- **Error** (❌): Critical failures -- **Progress** (🔄): Operation progress updates - -### Output Format -``` -[EMOJI] [TIMESTAMP] [LEVEL] Message -``` - -### Summary Report -``` -Operation Summary -================= -Total repositories processed: 15 -Successful removals: 12 -Failed removals: 1 -Skipped removals: 2 -Success rate: 80% -``` - -## Prerequisites - -### Required Tools -- `curl` - HTTP client for API requests -- `jq` - JSON processor for API responses - -### GitHub Token Permissions -- `repo` - Repository access (read/write) -- `admin:org` - Organization administration - -### Environment Requirements -- Bash 4.0 or higher -- Internet connectivity for GitHub API access -- Valid GitHub Personal Access Token - -## Security Considerations - -### Token Security -- Tokens are not stored or logged -- Input is hidden during token entry -- Tokens are validated before use - -### Operation Safety -- Interactive confirmations prevent accidental operations -- Detailed logging for audit trails -- Rollback information provided for manual recovery - -### Access Control -- Respects GitHub repository permissions -- Validates user access before operations -- Handles permission errors gracefully - -## Troubleshooting - -### Common Issues -1. **Token Validation Fails**: Ensure token has correct permissions -2. **Repository Not Found**: Check repository name and access permissions -3. **User Not Found**: Verify username spelling and existence -4. **Rate Limiting**: Wait for rate limit reset or use token with higher limits - -### Debug Mode -Enable verbose output by setting: -```bash -export DEBUG=1 -./remove_user_interactive.sh -``` - -## Integration - -The tool integrates seamlessly with other ByteLyst DevOps tools: -- Can be called from `bytelyst-cli.sh` -- Supports JSON configuration files -- Compatible with existing workflow scripts -- Follows established logging and error handling patterns - -## Performance - -### Optimization Features -- Paginated API requests for large datasets -- Parallel processing where possible -- Efficient pattern matching algorithms -- Minimal memory footprint - -### Scale Considerations -- Handles hundreds of repositories efficiently -- Respects GitHub API rate limits -- Provides progress feedback for long operations -- Supports interruption and resumption \ No newline at end of file +- Prefer running without `--non-interactive` first unless you have already validated the repository pattern. +- Use a narrow pattern before using `*` against a large owner or organization. +- Keep the token out of shell history when possible. Prompted entry is safer than embedding it directly in commands. +- Cross-check `docs/operations.md` and `docs/supported-scripts.md` if you are choosing between this script, `remove_user_guided.sh`, and `remove_user_from_repos.sh`. diff --git a/docs/repo-map.md b/docs/repo-map.md index d0a68a9..beac8d5 100644 --- a/docs/repo-map.md +++ b/docs/repo-map.md @@ -8,6 +8,8 @@ High-level map of the repository for contributors and AI agents. - `bytelyst-cli.sh` - Main CLI entry point for common GitHub operations. +- `setup.sh` + - Bootstrap script for installing `pre-commit` and local git hooks. - `remove_user_interactive.sh` - Interactive collaborator removal across matching repositories. - `remove_user_guided.sh` @@ -23,6 +25,7 @@ High-level map of the repository for contributors and AI agents. ### Supporting Data And Inputs +- `accounts.example.json` - `github_repos.json` - `github_acc_input.json` - `repos.json` @@ -44,8 +47,18 @@ Current key files: - `docs/getting-started.md` - `docs/repo-map.md` - `docs/tooling-status.md` +- `docs/supported-scripts.md` +- `docs/operations.md` - `docs/remove_user_interactive.md` +### `.github/workflows/` + +Repository automation and validation. + +Key files: + +- `pre-commit.yml` +- `shell-validation.yml` ### `git-work-safety-tools/` Scripts for scanning many git repositories and performing safer bulk workflows. @@ -75,6 +88,7 @@ Key files: - `check_repo_access.sh` - `list_user_repos.sh` +- `README.md` ### `github_repo_scanners/` @@ -140,10 +154,23 @@ These still exist but are no longer the best starting point: Prefer `docs/` for new canonical documentation. +## Legacy Root Utilities + +These still exist in the repo root but are not the preferred discovery path: + +- `interactive_user_removal.sh` +- `cleanup.sh` +- `sync_repos.sh` +- `test.sh` +- `test_interactive.sh` + +Treat them as compatibility or one-off helpers unless a canonical doc points to them directly. + ## Cleanup Guidance When cleaning this repo: - do not delete tracked outputs or snapshots unless explicitly asked - prefer adding orientation docs over moving operational files +- prefer documenting supported entry points over renaming legacy root scripts - avoid renaming legacy scripts unless you also provide compatibility shims or update all references