# Track A: Azure VM — Gateway, Gitea & Security

> **Agent:** Codex on Azure VM (`/opt/bytelyst/`)
> **Prompts:** 4 (run in dependency order below)
> **Unblocks:** Track B Prompt B4 (`.npmrc` migration — needs `gitea.bytelyst.com` live)
> **Reference docs:**
>
> - [`SECURE_API_EXPOSURE.md`](../single_azure_vm/docker/SECURE_API_EXPOSURE.md) — Caddy decision + architecture
> - [`DEPLOYMENT_STATUS_2026-03-29.md`](../single_azure_vm/docker/DEPLOYMENT_STATUS_2026-03-29.md) — current VM status
> - [`ECOSYSTEM_WEB_APPS_INVENTORY.md`](./ECOSYSTEM_WEB_APPS_INVENTORY.md) — full Vercel audit
> - [`GODADDY_DNS_SETUP_BYTELYST.md`](./GODADDY_DNS_SETUP_BYTELYST.md) — GoDaddy DNS runbook for `bytelyst.com`

---

## Progress Tracker

| #   | Prompt                     |     Status     | Commit SHA | Verified |
| --- | -------------------------- | :------------: | :--------: | :------: |
| A1  | Caddy Gateway Setup        | 🟨 In progress |     —      |    ⬜    |
| A2  | Gitea HTTPS Exposure       | ⬜ Not started |     —      |    ⬜    |
| A3  | Internal Dashboard Hosting | ⬜ Not started |     —      |    ⬜    |
| A4  | NSG Lockdown               | ⬜ Not started |     —      |    ⬜    |

**Execution order:**

```
A1: Caddy Gateway  ──→  A2: Gitea HTTPS  ──→  A4: NSG Lockdown
                    └──→  A3: Internal Dashboards on VM Docker
```

---

## DNS Prerequisites (manual — do before starting)

These VM-hosted DNS records must exist before Caddy can obtain Let's Encrypt certificates:

| Record                 | Type | Value                  | Purpose             |
| ---------------------- | ---- | ---------------------- | ------------------- |
| `api.bytelyst.com`     | A    | `<Azure VM public IP>` | Backend API gateway |
| `gitea.bytelyst.com`   | A    | `<Azure VM public IP>` | Gitea npm registry  |
| `admin.bytelyst.com`   | A    | `<Azure VM public IP>` | Internal admin UI   |
| `tracker.bytelyst.com` | A    | `<Azure VM public IP>` | Internal tracker UI |

**Verify DNS before starting A1:**

```bash
dig +short api.bytelyst.com      # must return VM IP
dig +short gitea.bytelyst.com    # must return VM IP
dig +short admin.bytelyst.com    # must return VM IP
dig +short tracker.bytelyst.com  # must return VM IP
```

### Internal-only services policy

Keep these classes of services on the VM in Docker and treat them as internal-only:

- admin and operator dashboards
- monitoring and observability tools such as Grafana, Loki, Prometheus, and Alertmanager
- future cache and datastore services such as Valkey and PostgreSQL + pgvector

Do not expose raw service ports publicly. If browser access is required, front the service through Caddy with authentication and preferably IP allowlisting, VPN, or SSO.

---

## A1: Caddy Gateway — Replace Traefik with Caddy

> **Depends on:** DNS records for `api.bytelyst.com`
> **Blocks:** A2, A4

### Preconditions

- [ ] `api.bytelyst.com` DNS A record resolves to Azure VM IP
- [ ] All 13 backend services are healthy: `docker compose ps | grep -c healthy` → 13+
- [ ] Ports 80 and 443 are open in Azure NSG

### Prompt

```
You are working on the ByteLyst single-Azure-VM Docker deployment at /opt/bytelyst/.

TASK: Replace the existing Traefik gateway container with Caddy to provide HTTPS termination and path-based routing for all backend services.

CONTEXT:
- The decision doc is at docs/devops/single_azure_vm/docker/SECURE_API_EXPOSURE.md in the learning_ai_common_plat repo
- The domain is bytelyst.com — DNS A records already point to this VM
- API hostname: api.bytelyst.com
- Currently 13 backend services are healthy on ports 4003, 4005, 4007, 4010-4019
- Traefik is running but only as a passthrough — no TLS, no path routing hardened
- All backend containers are on a Docker network

TASK LIST:
- [ ] 1. Create a Caddyfile at /opt/bytelyst/Caddyfile with:
   - api.bytelyst.com as the main host (automatic Let's Encrypt TLS)
   - Path-based routing to all 13 backends:
     /platform/*    → platform-service:4003
     /extraction/*  → extraction-service:4005
     /mcp/*         → mcp-server:4007
     /peakpulse/*   → peakpulse:4010
     /chronomind/*  → chronomind:4011
     /jarvisjr/*    → jarvisjr:4012
     /nomgap/*      → nomgap:4013
     /mindlyst/*    → mindlyst:4014
     /lysnrai/*     → lysnrai:4015
     /notelett/*    → notelett:4016
     /flowmonk/*    → flowmonk:4017
     /actiontrail/* → actiontrail:4018
     /localmemgpt/* → localmemgpt:4019
   - Strip the path prefix before forwarding (backends expect / not /platform/)
- [ ] 2. Add a caddy service to docker-compose replacing or alongside the gateway service
         - Image: caddy:2-alpine
         - Volumes: ./Caddyfile:/etc/caddy/Caddyfile, caddy_data:/data, caddy_config:/config
         - Ports: 80:80, 443:443
         - Network: same Docker network as all backends
- [ ] 3. Remove published backend ports from docker-compose (4003, 4005, 4007, 4010-4019)
         — backends remain accessible internally via Docker DNS
- [ ] 4. Restart the stack: docker compose up -d
- [ ] 5. Wait for Caddy to obtain TLS certificates (check logs: docker logs caddy --tail 50)
- [ ] 6. Update docs/devops/single_azure_vm/docker/README.md with the new access model

VERIFICATION (run all — every check must pass):
  # TLS certificate obtained
  curl -sI https://api.bytelyst.com/platform/health | head -5
  # Expected: HTTP/2 200 + valid TLS

  # All 13 backends reachable via path routing
  for svc in platform extraction mcp peakpulse chronomind jarvisjr nomgap mindlyst lysnrai notelett flowmonk actiontrail localmemgpt; do
    echo -n "$svc: "; curl -sf https://api.bytelyst.com/$svc/health | jq -r '.status // "FAIL"'
  done
  # Expected: all 13 print "ok"

  # Direct port access blocked from outside
  curl -sf --max-time 3 http://<VM_PUBLIC_IP>:4003/health && echo "FAIL: port still open" || echo "PASS: port closed"
  # Expected: "PASS: port closed"

SUCCESS CRITERIA:
- All 13 health checks return "ok" via https://api.bytelyst.com/<service>/health
- TLS certificate is valid (Let's Encrypt)
- No backend ports (4003-4019) accessible directly from public internet
- Traefik config preserved but not in use

ROLLBACK:
If Caddy fails, revert docker-compose changes and restart Traefik:
  git checkout -- docker-compose*.yml
  docker compose up -d

DO NOT:
- Delete the existing Traefik config — just stop using it
- Expose any backend port publicly
- Hardcode IP addresses in the Caddyfile

COMMIT: feat(gateway): replace Traefik with Caddy for HTTPS path routing
```

---

## A2: Gitea HTTPS Exposure via Caddy

> **Depends on:** A1 ✅ (Caddy running + TLS working)
> **Blocks:** Track B → B4 (`.npmrc` update across all repos)

### Preconditions

- [ ] A1 is complete — Caddy is running with valid TLS on `api.bytelyst.com`
- [ ] `gitea.bytelyst.com` DNS A record resolves to Azure VM IP
- [ ] Gitea container is running: `docker compose ps gitea` → healthy

### Prompt

```
You are working on the ByteLyst single-Azure-VM Docker deployment at /opt/bytelyst/.

TASK: Expose the Gitea npm registry over HTTPS via the Caddy gateway so that external build systems (Vercel) can install @bytelyst/* packages.

CONTEXT:
- Gitea is running on port 3300 inside the Docker network
- Caddy is already configured for api.bytelyst.com (from A1) and has valid TLS
- The Gitea npm registry URL pattern is: /api/packages/ByteLyst/npm/
- Gitea needs its own subdomain because its internal routing expects to be at root
- Target URL: https://gitea.bytelyst.com

TASK LIST:
- [ ] 1. Add a gitea.bytelyst.com block to the Caddyfile:
         gitea.bytelyst.com {
             reverse_proxy gitea:3300
         }
- [ ] 2. Reload Caddy: docker exec caddy caddy reload --config /etc/caddy/Caddyfile
- [ ] 3. Update Gitea's ROOT_URL to https://gitea.bytelyst.com
         — Edit docker-compose env: SERVER__ROOT_URL=https://gitea.bytelyst.com
         — Restart Gitea container: docker compose restart gitea
- [ ] 4. Remove port 3300 from docker-compose published ports (Gitea only via Caddy)
- [ ] 5. Remove stale public NSG rules for ports that should not be exposed:
         - 3300 (Gitea)
         - 8025 (Mailpit)
         - 8080 (Traefik dashboard — no longer needed)
         - 10000 (Azurite)

VERIFICATION (run all — every check must pass):
  # Gitea web UI loads
  curl -sI https://gitea.bytelyst.com | head -3
  # Expected: HTTP/2 200 (or 302 redirect to login)

  # npm registry returns package metadata
  curl -sf https://gitea.bytelyst.com/api/packages/ByteLyst/npm/@bytelyst%2ferrors | jq '.name'
  # Expected: "@bytelyst/errors"

  # Authenticated npm install works
  export GITEA_NPM_TOKEN=<token>
  npm view @bytelyst/errors --registry=https://gitea.bytelyst.com/api/packages/ByteLyst/npm/
  # Expected: package info printed (version, description, etc.)

  # Direct port 3300 access blocked from outside
  curl -sf --max-time 3 http://<VM_PUBLIC_IP>:3300 && echo "FAIL" || echo "PASS: port closed"
  # Expected: "PASS: port closed"

  # Internal Docker access still works
  docker exec platform-service curl -sf http://gitea:3300/api/v1/version | jq '.version'
  # Expected: Gitea version string

SUCCESS CRITERIA:
- https://gitea.bytelyst.com serves Gitea web UI with valid TLS
- npm registry API returns valid package metadata
- Authenticated `npm view` succeeds
- Port 3300 not accessible from public internet
- Docker-internal access to gitea:3300 still works (other containers unaffected)

DO NOT:
- Break existing Docker-internal Gitea access (other containers still use gitea:3300)
- Remove the Gitea container or change its internal port

COMMIT: feat(gateway): expose Gitea npm registry via HTTPS at gitea.bytelyst.com

AFTER COMPLETING: Notify the operator that Track B → B4 (.npmrc migration) is now unblocked.
```

---

## A3: Keep admin-web and tracker-web on the VM in Docker

> **Depends on:** A1 ✅ for `api.bytelyst.com`
> **Blocks:** Nothing on the VM track
> **Can run in parallel with:** A2

### Preconditions

- [ ] `api.bytelyst.com` is live on the VM with HTTPS
- [ ] `admin.bytelyst.com` and `tracker.bytelyst.com` DNS A records resolve to the Azure VM IP
- [ ] `admin-web` and `tracker-web` source code is available in the repo

### Prompt

```
You are working on the ByteLyst deployment split where internal web apps stay on the Azure VM in Docker, while only clearly public sites may be deployed outside the VM later.

TASK: Keep admin-web and tracker-web on the Azure VM as Docker-hosted internal dashboards fronted by Caddy, and document the access-control expectations for those internal tools.

CONTEXT:
- Internal/admin/operator-facing web apps should stay on the VM
- admin-web and tracker-web are Next.js frontends from learning_ai_common_plat under dashboards/
- Monitoring and observability tools should also stay on the VM:
  - Grafana
  - Loki
  - Prometheus / Alertmanager if added
  - other internal ops dashboards
- Their hostnames should be:
  - https://admin.bytelyst.com
  - https://tracker.bytelyst.com
- Those domains should point to the VM, not Vercel
- Browser/API configuration should use the VM backend entrypoint at https://api.bytelyst.com
- These dashboards are internal surfaces and should not be treated as public sites
- Monitoring stacks must also remain internal-only and should not expose raw ports publicly

TASK LIST:
- [ ] 1. Audit both dashboards for browser-visible backend URLs and ensure they use https://api.bytelyst.com where appropriate
- [ ] 2. Confirm both dashboard containers are part of the VM compose topology and expose only Docker-internal ports
- [ ] 3. Add Caddy host blocks for:
         - admin.bytelyst.com -> admin-web:3001
         - tracker.bytelyst.com -> tracker-web:3003
- [ ] 4. Ensure `admin.bytelyst.com` and `tracker.bytelyst.com` DNS A records point to the VM
- [ ] 5. Update deployment docs to record that internal dashboards stay on the VM in Docker
- [ ] 6. Document required access controls for internal dashboards:
         - app auth at minimum
         - optional IP allowlist/VPN/SSO recommendation
         - do not treat these as public sites

VERIFICATION (run all — every check must pass):
  # DNS should point at the VM
  dig +short admin.bytelyst.com
  dig +short tracker.bytelyst.com
  # Expected: VM public IP for both

  # Frontends load through VM Caddy
  curl -sI https://admin.bytelyst.com | head -3
  curl -sI https://tracker.bytelyst.com | head -3
  # Expected: HTTP 200 or 302 with valid TLS

  # VM backend stays the single API entrypoint
  curl -sf https://api.bytelyst.com/platform/health | jq -r '.status'
  # Expected: ok

  # Caddy config includes dashboard host routing
  rg -n "admin.bytelyst.com|tracker.bytelyst.com" /opt/bytelyst/Caddyfile

SUCCESS CRITERIA:
- admin.bytelyst.com is served by the VM through Caddy
- tracker.bytelyst.com is served by the VM through Caddy
- Both frontends use https://api.bytelyst.com for backend access
- The dashboards remain internal VM-hosted tools with documented access controls

DO NOT:
- Move admin/tracker to Vercel
- Treat admin/tracker as public-facing web apps
- Expose raw dashboard container ports publicly

COMMIT: docs(architecture): keep internal dashboards on VM Docker
```

---

## A4: NSG Lockdown — Reduce Public Port Exposure

> **Depends on:** A1 + A2 complete
> **Blocks:** Nothing
> **Run this LAST in Track A**

### Preconditions

- [ ] A1 complete — all backends accessible via `api.bytelyst.com`
- [ ] A2 complete — Gitea accessible via `gitea.bytelyst.com`
- [ ] All services verified working through Caddy (no direct port access needed)

### Prompt

```
TASK: Reduce the Azure NSG (Network Security Group) rules so only ports 80, 443, and 22 are publicly accessible.

CONTEXT:
- The Azure VM currently exposes many service ports publicly (3000-3300, 4003-4019, 8025, 8080, 10000, 11434)
- Caddy now handles the VM-hosted public HTTPS traffic on ports 80/443
- Only backend and self-hosted services should remain on the VM
- Only SSH (22) and HTTP/HTTPS (80/443) should remain publicly open
- This follows the recommendation in SECURE_API_EXPOSURE.md

TASK LIST:
- [ ] 1. Snapshot current NSG rules (save output for rollback reference):
         az network nsg rule list --resource-group <rg> --nsg-name <nsg> -o table > /opt/bytelyst/nsg-rules-before.txt
         cat /opt/bytelyst/nsg-rules-before.txt
- [ ] 2. Identify rules to remove — everything except:
         - SSH (22/tcp)
         - HTTP (80/tcp)
         - HTTPS (443/tcp)
- [ ] 3. Delete each unnecessary rule:
         az network nsg rule delete --resource-group <rg> --nsg-name <nsg> --name <rule-name>
- [ ] 4. List final NSG rules to confirm only 3 remain:
         az network nsg rule list --resource-group <rg> --nsg-name <nsg> -o table
- [ ] 5. Document changes in DEPLOYMENT_STATUS_2026-03-29.md

VERIFICATION (run all — every check must pass):
  # SSH still works
  ssh -o ConnectTimeout=5 <user>@<VM_IP> echo "SSH OK"
  # Expected: "SSH OK"

  # Caddy still serves all routes
  curl -sf https://api.bytelyst.com/platform/health | jq '.status'
  # Expected: "ok"

  curl -sf https://gitea.bytelyst.com/api/v1/version | jq '.version'
  # Expected: version string

  # Direct port access blocked
  for port in 3001 3003 3300 4003 4005 8025 8080 10000; do
    curl -sf --max-time 3 http://<VM_PUBLIC_IP>:$port && echo "FAIL: $port open" || echo "PASS: $port closed"
  done
  # Expected: all "PASS: <port> closed"

  # Only 3 NSG rules remain
  az network nsg rule list --resource-group <rg> --nsg-name <nsg> -o table | grep -c Allow
  # Expected: 3 (SSH, HTTP, HTTPS)

SUCCESS CRITERIA:
- Only ports 22, 80, 443 open in NSG
- SSH access confirmed working
- All Caddy routes still functional
- No direct port access possible from public internet
- Before/after NSG snapshots saved

ROLLBACK:
If services become unreachable after lockdown:
  # Re-add a rule temporarily
  az network nsg rule create --resource-group <rg> --nsg-name <nsg> \
    --name AllowTemp --priority 200 --access Allow --direction Inbound \
    --source-address-prefixes '*' --destination-port-ranges <port> --protocol Tcp
  # Then investigate Caddy routing

DO NOT:
- Remove the SSH (22) rule — you will lose access
- Remove the HTTP (80) or HTTPS (443) rules — Caddy needs them
- Make changes without first saving the current rules snapshot

COMMIT: chore(security): lock down Azure NSG to 22/80/443 only
```

---

## Track A — Final Verification Checklist

After all 4 prompts are complete, run this full end-to-end verification:

```bash
echo "=== Track A Final Verification ==="

echo "--- DNS ---"
for d in api.bytelyst.com gitea.bytelyst.com admin.bytelyst.com tracker.bytelyst.com; do
  echo -n "$d → "; dig +short $d
done

echo "--- Backend Health (13 services) ---"
for svc in platform extraction mcp peakpulse chronomind jarvisjr nomgap mindlyst lysnrai notelett flowmonk actiontrail localmemgpt; do
  echo -n "$svc: "; curl -sf https://api.bytelyst.com/$svc/health | jq -r '.status // "FAIL"'
done

echo "--- Gitea npm registry ---"
curl -sf https://gitea.bytelyst.com/api/packages/ByteLyst/npm/@bytelyst%2ferrors | jq '.name'

echo "--- Dashboards (VM-hosted internal tools) ---"
curl -sI https://admin.bytelyst.com | head -1
curl -sI https://tracker.bytelyst.com | head -1

echo "--- Port lockdown ---"
for port in 3001 3003 3300 4003 4005 8025 8080 10000; do
  curl -sf --max-time 3 http://<VM_PUBLIC_IP>:$port > /dev/null 2>&1 && echo "FAIL: $port open" || echo "PASS: $port closed"
done

echo "--- Docker containers ---"
docker compose ps --format 'table {{.Name}}\t{{.Status}}' | grep -E "(unhealthy|Exit)" | wc -l
# Expected: 0
```

**All checks must pass before Track A is considered complete.**
Once A2 verification passes, signal to Track B that Prompt B4 is unblocked.
