Hermes VM 76ef17f26b feat(vm): Phase 2.3 closure — OOM watchdog + orphan-container docs

OOM watchdog:
- vm-oom-watchdog.sh — scans journalctl -k since cursor for oom-kill,
  killed-process, and "out of memory ... killed" entries; maps cgroup
  hits back to container names via docker inspect; posts a single
  Telegram alert per scan window (no dedupe needed — cursor advances
  on every run). Cursor at /var/log/vm-oom-cursor, log at
  /var/log/vm-oom-watchdog.log.
- Systemd: OnBootSec=10min, OnUnitActiveSec=1h, Persistent=true.

Orphan containers (no compose file on disk):
- trading-backend → docker update --memory=768m (high-I/O bot)
- gitea-npm-registry → docker update --memory=512m
- orphan-containers.md captures canonical configs for recovery
  (env, mounts, networks, restart policy, memory limits).

Closes Phase 2.3 (post-monitoring) and Phase 3.3 (orphan limits).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-05-30 05:26:49 +00:00

2.8 KiB

Raw Blame History

Orphan Containers — Canonical Configuration

Two containers on this VM run without an on-disk compose file:

Container	Image	Memory Limit	Restart Policy	Source
`trading-backend`	`bytelyst-trading-bot-service-trading-backend`	768 MiB	always	compose file deleted; image present
`gitea-npm-registry`	`gitea/gitea:1.22`	512 MiB	unless-stopped	started via `docker run`

Both have runtime memory limits applied via docker update. Limits persist across restarts of the same container but are lost on docker rm. If recreated, use the recovery procedures below.

trading-backend

Compose file referenced by container labels: /opt/bytelyst/trading/bytelyst-trading-bot-service/docker-compose.yml — missing on disk.

Only the logs/ directory remains; the bind mount references it.

Recovery procedure

If the container is removed, recreate the compose file with:

services:
  trading-backend:
    image: bytelyst-trading-bot-service-trading-backend
    container_name: trading-backend
    user: node
    working_dir: /app
    entrypoint: ["docker-entrypoint.sh"]
    command: ["node", "dist/index.js"]
    env_file:
      - .env                # holds secrets — restore from backup
    environment:
      ENABLE_TRADING: "true"
      PAPER_TRADING: "true"
      EXECUTION_PROVIDER: alpaca
      ALPACA_SUBTAG_ENV: paper
      SYMBOLS: "BTC/USDT,ETH/USDT,SOL/USDT,DOGE/USDT,PEPE/USDT"
      TOTAL_CAPITAL: "1000"
      EXECUTION_TIMEFRAME: 15m
      POLLING_INTERVAL: "60000"
    volumes:
      - /opt/bytelyst/trading/bytelyst-trading-bot-service/logs:/app/logs
    networks:
      - default
    restart: always
    deploy:
      resources:
        limits:
          memory: 768m

networks:
  default:
    name: learning_ai_common_plat_default
    external: true

Full env captured via docker inspect is needed for secrets; check .env backup or recover from hermes drive backup.

gitea-npm-registry

Started via raw docker run rather than compose. The internal package registry service used by ByteLyst monorepos for @bytelyst/* package distribution.

Recovery procedure

docker run -d \
  --name gitea-npm-registry \
  --restart unless-stopped \
  --memory 512m --memory-swap 512m \
  -p 3300:3000 \
  -v gitea-data:/data \
  -e GITEA__server__DOMAIN=gitea.bytelyst.com \
  -e GITEA__server__SSH_DOMAIN=gitea.bytelyst.com \
  -e GITEA__server__ROOT_URL=https://gitea.bytelyst.com/ \
  -e GITEA__server__HTTP_PORT=3000 \
  -e GITEA__packages__ENABLED=true \
  -e GITEA__security__INSTALL_LOCK=true \
  -e INSTALL_LOCK=true \
  --network bridge \
  gitea/gitea:1.22

docker network connect learning_ai_common_plat_default gitea-npm-registry

The gitea-data named volume contains all repository + package state; never delete it without a backup.

2.8 KiB Raw Blame History

Orphan Containers — Canonical Configuration

trading-backend

Recovery procedure

gitea-npm-registry

Recovery procedure

2.8 KiB

Raw Blame History