97 lines
3.5 KiB
Markdown
97 lines
3.5 KiB
Markdown
# ByteLyst private Open Notebook pilot
|
|
|
|
Status: private localhost-only pilot. Do not publish as a public ByteLyst dashboard/subdomain.
|
|
|
|
## Files and runtime locations
|
|
|
|
- Compose: `deployments/open-notebook-pilot/docker-compose.yml`
|
|
- Protected env/secrets: `/etc/bytelyst/open-notebook-pilot.env` (mode `0600`, not in git)
|
|
- Runtime data: `/opt/bytelyst/open-notebook-pilot/`
|
|
- `surreal_data/`: SurrealDB RocksDB data
|
|
- `notebook_data/`: Open Notebook uploaded/app data
|
|
- `ollama/`: local Ollama model cache
|
|
- `test-docs/`: sanitized pilot documents
|
|
- `reports/`: smoke/evaluation outputs
|
|
- `backups/`: protected backup archives
|
|
|
|
## Network hardening
|
|
|
|
- Web UI binds to `127.0.0.1:8502` only.
|
|
- REST API binds to `127.0.0.1:5055` only.
|
|
- SurrealDB has no host-published port.
|
|
- Ollama has no host-published port.
|
|
- Remote access, if needed, must use SSH tunnel/VPN/private route only:
|
|
- `ssh -L 8502:127.0.0.1:8502 -L 5055:127.0.0.1:5055 <vm>`
|
|
- Do not add this service to the public Caddyfile. If a private reverse proxy is ever required, put it behind strong auth and keep `/docs`, `/openapi.json`, and `/health` away from the public internet.
|
|
|
|
## Secrets
|
|
|
|
`/etc/bytelyst/open-notebook-pilot.env` contains:
|
|
|
|
- `OPEN_NOTEBOOK_ENCRYPTION_KEY`
|
|
- `OPEN_NOTEBOOK_PASSWORD`
|
|
- `SURREAL_USER`
|
|
- `SURREAL_PASSWORD`
|
|
- explicit `CORS_ORIGINS`
|
|
- `OLLAMA_BASE_URL=http://ollama:11434`
|
|
|
|
Do not print this file in logs or commit it. Losing or changing the encryption key can make stored provider credentials unreadable.
|
|
|
|
## Operations
|
|
|
|
From the repo root:
|
|
|
|
```bash
|
|
docker compose --env-file /etc/bytelyst/open-notebook-pilot.env -f deployments/open-notebook-pilot/docker-compose.yml config --quiet
|
|
docker compose --env-file /etc/bytelyst/open-notebook-pilot.env -f deployments/open-notebook-pilot/docker-compose.yml up -d
|
|
```
|
|
|
|
Health checks:
|
|
|
|
```bash
|
|
curl -fsS http://127.0.0.1:5055/health
|
|
curl -fsS http://127.0.0.1:5055/api/auth/status
|
|
curl -fsSI http://127.0.0.1:8502
|
|
```
|
|
|
|
Use the bearer password from the protected env file for API calls:
|
|
|
|
```bash
|
|
curl -fsS -H "Authorization: Bearer <password-from-/etc/bytelyst/open-notebook-pilot.env>" http://127.0.0.1:5055/api/notebooks
|
|
```
|
|
|
|
## Ollama local provider
|
|
|
|
The compose stack includes an internal Ollama service at `http://ollama:11434` for the Open Notebook container. Initial pilot model targets:
|
|
|
|
- LLM: `qwen2.5:0.5b` or another small local model that fits current VM headroom.
|
|
- Embeddings: `nomic-embed-text` if pull time/resources allow.
|
|
|
|
Configure/register models through Open Notebook Settings or the `/models` API. Cloud providers are allowed only with sanitized comparison data.
|
|
|
|
## Sanitized ingestion and smoke search
|
|
|
|
Sanitized docs live in `/opt/bytelyst/open-notebook-pilot/test-docs`. Run:
|
|
|
|
```bash
|
|
scripts/open-notebook-pilot/ingest-and-search-smoke.sh
|
|
```
|
|
|
|
The script creates a pilot notebook, ingests text sources without embeddings, and writes text-search smoke outputs under `/opt/bytelyst/open-notebook-pilot/reports/`.
|
|
|
|
## Backup and restore test
|
|
|
|
Backup:
|
|
|
|
```bash
|
|
scripts/open-notebook-pilot/backup.sh
|
|
```
|
|
|
|
Restore-readiness test:
|
|
|
|
```bash
|
|
scripts/open-notebook-pilot/restore-test.sh /opt/bytelyst/open-notebook-pilot/backups/<archive>.tar.gz
|
|
```
|
|
|
|
The restore test extracts to `/opt/bytelyst/open-notebook-pilot/restore-test/extracted` and verifies the backup contains SurrealDB data, notebook data, and the protected env file. Actual restore should be performed only during planned downtime by stopping the stack and replacing the runtime directories/env from the verified archive.
|