Add Cosmos emulator prototype support

This commit is contained in:
root 2026-03-14 05:24:01 +00:00
parent 9e2fdb9643
commit a5e8890df2
9 changed files with 59 additions and 20 deletions

View File

@ -3,15 +3,17 @@
# ── Azure Key Vault (optional — secrets fall back to env vars) ─ # ── Azure Key Vault (optional — secrets fall back to env vars) ─
# Set this to resolve secrets from AKV instead of .env: # Set this to resolve secrets from AKV instead of .env:
AZURE_KEYVAULT_URL=https://kv-mywisprai.vault.azure.net AZURE_KEYVAULT_URL=
# ── Azure Cosmos DB ──────────────────────────────────────────── # ── Cosmos DB (prototype defaults to local emulator) ───────────
COSMOS_ENDPOINT=https://cosmos-mywisprai.documents.azure.com:443/ # For the Docker prototype stack, leave these pointed at the local emulator.
COSMOS_KEY=your-cosmos-key # When you move to a managed environment later, replace them with real Azure values.
COSMOS_ENDPOINT=http://cosmos-emulator:8081
COSMOS_KEY=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==
COSMOS_DATABASE=lysnrai COSMOS_DATABASE=lysnrai
# ── Auth (platform-service) ───────────────────────── # ── Auth (platform-service) ─────────────────────────
JWT_SECRET=your-jwt-secret JWT_SECRET=change-me-prototype-jwt-secret
# ── Azure Blob Storage (platform-service) ───────────────────── # ── Azure Blob Storage (platform-service) ─────────────────────
AZURE_BLOB_CONNECTION_STRING= AZURE_BLOB_CONNECTION_STRING=

1
.gitignore vendored
View File

@ -6,6 +6,7 @@ coverage/
# Env / Secrets # Env / Secrets
.env .env
.env.bak
.env.local .env.local
.env.*.local .env.*.local
*.pem *.pem

View File

@ -42,6 +42,8 @@ cp .env.example .env
See [docs/PROTOTYPE_DEPLOYMENT.md](docs/PROTOTYPE_DEPLOYMENT.md) for the required environment variables and day-to-day commands. See [docs/PROTOTYPE_DEPLOYMENT.md](docs/PROTOTYPE_DEPLOYMENT.md) for the required environment variables and day-to-day commands.
The prototype stack now includes a local Cosmos DB Emulator container, so the default `.env.example` values are wired for single-VM Docker use.
## Current Capability Surface ## Current Capability Surface
- **Shared packages** — 36 `@bytelyst/*` packages covering auth, config, API clients, storage, sync, telemetry, diagnostics, design tokens, SDK support, and testing. - **Shared packages** — 36 `@bytelyst/*` packages covering auth, config, API clients, storage, sync, telemetry, diagnostics, design tokens, SDK support, and testing.

View File

@ -1,4 +1,26 @@
services: services:
# ── Azure Cosmos DB Emulator (prototype only) ─────────────────
cosmos-emulator:
image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-preview
ports:
- '8081:8081'
- '1234:1234'
environment:
- PROTOCOL=http
- ENABLE_EXPLORER=true
- GATEWAY_PUBLIC_ENDPOINT=cosmos-emulator
healthcheck:
test:
[
'CMD-SHELL',
'bash -lc ''exec 3<>/dev/tcp/127.0.0.1/8080; printf "GET /ready HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n" >&3; grep -q "200 OK" <&3''',
]
interval: 10s
timeout: 5s
retries: 12
start_period: 20s
restart: unless-stopped
# ── Loki (Log Aggregation) ──────────────────────────────────── # ── Loki (Log Aggregation) ────────────────────────────────────
loki: loki:
image: grafana/loki:3.3.2 image: grafana/loki:3.3.2
@ -74,6 +96,9 @@ services:
- PORT=4003 - PORT=4003
# Local/dev convenience: ensure Cosmos DB + containers exist. # Local/dev convenience: ensure Cosmos DB + containers exist.
- COSMOS_AUTO_INIT=true - COSMOS_AUTO_INIT=true
depends_on:
cosmos-emulator:
condition: service_healthy
labels: labels:
- 'traefik.enable=true' - 'traefik.enable=true'
- 'traefik.http.routers.platform.rule=PathPrefix(`/api`) || PathPrefix(`/public`) || PathPrefix(`/health`)' - 'traefik.http.routers.platform.rule=PathPrefix(`/api`) || PathPrefix(`/public`) || PathPrefix(`/health`)'
@ -97,13 +122,22 @@ services:
environment: environment:
- PORT=4005 - PORT=4005
- PYTHON_SIDECAR_URL=http://localhost:4006 - PYTHON_SIDECAR_URL=http://localhost:4006
depends_on:
cosmos-emulator:
condition: service_healthy
labels: labels:
- 'traefik.enable=true' - 'traefik.enable=true'
- 'traefik.http.routers.extraction.rule=PathPrefix(`/api/extract`) || PathPrefix(`/api/tasks`)' - 'traefik.http.routers.extraction.rule=PathPrefix(`/api/extract`) || PathPrefix(`/api/tasks`)'
- 'traefik.http.services.extraction.loadbalancer.server.port=4005' - 'traefik.http.services.extraction.loadbalancer.server.port=4005'
restart: unless-stopped restart: unless-stopped
healthcheck: healthcheck:
test: ['CMD', 'wget', '-q', '--spider', 'http://127.0.0.1:4005/health'] test:
[
'CMD',
'node',
'-e',
'fetch("http://127.0.0.1:4005/health").then(r => process.exit(r.ok ? 0 : 1)).catch(() => process.exit(1))',
]
interval: 30s interval: 30s
timeout: 10s timeout: 10s
retries: 3 retries: 3

View File

@ -7,17 +7,17 @@ This repo is currently set up to run as a single-host prototype with Docker Comp
- `platform-service` - `platform-service`
- `extraction-service` - `extraction-service`
- `mcp-server` - `mcp-server`
- `cosmos-emulator`
- `gateway` (Traefik) - `gateway` (Traefik)
- `loki` - `loki`
- `grafana` - `grafana`
## What Stays External ## What Stays External
- Azure Cosmos DB
- Azure Key Vault if you choose to use it - Azure Key Vault if you choose to use it
- Any real API credentials such as Stripe or Gemini - Any real API credentials such as Stripe or Gemini
For the prototype phase, keep secrets in `.env` and keep state in managed external services rather than adding more local containers. For this VM prototype, Cosmos is self-hosted through the Linux Cosmos DB Emulator container. Everything else should still stay in `.env` and move to a real secret manager later.
## First-Time Setup ## First-Time Setup
@ -27,8 +27,6 @@ cp .env.example .env
Fill in at least: Fill in at least:
- `COSMOS_ENDPOINT`
- `COSMOS_KEY`
- `JWT_SECRET` - `JWT_SECRET`
If you want extraction features that call Gemini, also set: If you want extraction features that call Gemini, also set:
@ -44,8 +42,8 @@ If you want extraction features that call Gemini, also set:
That script will: That script will:
1. Validate the required environment variables. 1. Validate the required environment variables.
2. Build the shared packages needed by the Docker images. 2. Start the local Cosmos DB emulator.
3. Build and start the Compose stack. 3. Build and start the rest of the Compose stack.
## Day-To-Day Commands ## Day-To-Day Commands
@ -54,11 +52,15 @@ docker compose ps
docker compose logs -f platform-service docker compose logs -f platform-service
docker compose logs -f extraction-service docker compose logs -f extraction-service
docker compose logs -f mcp-server docker compose logs -f mcp-server
docker compose logs -f cosmos-emulator
docker compose down docker compose down
``` ```
The Cosmos Data Explorer is exposed on `http://localhost:1234`.
## Notes ## Notes
- This is intended for early prototype use on a single machine. - This is intended for early prototype use on a single machine.
- Do not commit `.env`. - Do not commit `.env`.
- When the project moves to a more secure environment later, keep the same service boundaries and move secrets out of `.env` into a proper secret manager. - The Linux emulator is a preview and is only appropriate for local or prototype use.
- When the project moves to a more secure environment later, replace the emulator with a real Azure Cosmos DB account and move secrets out of `.env` into a proper secret manager.

View File

@ -92,6 +92,7 @@ async function createContainerSafe(
id: name, id: name,
partitionKey: { partitionKey: {
paths: [config.partitionKeyPath], paths: [config.partitionKeyPath],
kind: 'Hash',
} as PartitionKeyDefinition, } as PartitionKeyDefinition,
...(config.defaultTtl != null && { defaultTtl: config.defaultTtl }), ...(config.defaultTtl != null && { defaultTtl: config.defaultTtl }),
}; };

View File

@ -21,14 +21,9 @@
"build": "tsc", "build": "tsc",
"test": "vitest run" "test": "vitest run"
}, },
"peerDependencies": { "dependencies": {
"@azure/storage-blob": ">=12.0.0" "@azure/storage-blob": ">=12.0.0"
}, },
"peerDependenciesMeta": {
"@azure/storage-blob": {
"optional": true
}
},
"devDependencies": { "devDependencies": {
"vitest": "^3.0.0" "vitest": "^3.0.0"
} }

View File

@ -11,7 +11,6 @@ if [[ ! -f .env ]]; then
fi fi
"$REPO_ROOT/scripts/check-prototype-env.sh" "$REPO_ROOT/.env" "$REPO_ROOT/scripts/check-prototype-env.sh" "$REPO_ROOT/.env"
"$REPO_ROOT/scripts/docker-prep.sh"
echo "Starting prototype stack with Docker Compose..." echo "Starting prototype stack with Docker Compose..."
docker compose up -d --build docker compose up -d --build

View File

@ -13,11 +13,14 @@ COPY packages/blob/package.json packages/blob/
COPY packages/config/package.json packages/config/ COPY packages/config/package.json packages/config/
COPY packages/auth/package.json packages/auth/ COPY packages/auth/package.json packages/auth/
COPY packages/api-client/package.json packages/api-client/ COPY packages/api-client/package.json packages/api-client/
COPY packages/datastore/package.json packages/datastore/
COPY packages/events/package.json packages/events/
COPY packages/fastify-core/package.json packages/fastify-core/ COPY packages/fastify-core/package.json packages/fastify-core/
COPY packages/logger/package.json packages/logger/ COPY packages/logger/package.json packages/logger/
COPY packages/monitoring/package.json packages/monitoring/ COPY packages/monitoring/package.json packages/monitoring/
COPY packages/react-auth/package.json packages/react-auth/ COPY packages/react-auth/package.json packages/react-auth/
COPY packages/design-tokens/package.json packages/design-tokens/ COPY packages/design-tokens/package.json packages/design-tokens/
COPY packages/storage/package.json packages/storage/
COPY packages/testing/package.json packages/testing/ COPY packages/testing/package.json packages/testing/
COPY services/platform-service/package.json services/platform-service/ COPY services/platform-service/package.json services/platform-service/