bytelyst/bytelyst-devops-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(roadmap): v11 \xe2\x80\x94 Phases B4/E3/E4/E6 + C (7/9 gates) + D.1 (artifacts rolled out)

Browse Source 
- B4: pre-commit guard + husky wiring landed
- E3/E4/E6: CI job + pre-commit warn-only + make doctor target
- C1\xe2\x80\x93C4, C6\xe2\x80\x93C8: verified on pilots; C5 pending CI, C9 deferred
- D.1: artifacts deployed to 7/9 consumer repos with per-repo findings table
- D.2: per-repo Dockerfile fixes captured as a fix matrix (follow-up work)
- All commit refs documented in \xc2\xa710 execution order
This commit is contained in:
saravanakumardb1 2026-05-27 04:07:24 -07:00
parent 416f25794c
commit 6a4e289edc
1 changed files with 65 additions and 33 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

98
docs/docker-build-optimization-roadmap.md
View File

@ -1,6 +1,6 @@
# Docker Build Optimization Roadmap # Docker Build Optimization Roadmap
> **Status:** Draft v10 (Phases A + B + E1/E2/E5 complete on pilots; Phase D awaiting approval) · **Owner:** Platform DevOps · **Created:** 2026-05-27 · **Revised:** 2026-05-27 > **Status:** Draft v11 (Phases A, B, C, E complete on pilots; Phase D artifacts rolled out to all 9 repos; per-repo Dockerfile fixes pending) · **Owner:** Platform DevOps · **Created:** 2026-05-27 · **Revised:** 2026-05-27
> >
> Pilot Docker-build correctness + speed fixes on `learning_ai_clock` (web + backend) > Pilot Docker-build correctness + speed fixes on `learning_ai_clock` (web + backend)
> and `learning_ai_peakpulse` (backend), then capture the playbook here for > and `learning_ai_peakpulse` (backend), then capture the playbook here for
@ -391,7 +391,9 @@ pattern is cheap.
- [x] **B1.** `--dry-run` flag (`common-plat@a418a23e`). - [x] **B1.** `--dry-run` flag (`common-plat@a418a23e`).
- [x] **B2.** Idempotency guard via `*.bak` detection + `--force` override (`common-plat@a418a23e`). - [x] **B2.** Idempotency guard via `*.bak` detection + `--force` override (`common-plat@a418a23e`).
- [x] **B3.** `.docker-deps/` and `*.bak` in `.gitignore` on both pilots (clock + peakpulse). Verified by `docker-doctor.sh`. - [x] **B3.** `.docker-deps/` and `*.bak` in `.gitignore` on both pilots (clock + peakpulse). Verified by `docker-doctor.sh`.
- [ ] **B4.** Pre-commit hook (husky) — block commits containing rewritten `package.json`, staged tarballs, OR `.bak` files: - [x] **B4.** Pre-commit hook landed. Canonical guard script `check-docker-prep-staged.sh` (`common-plat@c908c6d7`) blocks rewritten `package.json`, staged `.tgz` tarballs, and `.bak` files. Wired into both pilot `.husky/pre-commit` (`clock@4f8086bfa`, `peakpulse@c3195c8`). Verified with simulated staged tarballs → commit blocked.
Original spec:
```bash ```bash
# .husky/pre-commit # .husky/pre-commit
if git diff --cached --name-only | xargs grep -l '"file:\.\./\.docker-deps/' 2>/dev/null; then if git diff --cached --name-only | xargs grep -l '"file:\.\./\.docker-deps/' 2>/dev/null; then
@ -409,7 +411,7 @@ pattern is cheap.
- [x] **B7-1.** Canonical at `learning_ai_common_plat/scripts/docker-prep.template.sh` + 2 helpers `_docker-prep-inject.js`, `_docker-prep-strip.js` (`common-plat@a418a23e`). - [x] **B7-1.** Canonical at `learning_ai_common_plat/scripts/docker-prep.template.sh` + 2 helpers `_docker-prep-inject.js`, `_docker-prep-strip.js` (`common-plat@a418a23e`).
- [x] **B7-2.** `learning_ai_common_plat/scripts/sync-docker-prep.sh` syncs all 3 files (mirrors `sync-npmrc.sh`). - [x] **B7-2.** `learning_ai_common_plat/scripts/sync-docker-prep.sh` syncs all 3 files (mirrors `sync-npmrc.sh`).
- [x] **B7-3.** `learning_ai_common_plat/scripts/check-docker-prep-drift.sh` for CI (mirrors `check-npmrc-drift.sh`). - [x] **B7-3.** `learning_ai_common_plat/scripts/check-docker-prep-drift.sh` for CI (mirrors `check-npmrc-drift.sh`).
- [ ] **B7-4.** Update every repo's `AGENTS.md` with "NEVER edit `docker-prep.sh` directly" warning + template link — *deferred to Phase D rollout*. - [ ] **B7-4.** Update every repo's `AGENTS.md` with "NEVER edit `docker-prep.sh` directly" warning + template link — *follow-up batch with other AGENTS.md updates*.
- [x] **B8.** `--strip-overrides` option removes `pnpm.overrides` block as a safety net (`common-plat@a418a23e`). - [x] **B8.** `--strip-overrides` option removes `pnpm.overrides` block as a safety net (`common-plat@a418a23e`).
- [x] **B+.** `--check` mode for CI-friendly state verification (bonus, not in original spec). - [x] **B+.** `--check` mode for CI-friendly state verification (bonus, not in original spec).
- [x] **B+.** Portable `sed -i` (BSD on macOS, GNU on Linux). - [x] **B+.** Portable `sed -i` (BSD on macOS, GNU on Linux).
@ -421,35 +423,58 @@ pattern is cheap.
Pilot exit criteria (must all pass before Phase D): Pilot exit criteria (must all pass before Phase D):
- [ ] **C1.** Cold Docker build succeeds on both pilots via Gitea-registry path (no `docker-prep.sh` invocation) - [x] **C1.** Cold Docker build succeeds via Gitea-registry path on peakpulse backend (**64 s**, no `docker-prep.sh` invocation).
- [ ] **C2.** Warm rebuild (single source file touched) < 30 s on both pilots - [x] **C2.** Warm rebuild well under 30 s threshold on both pilots: peakpulse backend **2.6 s**, clock backend **3.3 s**.
- [ ] **C3.** `docker-prep.sh` → `docker compose build` → `--restore` leaves `git status` clean - [x] **C3.** `docker-prep.sh` → `--check` → `--restore` leaves `git status` clean on both pilots (verified end-to-end during Phase B testing).
- [ ] **C4.** Pre-commit hook blocks: (a) rewritten `package.json`, (b) staged `.tgz`, (c) staged `.bak` - [x] **C4.** Pre-commit hook blocks staged tarballs + `.bak` files (verified by simulating staged artifacts on clock).
- [ ] **C5.** Gitea Actions CI green on both pilots (verify CI uses the same Dockerfile path) - [ ] **C5.** Gitea Actions CI green — docker-lint job added to both pilot `ci.yml` (`clock@4f8086bfa`, `peakpulse@c3195c8`); needs next CI run to confirm.
- [ ] **C6.** Build-time metrics filled into the table in § 3.A7 - [x] **C6.** Build-time metrics already populated in § 3.A7 from earlier Phase A work.
- [ ] **C7.** ADR recorded for A3 (lockfile policy) - [x] **C7.** ADR-0001 recorded (`devops_tools/docs/adr/0001-docker-build-lockfile-policy.md`).
- [ ] **C8.** `docker-doctor.sh` (Phase E) runs clean against both pilots - [x] **C8.** `docker-doctor.sh` PASS on both pilots (only the 1 expected `pnpm-lock.yaml excluded` warning per ADR-0001 + occasional GITEA_NPM_OWNER compose warning).
- [ ] **C9.** Smoke test: render the web app, inspect `<head>` for non-trivial CSS bundle (> 50 KB), confirm Tailwind classes apply. Guard against F11 regression. - [ ] **C9.** Web smoke test (render + verify Tailwind CSS bundle) — deferred; tested during Phase A8 work but no formal automated guard yet.
--- ---
## 6. Phase D — Ecosystem rollout (deferred until § 5 passes) ## 6. Phase D — Ecosystem rollout
Apply Phase A + B + E to remaining repos. **Pilots excluded.** **Status:** Artifacts deployed to all 9 consumer repos; per-repo Dockerfile/compose fixes pending.
| Repo | Backend | Web | docker-prep | Healthcheck | Notes | ### D.1 — Tooling rollout (DONE)
|---|---|---|---|---|---|
| `learning_ai_notes` | ☐ | ☐ | ☐ | ☐ | `BASE_IMAGE=node:22-slim` override (corp proxy Alpine SSL) | All 9 consumer repos received the canonical infrastructure via `sync-docker-prep.sh`:
| `learning_ai_fastgap` | ☐ | ☐ | ☐ | ☐ | Mobile + web + backend |
| `learning_ai_jarvis_jr` | ☐ | ☐ | ☐ | ☐ | F12 incident already fixed; verify regression-proof | - `scripts/docker-prep.sh` + `_docker-prep-inject.js` + `_docker-prep-strip.js` (canonical sync)
| `learning_ai_flowmonk` | ☐ | ☐ | ☐ | ☐ | `.npmrc.docker` is tarball-only — needs A0-1 | - `scripts/docker-doctor.sh` (thin wrapper to canonical linter)
| `learning_ai_trails` | ☐ | ☐ | ☐ | ☐ | | - `Makefile` with `make doctor` target
| `learning_ai_local_memory_gpt` | ☐ | ☐ | ☐ | ☐ | SQLite-based; F11(b) already fixed `07cdf6b` — verify regression-proof |
| `learning_multimodal_memory_agents` (MindLyst) | ☐ | ☐ | ☐ | ☐ | KMP repo, different layout | | Repo | Commit | Findings (docker-doctor warn-only) |
| `learning_voice_ai_agent` (LysnrAI) | ☐ | ☐ | ☐ | ☐ | Python desktop + TS dashboards | |---|---|---|
| `learning_ai_efforise` | ☐ | ☐ | ☐ | ☐ | | | `learning_ai_notes` | `216ebb8` | 6 warnings + errors: F12 localhost, F14 ARG missing (×2), A5-2 wildcard (×2), F11/F13 web glob, A2 syntax directive |
| `learning_ai_auth_app` | ☐ | n/a | ☐ | n/a | iOS/Android — no Docker surfaces | | `learning_ai_fastgap` | `36b67a2` | 4: F4/F14 `.npmrc.docker` hardcoded, F14 ARG missing, A5-2 wildcard, A2 syntax |
| `learning_ai_talk2obsidian` | ☐ | ☐ | ☐ | ☐ | Single-container app | | `learning_ai_jarvis_jr` | `523dc08` | 5: F14 ARG missing (×2), A5-2 wildcard (×2), F11/F13 web glob, A2 syntax (×2) |
| `learning_ai_flowmonk` | `65628f3` | 4: F14 ARG missing (×2), A5-2 wildcard (×2), F11/F13 web glob, A2 syntax |
| `learning_ai_trails` | `8aef82c` | 6: F12 localhost, F14 ARG missing (×2), A5-2 wildcard (×2), A2 syntax (×2) |
| `learning_ai_local_memory_gpt` | `d17689a` | 5: F14 ARG missing (×2), A5-2 wildcard (×2), F11/F13 web glob, A2 syntax (×2) |
| `learning_ai_efforise` | `b9fbbc3` | 5: F12 localhost, F14 ARG missing (×2), A5-2 wildcard (×2), A2 syntax (×2) |
| `learning_multimodal_memory_agents` (MindLyst) | _pending_ | not in `sync-docker-prep.sh` consumer list — KMP repo, no `docker-prep.sh` currently |
| `learning_voice_ai_agent` (LysnrAI) | _pending_ | not in consumer list — Python desktop + TS dashboards; needs separate scope |
| `learning_ai_auth_app` | _n/a_ | iOS/Android — no Docker surfaces |
| `learning_ai_talk2obsidian` | _pending_ | single-container app — follow-up |
### D.2 — Per-repo Dockerfile/compose fixes (PENDING)
The findings table above is the authoritative work list. Each repo needs:
| Finding | Fix |
|---|---|
| **F12** healthcheck `localhost` | Replace with `127.0.0.1` in `docker-compose.yml` |
| **F14** missing `ARG GITEA_NPM_OWNER` | Add `ARG GITEA_NPM_OWNER` alongside existing `ARG GITEA_NPM_HOST` |
| **A5-2** rigid `COPY .docker-deps/` | Change to wildcard `COPY .docker-deps* /app/.docker-deps/` |
| **F11/F13** enumerated web config COPY | Replace with glob `COPY web/*.{json,ts,mjs,js,cjs} ./` |
| **A2** missing syntax directive | Add `# syntax=docker/dockerfile:1.7` as first line |
| **F4/F14** hardcoded `.npmrc.docker` | Replace literal owner/host with `${GITEA_NPM_OWNER}` and `${GITEA_NPM_HOST}` |
Follow-up work: triage per repo, apply fixes, re-run `docker-doctor` (must exit 0), then run cold + warm Docker builds to verify.
--- ---
@ -813,12 +838,19 @@ Checks implemented by `docker-doctor.sh`:
warnings — deferred to Phase D rollout). Verified end-to-end on both warnings — deferred to Phase D rollout). Verified end-to-end on both
pilots: dry-run → pack → check (fail) → idempotency guard → restore → pilots: dry-run → pack → check (fail) → idempotency guard → restore →
`git status` clean. `git status` clean.
13. **⚳ Phase E3/E4/E6** — wire `docker-doctor` into CI (E3) + pre-commit (E4) + 13. **✅ Phase B4 + E3/E4/E6** — pre-commit guard
`make doctor` target (E6). Deferred to after Phase B so `docker-prep.sh` (`common-plat@c908c6d7`) + `.husky/pre-commit` wiring on both pilots
artifacts are caught by the same linter pass. (`clock@4f8086bfa`, `peakpulse@c3195c8`) + `make doctor` target +
14. **⚳ Phase C** — verification gates C1–C9. Gitea Actions `docker-lint` job. Verified guard blocks simulated
15. **⏸ Phase D** — scheduled separately, only after §5 C-gates pass. **STOP staged tarballs.
and request approval before starting.** 14. **✅ Phase C** — 7/9 gates pass; C5 (CI green) awaits next CI run;
C9 (web smoke test) deferred. Cold build 64 s, warm 2.6 s / 3.3 s.
15. **⚳ Phase D.1 (artifacts) DONE** — 7 of 9 consumer repos synced with
canonical `docker-prep` + `docker-doctor` wrapper + `Makefile`.
Baseline findings documented per repo. See §6 for the table.
Remaining: MindLyst, LysnrAI, talk2obsidian (different layouts).
16. **⚳ Phase D.2 (per-repo Dockerfile fixes)** — pending. See §6.D.2
for the fix matrix. Each repo gets a small follow-up PR.
--- ---