bytelyst/learning_ai_common_plat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
554a5137ec
learning_ai_common_plat/__LOCAL_LLMs/docs
History
saravanakumardb1 554a5137ec docs(local-llm): improve dashboard review — add checkboxes, commit log, new findings 
Rewrite DASHBOARD_REVIEW.md with progress-tracking improvements:
- Add GitHub-style checkboxes to all 41 actionable items
- Add file inventory table with line counts and purposes
- Add commit log section for tracking implementation progress
- Add sprint tracker tables with effort estimates and commit columns
- New finding B11: prompt text not cleared on Escape close
- New finding CQ6: no Zod validation on API responses
- Consolidate priority matrix into sprint tables (less redundancy)
- Add deferred items section with dependency notes
- Improve item descriptions with more precise file:line references
- Add stack summary and total effort estimate (14–17 hrs)
2026-02-19 15:11:19 -08:00
..
01-hardware-and-prerequisites.md docs(local-llm): add docs index, hardware specs, and whisper-cpp setup 2026-02-19 13:00:48 -08:00
02-ollama-setup-and-models.md docs(local-llm): add Ollama setup, extraction evals, and env vars reference 2026-02-19 13:01:05 -08:00
03-whisper-cpp-setup.md docs(local-llm): add docs index, hardware specs, and whisper-cpp setup 2026-02-19 13:00:48 -08:00
04-multimodal-local-stack.md docs(local-llm): add multimodal stack, model recommendations, and troubleshooting 2026-02-19 13:01:22 -08:00
05-mission-control-dashboard.md docs(local-llm): add Mission Control dashboard documentation 2026-02-19 13:03:30 -08:00
06-extraction-service-evals.md docs(local-llm): add Ollama setup, extraction evals, and env vars reference 2026-02-19 13:01:05 -08:00
07-model-recommendations.md docs(local-llm): add multimodal stack, model recommendations, and troubleshooting 2026-02-19 13:01:22 -08:00
08-troubleshooting.md docs(local-llm): add multimodal stack, model recommendations, and troubleshooting 2026-02-19 13:01:22 -08:00
09-environment-variables.md docs(local-llm): add Ollama setup, extraction evals, and env vars reference 2026-02-19 13:01:05 -08:00
DASHBOARD_REVIEW.md docs(local-llm): improve dashboard review — add checkboxes, commit log, new findings 2026-02-19 15:11:19 -08:00
README.md docs(local-llm): add docs index, hardware specs, and whisper-cpp setup 2026-02-19 13:00:48 -08:00

README.md

Local LLM Stack — Documentation Index

Complete guide for the local AI inference stack on the ByteLyst development machine. Hardware: Apple M4 Pro · 48 GB LPDDR5 · macOS Tahoe Last updated: 2026-02-19

Quick Start

# 1. Start Ollama
ollama serve                    # or: brew services start ollama

# 2. Load a model
ollama run qwen2.5-coder:32b   # best coding model for this hardware

# 3. Launch Mission Control dashboard
cd __LOCAL_LLMs/dashboard && npm run dev -- -p 3100
# Open http://localhost:3100

Documentation

# Document Description
01 Hardware & Prerequisites Machine specs, installed toolchain, disk/RAM budget
02 Ollama Setup & Models Installation, server config, model management, memory behavior
03 Whisper.cpp Setup Speech-to-text: installation, models, CLI usage, real-time streaming
04 Multimodal Local Stack Vision models, audio pipeline, video understanding status
05 Mission Control Dashboard Next.js dashboard: architecture, API routes, features, running
06 Extraction Service Evals promptfoo eval suite, Ollama vs Gemini comparison, Python sidecar
07 Model Recommendations Tiered model guide by use case, size, and quality for M4 Pro 48GB
08 Troubleshooting & Corporate Proxy Common issues, Forcepoint proxy workarounds, MLX warnings
09 Environment Variables All config vars for Ollama, Whisper, dashboard, evals

Directory Structure

__LOCAL_LLMs/
├── README.md                        ← you are here (moved from LOCAL_LLMs_setup_mac_m4_48gb.md)
├── docs/
│   ├── README.md                    ← this index
│   ├── 01-hardware-and-prerequisites.md
│   ├── 02-ollama-setup-and-models.md
│   ├── 03-whisper-cpp-setup.md
│   ├── 04-multimodal-local-stack.md
│   ├── 05-mission-control-dashboard.md
│   ├── 06-extraction-service-evals.md
│   ├── 07-model-recommendations.md
│   ├── 08-troubleshooting.md
│   └── 09-environment-variables.md
├── dashboard/                       ← Next.js Mission Control app (port 3100)
│   ├── src/app/page.tsx             ← main dashboard UI
│   ├── src/app/api/ollama/route.ts  ← Ollama API proxy (list, load, unload, generate)
│   ├── src/app/api/whisper/route.ts ← Whisper binary/model discovery
│   └── src/app/api/system/route.ts  ← System info (chip, RAM, disk, brew)
└── LOCAL_LLMs_setup_mac_m4_48gb.md  ← original doc (preserved, see docs/ for latest)

Current Installation Status (2026-02-19)

Component Version Status Disk Usage
Ollama 0.16.2 ✅ Installed via brew —
qwen2.5-coder:32b — ✅ Downloaded 19 GB
llama3.1:8b — ✅ Downloaded 4.9 GB
whisper-cpp 1.8.3 ✅ Installed via brew 9.6 MB
whisper model (ggml-large-v3-turbo) — ❌ Blocked by corporate proxy —
ffmpeg 8.0.1 ✅ Installed via brew 53.3 MB
Mission Control Dashboard Next.js 16 ✅ Built, runs on :3100 —

Related Resources