bytelyst/learning_ai_common_plat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6d18344fe0
learning_ai_common_plat/__LOCAL_LLMs/windows_specific
History
saravanakumardb1 6d18344fe0 docs(local-llms): add 7 RTX 5090 capability deep-dive guides 
New capabilities/ subfolder with detailed guides:
- 01: GPU inference speed (benchmarks, Ollama tuning, API usage)
- 02: Whisper batch transcription (scripts, Python integration, use cases)
- 03: TTS generation at scale (Orpheus + Qwen3, batch scripts, voice cloning)
- 04: Fine-tuning / training (LoRA, QLoRA, data prep, Ollama export)
- 05: CUDA / TensorRT / ML research (toolchain setup, Triton kernels, profiling)
- 06: Stable Diffusion / image gen (ComfyUI, SDXL, FLUX, batch generation)
- 07: Multi-GPU workloads (scaling path, eGPU, cloud, cost planning)
- README: index with learning order and prerequisites

Each guide covers: what it is, how to use it, benefits, skills to learn
2026-02-21 20:36:21 -08:00
..
capabilities docs(local-llms): add 7 RTX 5090 capability deep-dive guides 2026-02-21 20:36:21 -08:00
mac-vs-windows-comparison.md docs(local-llms): add Mac vs Windows side-by-side comparison 2026-02-21 17:59:58 -08:00
razer-blade-18-spec.md ci: update CI/CD configuration 2026-02-21 14:13:07 -08:00
README.md feat(local-llms): add one-click Windows setup scripts 2026-02-21 16:28:02 -08:00
setup-guide.md feat(local-llms): add one-click Windows setup scripts 2026-02-21 16:28:02 -08:00
setup-windows.ps1 feat(local-llms): add one-click Windows setup scripts 2026-02-21 16:28:02 -08:00
setup-wsl.sh feat(local-llms): add one-click Windows setup scripts 2026-02-21 16:28:02 -08:00

README.md

Windows Setup — Local LLM Stack

Two scripts. Zero IDE required. Fresh machine to running dashboard in ~30 minutes.

Quick Start

Step 1: Windows Side (PowerShell as Admin)

# Allow script execution for this session
Set-ExecutionPolicy -Scope Process Bypass

# Run the Windows setup (installs Ollama, pulls models, installs WSL2)
.\setup-windows.ps1

What it does: Verifies NVIDIA drivers, installs Ollama, pulls 5 models (~52 GB), installs WSL2 Ubuntu 24.04.

After WSL2 install you may need to reboot. Ubuntu will ask for a username/password on first launch.

Step 2: WSL2 Side (Ubuntu terminal)

# One-liner — downloads and runs the WSL2 setup script
curl -fsSL https://raw.githubusercontent.com/saravanakumardb1/learning_ai_common_plat/main/__LOCAL_LLMs/windows_specific/setup-wsl.sh | bash

What it does: Installs Node.js, Python, ffmpeg, cmake → builds Whisper.cpp with CUDA → sets up TTS → starts the dashboard.

Step 3: Open Browser

http://localhost:3000

Dashboard should show all green. Done.

What Gets Installed

Component Where Size
NVIDIA drivers Windows pre-installed
Ollama Windows (native) ~200 MB
5 LLM models Windows (%USERPROFILE%\.ollama\) ~52 GB
WSL2 Ubuntu 24.04 Windows ~1 GB
Node.js 20 LTS WSL2 ~50 MB
Python 3.12 + venv WSL2 ~200 MB
whisper.cpp (CUDA) WSL2 /usr/local/bin/ ~50 MB
Whisper model WSL2 ~/whisper-models/ ~1.5 GB
SNAC decoder WSL2 (repo models/) ~76 MB
Qwen3-TTS 0.6B WSL2 (repo models/) ~1.7 GB
PyTorch CUDA WSL2 (.venv-qwen-tts/) ~2.5 GB
Dashboard deps WSL2 (dashboard/node_modules/) ~200 MB

Total: ~60 GB (mostly Ollama models)

Files in This Directory

File Purpose
README.md This file — quick start guide
setup-windows.ps1 PowerShell script — Windows-side setup
setup-wsl.sh Bash script — WSL2-side setup
setup-guide.md Detailed manual guide with troubleshooting
razer-blade-18-spec.md Full hardware specs for the Razer Blade 18

After Setup

# Daily usage — start everything
cd ~/code/mygh/learning_ai_common_plat/__LOCAL_LLMs
bash start-dashboard.sh

# Check status
bash start-dashboard.sh status

# Stop
bash start-dashboard.sh stop

# Test TTS
.venv-qwen-tts/bin/python test_orpheus_tts.py
.venv-qwen-tts/bin/python test_qwen_tts.py

Troubleshooting

See setup-guide.md for common issues:

  • Ollama not accessible from WSL2
  • CUDA not visible in WSL2
  • Slow filesystem performance