# 5. CUDA / TensorRT / ML Research

> **RTX 5090:** Full NVIDIA toolchain — CUDA 13.x, cuDNN, TensorRT, Triton
> **Why it matters:** Most ML papers, frameworks, and production systems are CUDA-first. This is the industry-standard GPU compute platform.

---

## What Is the NVIDIA ML Toolchain?

NVIDIA provides a layered stack of tools that turn your GPU into a general-purpose compute engine:

```
┌──────────────────────────────────────────────────────────────────────┐
│ NVIDIA ML Toolchain (RTX 5090)                                       │
│                                                                      │
│  ┌─────────────────────────────────────────────────────────────┐    │
│  │ Your Code (Python / C++ / TypeScript)                        │    │
│  └────────────────────────┬────────────────────────────────────┘    │
│                           ▼                                          │
│  ┌─────────────────────────────────────────────────────────────┐    │
│  │ ML Frameworks                                                │    │
│  │ PyTorch · TensorFlow · JAX · ONNX Runtime                  │    │
│  └────────────────────────┬────────────────────────────────────┘    │
│                           ▼                                          │
│  ┌─────────────────────────────────────────────────────────────┐    │
│  │ NVIDIA Libraries                                             │    │
│  │ TensorRT · Triton · cuDNN · cuBLAS · NCCL                  │    │
│  └────────────────────────┬────────────────────────────────────┘    │
│                           ▼                                          │
│  ┌─────────────────────────────────────────────────────────────┐    │
│  │ CUDA Runtime + Driver                                        │    │
│  │ CUDA 13.x · GPU scheduling · memory management             │    │
│  └────────────────────────┬────────────────────────────────────┘    │
│                           ▼                                          │
│  ┌─────────────────────────────────────────────────────────────┐    │
│  │ Hardware                                                     │    │
│  │ RTX 5090: ~18K CUDA cores · 24 GB GDDR7 · Tensor cores     │    │
│  └─────────────────────────────────────────────────────────────┘    │
│                                                                      │
└──────────────────────────────────────────────────────────────────────┘
```

### Component Breakdown

| Component           | What It Does                                                | Why You Need It                  |
| ------------------- | ----------------------------------------------------------- | -------------------------------- |
| **CUDA**            | General-purpose GPU programming                             | Foundation for everything        |
| **cuDNN**           | Optimized neural network primitives (conv, attention, etc.) | 2–5× faster training/inference   |
| **TensorRT**        | Model optimization + inference engine                       | 2–4× faster deployment inference |
| **Triton** (NVIDIA) | Inference server for serving models at scale                | Production model serving         |
| **Triton** (OpenAI) | GPU kernel compiler (write custom GPU kernels in Python)    | Research + custom ops            |
| **cuBLAS**          | Optimized matrix multiplication                             | Core of all neural network math  |
| **NCCL**            | Multi-GPU communication                                     | Distributed training (future)    |

---

## How to Set Up

### 1. CUDA Toolkit (Inside WSL2)

```bash
# Check if CUDA is already available (from Windows driver passthrough)
nvidia-smi
nvcc --version  # CUDA compiler

# If nvcc is not found, install CUDA toolkit
# (nvidia-smi works from driver passthrough, but nvcc needs the toolkit)
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update
sudo apt install -y cuda-toolkit-12-4

# Add to PATH
echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc

# Verify
nvcc --version
# Should show: CUDA 12.4+
```

### 2. cuDNN

```bash
# cuDNN is usually bundled with PyTorch, but for custom builds:
sudo apt install -y libcudnn8 libcudnn8-dev

# Verify in Python
python3 -c "import torch; print(f'cuDNN: {torch.backends.cudnn.version()}')"
```

### 3. TensorRT

```bash
# Install TensorRT
pip install tensorrt

# Or via apt for system-wide
sudo apt install -y tensorrt

# Verify
python3 -c "import tensorrt; print(f'TensorRT: {tensorrt.__version__}')"
```

### 4. PyTorch with Full CUDA Support

```bash
# Create a research environment
python3.12 -m venv ~/.venv-ml-research
source ~/.venv-ml-research/bin/activate

# Install PyTorch with CUDA 12.4
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

# Verify
python3 -c "
import torch
print(f'PyTorch: {torch.__version__}')
print(f'CUDA available: {torch.cuda.is_available()}')
print(f'GPU: {torch.cuda.get_device_name(0)}')
print(f'VRAM: {torch.cuda.get_device_properties(0).total_mem / 1e9:.1f} GB')
print(f'cuDNN: {torch.backends.cudnn.version()}')
"
```

---

## How to Use It

### CUDA Programming Basics (Python)

```python
"""Your first CUDA program — matrix multiplication on the GPU."""
import torch
import time

device = torch.device("cuda")

# Create two large matrices on the GPU
A = torch.randn(4096, 4096, device=device)
B = torch.randn(4096, 4096, device=device)

# Warm up
_ = torch.mm(A, B)
torch.cuda.synchronize()

# Benchmark
start = time.perf_counter()
for _ in range(100):
    C = torch.mm(A, B)
torch.cuda.synchronize()
elapsed = time.perf_counter() - start

tflops = (2 * 4096**3 * 100) / elapsed / 1e12
print(f"Matrix multiply: {elapsed:.3f}s for 100 iterations")
print(f"Throughput: {tflops:.1f} TFLOPS")
# Expected on RTX 5090: ~100–200 TFLOPS (FP32) or ~200–400 TFLOPS (FP16)
```

### TensorRT: Optimize a Model for Faster Inference

```python
"""Convert a PyTorch model to TensorRT for 2–4× faster inference."""
import torch
import torch_tensorrt

# Load a model
model = torch.hub.load('pytorch/vision', 'resnet50', pretrained=True).eval().cuda()

# Compile with TensorRT
trt_model = torch_tensorrt.compile(
    model,
    inputs=[torch_tensorrt.Input(shape=[1, 3, 224, 224], dtype=torch.float16)],
    enabled_precisions={torch.float16},
)

# Benchmark
input_tensor = torch.randn(1, 3, 224, 224, device="cuda", dtype=torch.float16)

# PyTorch baseline
with torch.no_grad():
    start = time.perf_counter()
    for _ in range(1000):
        _ = model(input_tensor.float())
    torch.cuda.synchronize()
    pytorch_time = time.perf_counter() - start

# TensorRT optimized
with torch.no_grad():
    start = time.perf_counter()
    for _ in range(1000):
        _ = trt_model(input_tensor)
    torch.cuda.synchronize()
    trt_time = time.perf_counter() - start

print(f"PyTorch: {pytorch_time:.3f}s")
print(f"TensorRT: {trt_time:.3f}s")
print(f"Speedup: {pytorch_time/trt_time:.1f}×")
```

### Reproducing ML Research Papers

Most ML papers provide CUDA-only code. The RTX 5090 lets you run them directly:

```bash
# Example: Run a recent paper's code
git clone https://github.com/some-researcher/cool-new-model.git
cd cool-new-model

# Typical requirements
pip install -r requirements.txt

# Run training/evaluation
python train.py --device cuda --epochs 10 --batch-size 16

# This would NOT work on Mac (CUDA-only dependencies)
```

### Custom CUDA Kernels with Triton (OpenAI)

```python
"""Write a custom GPU kernel in Python using Triton."""
import triton
import triton.language as tl
import torch

@triton.jit
def add_kernel(x_ptr, y_ptr, output_ptr, n_elements, BLOCK_SIZE: tl.constexpr):
    """Simple vector addition kernel running on GPU."""
    pid = tl.program_id(axis=0)
    block_start = pid * BLOCK_SIZE
    offsets = block_start + tl.arange(0, BLOCK_SIZE)
    mask = offsets < n_elements

    x = tl.load(x_ptr + offsets, mask=mask)
    y = tl.load(y_ptr + offsets, mask=mask)
    output = x + y
    tl.store(output_ptr + offsets, output, mask=mask)

# Run the kernel
n = 1_000_000
x = torch.randn(n, device="cuda")
y = torch.randn(n, device="cuda")
output = torch.empty_like(x)

grid = lambda meta: (triton.cdiv(n, meta["BLOCK_SIZE"]),)
add_kernel[grid](x, y, output, n, BLOCK_SIZE=1024)

print(f"Result correct: {torch.allclose(output, x + y)}")
```

---

## Real-World Use Cases for Your Projects

### 1. LysnrAI Inference Optimization

Convert your Whisper or TTS models to TensorRT for even faster inference:

```python
# Optimize Whisper encoder with TensorRT
# This can give another 2× speedup on top of whisper.cpp CUDA
```

### 2. Custom Embedding Models

Train domain-specific embedding models for LysnrAI's semantic search:

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2", device="cuda")

# Encode your documents
documents = ["meeting notes about Q1 budget", "grocery list for weekend", ...]
embeddings = model.encode(documents, batch_size=64, show_progress_bar=True)
# ~1000 documents/second on RTX 5090
```

### 3. Reproduce and Experiment with New Models

When a new paper drops (GPT-5 alternatives, new TTS models, new architectures):

```bash
# Clone → install → run — no CUDA compatibility issues
git clone https://github.com/new-cool-model
pip install -r requirements.txt
python evaluate.py --device cuda
```

### 4. Benchmarking and Profiling

```python
# Profile GPU usage during inference
with torch.profiler.profile(
    activities=[torch.profiler.ProfilerActivity.CPU, torch.profiler.ProfilerActivity.CUDA],
    with_stack=True,
) as prof:
    model(input_tensor)

print(prof.key_averages().table(sort_by="cuda_time_total"))
```

---

## Benefits

| Benefit                   | Description                                               |
| ------------------------- | --------------------------------------------------------- |
| **Industry standard**     | 95%+ of ML research and production uses CUDA              |
| **Framework support**     | PyTorch, TensorFlow, JAX — all CUDA-first                 |
| **Paper reproduction**    | Run any ML paper's code without compatibility issues      |
| **TensorRT optimization** | 2–4× faster inference on optimized models                 |
| **Custom kernels**        | Write GPU code in Python (Triton) or C++ (CUDA)           |
| **Profiling tools**       | nvidia-smi, Nsight, PyTorch profiler — rich debugging     |
| **Production parity**     | Code runs identically on cloud GPU instances (A100, H100) |

---

## Skills You'll Learn

| Skill                     | What You'll Learn                                     | Career Value                       |
| ------------------------- | ----------------------------------------------------- | ---------------------------------- |
| **CUDA fundamentals**     | GPU memory model, kernel launches, synchronization    | Core ML infrastructure skill       |
| **TensorRT**              | Model optimization, quantization, graph fusion        | Production ML deployment           |
| **Triton kernels**        | Custom GPU programming in Python                      | Research + performance engineering |
| **PyTorch profiling**     | Identifying bottlenecks, optimizing training loops    | Essential ML engineering           |
| **cuDNN**                 | Optimized neural network operations                   | Framework-level understanding      |
| **Mixed precision**       | FP16/BF16 training, loss scaling, numerical stability | Modern training standard           |
| **GPU memory management** | Memory pools, caching allocator, OOM debugging        | Practical ML engineering           |
| **Model optimization**    | Graph optimization, operator fusion, quantization     | ML deployment pipeline             |
| **Benchmark design**      | Fair comparison methodology, statistical significance | Research methodology               |

### Career Impact

CUDA proficiency is one of the most valuable ML engineering skills. Here's where it maps:

| Role                  | How CUDA Skills Apply                                 |
| --------------------- | ----------------------------------------------------- |
| **ML Engineer**       | Optimize training pipelines, reduce inference latency |
| **AI Infrastructure** | Build and maintain GPU clusters, optimize throughput  |
| **Research Engineer** | Implement custom operations for novel architectures   |
| **MLOps**             | TensorRT deployment, GPU monitoring, autoscaling      |
| **Full-Stack AI**     | End-to-end: train → optimize → serve → monitor        |

---

## Monitoring and Debugging

```bash
# Real-time GPU monitoring
watch -n 0.5 nvidia-smi

# Detailed GPU info
nvidia-smi -q

# GPU process list
nvidia-smi pmon

# Install nvtop (interactive GPU monitor)
sudo apt install nvtop
nvtop

# PyTorch memory debugging
python3 -c "
import torch
torch.cuda.memory_summary(device=0, abbreviated=True)
"
```

---

## Next Steps

- [ ] Install CUDA toolkit + cuDNN in WSL2
- [ ] Verify PyTorch CUDA with a matrix multiply benchmark
- [ ] Run a TensorRT optimization on a simple model
- [ ] Write a Triton kernel (vector add → custom attention)
- [ ] Profile an Ollama inference request with nvidia-smi
- [ ] Try reproducing a recent ML paper from GitHub
- [ ] Benchmark: PyTorch vs TensorRT inference speed for Whisper
