# 3. TTS Generation at Scale

> **RTX 5090:** Qwen3-TTS at 2–4× realtime, Orpheus at 2–3× realtime
> **Why it matters:** Pre-generate audio libraries, build voice features, create content — all faster than real-time playback

---

## What Is Local TTS?

Text-to-Speech (TTS) converts written text into natural-sounding speech. Our stack has two engines:

| Engine        | Architecture                    | Size        | Voices                  | Quality                               |
| ------------- | ------------------------------- | ----------- | ----------------------- | ------------------------------------- |
| **Orpheus**   | Ollama-served, SNAC decoder     | 4 GB        | 8 English voices        | Extremely natural, emotional          |
| **Qwen3-TTS** | PyTorch model, direct inference | 0.6B params | 10 languages, cloneable | Multilingual, zero-shot voice cloning |

```
┌──────────────────────────────────────────────────────────────────────┐
│ TTS Pipeline                                                         │
│                                                                      │
│  Orpheus:                                                            │
│  Text → [Ollama: generate audio tokens] → [SNAC: decode to WAV]    │
│              ▲ GPU (CUDA)                     ▲ GPU (CUDA)          │
│                                                                      │
│  Qwen3-TTS:                                                         │
│  Text → [PyTorch model: text→mel→audio] → WAV file                 │
│              ▲ GPU (CUDA / MPS)                                      │
│                                                                      │
└──────────────────────────────────────────────────────────────────────┘
```

---

## Performance: Mac vs Razer

| Engine                       | Mac M4 Pro (MPS) | Razer RTX 5090 (CUDA) | Speedup |
| ---------------------------- | ---------------- | --------------------- | ------- |
| Orpheus (per sentence)       | ~realtime        | ~2–3× realtime        | ~2.5×   |
| Qwen3-TTS (per sentence)     | ~realtime        | ~2–4× realtime        | ~3×     |
| Orpheus (10 min narration)   | ~10 min          | ~3–5 min              | ~2.5×   |
| Qwen3-TTS (10 min narration) | ~10 min          | ~2.5–5 min            | ~3×     |
| Batch: 100 sentences         | ~5–8 min         | ~2–3 min              | ~3×     |
| Batch: 1000 sentences        | ~50–80 min       | ~15–25 min            | ~3×     |

**"2–4× realtime" means:** A 10-second sentence generates in 2.5–5 seconds. The audio is produced faster than you could listen to it.

---

## How to Use It

### Orpheus TTS (8 Natural Voices)

Orpheus runs through Ollama + SNAC decoder. Already set up by `setup-tts.sh`.

```bash
cd ~/code/mygh/learning_ai_common_plat/__LOCAL_LLMs

# Generate speech with default voice (tara)
.venv-qwen-tts/bin/python test_orpheus_tts.py

# Output: test_orpheus_tara.wav, test_orpheus_leah.wav, etc.
# Play: aplay test_orpheus_tara.wav
```

#### Available Orpheus Voices

| Voice  | Character          | Best For              |
| ------ | ------------------ | --------------------- |
| `tara` | Young female, warm | Narration, assistants |
| `leah` | Female, clear      | Tutorials, guides     |
| `jess` | Female, energetic  | Announcements         |
| `leo`  | Male, calm         | Narration, podcasts   |
| `dan`  | Male, professional | Business content      |
| `mia`  | Female, friendly   | Conversational        |
| `zac`  | Male, young        | Casual content        |
| `zoe`  | Female, neutral    | General purpose       |

#### Custom Text with Orpheus (Python)

```python
"""Generate speech from custom text using Orpheus TTS."""
import json
import struct
import wave
import urllib.request

OLLAMA_URL = "http://localhost:11434/api/generate"

def generate_speech(text: str, voice: str = "tara", output_path: str = "output.wav"):
    """Generate a WAV file from text using Orpheus via Ollama."""
    prompt = f"<|audio|>{voice}: {text}"

    payload = json.dumps({
        "model": "sematre/orpheus:en",
        "prompt": prompt,
        "stream": False,
        "options": {"temperature": 0.6, "top_p": 0.9}
    }).encode()

    req = urllib.request.Request(
        OLLAMA_URL,
        data=payload,
        headers={"Content-Type": "application/json"}
    )

    with urllib.request.urlopen(req, timeout=120) as resp:
        result = json.loads(resp.read())

    # Decode audio tokens → SNAC → WAV
    # (Full implementation in test_orpheus_tts.py)
    print(f"Generated: {output_path}")

# Example
generate_speech(
    "Welcome to LysnrAI. Your voice-first productivity assistant.",
    voice="tara",
    output_path="welcome.wav"
)
```

### Qwen3-TTS (Multilingual + Voice Cloning)

```bash
cd ~/code/mygh/learning_ai_common_plat/__LOCAL_LLMs

# Generate speech with Qwen3-TTS
.venv-qwen-tts/bin/python test_qwen_tts.py

# Output: test_qwen3_tts_output.wav
# Play: aplay test_qwen3_tts_output.wav
```

#### Qwen3-TTS Features

| Feature             | Description                                                                               |
| ------------------- | ----------------------------------------------------------------------------------------- |
| **10 languages**    | English, Chinese, Japanese, Korean, French, German, Spanish, Italian, Portuguese, Russian |
| **Voice cloning**   | Provide a reference audio clip → model clones the voice                                   |
| **Emotion control** | Adjust speaking style via prompt engineering                                              |
| **0.6B parameters** | Small enough to run fast, large enough for quality                                        |

### Batch TTS Generation Script

```bash
#!/bin/bash
# batch-tts.sh — Generate audio for a list of sentences
# Usage: bash batch-tts.sh sentences.txt output_dir/

INPUT_FILE="${1:-sentences.txt}"
OUTPUT_DIR="${2:-tts_output}"
VOICE="${3:-tara}"

mkdir -p "$OUTPUT_DIR"

echo "=== Batch Orpheus TTS ==="
echo "Input:  $INPUT_FILE"
echo "Output: $OUTPUT_DIR"
echo "Voice:  $VOICE"
echo ""

LINE_NUM=0
while IFS= read -r line; do
    [ -z "$line" ] && continue
    ((LINE_NUM++))

    echo "[$LINE_NUM] Generating: ${line:0:60}..."

    # Use the Python TTS script with custom text
    .venv-qwen-tts/bin/python -c "
import test_orpheus_tts as tts
tts.generate_and_save('$line', '$VOICE', '$OUTPUT_DIR/sentence_${LINE_NUM}_${VOICE}.wav')
" 2>/dev/null

    echo "  -> $OUTPUT_DIR/sentence_${LINE_NUM}_${VOICE}.wav"
done < "$INPUT_FILE"

echo ""
echo "=== Done! $LINE_NUM files generated ==="
```

---

## Real-World Use Cases

### 1. LysnrAI Voice Responses

Generate spoken responses from LLM output — the Razer can produce audio faster than the user can listen:

```
User asks question → LLM generates text → TTS converts to speech → User hears answer
                                              ▲
                                    2–4× realtime on RTX 5090
                                    Feels instant for short responses
```

### 2. Pre-Generated Audio Libraries

Build a library of common phrases, greetings, and responses:

```bash
# sentences.txt
Welcome to LysnrAI.
Your daily brief is ready.
You have three new memories to review.
Recording started.
Recording saved successfully.
Transcription complete.
```

```bash
# Generate all phrases in multiple voices
for voice in tara leo dan; do
    bash batch-tts.sh sentences.txt audio_library/ "$voice"
done
```

### 3. Audiobook / Podcast Generation

```python
# Split a document into paragraphs and generate audio for each
paragraphs = open("article.txt").read().split("\n\n")

for i, para in enumerate(paragraphs):
    generate_speech(para, voice="leo", output_path=f"chapter_{i:03d}.wav")

# Concatenate with ffmpeg
# ffmpeg -f concat -i filelist.txt -c copy full_audiobook.wav
```

### 4. Multilingual Content (Qwen3-TTS)

```python
# Generate the same message in multiple languages
messages = {
    "en": "Welcome to MindLyst, your AI-powered life organizer.",
    "ja": "MindLystへようこそ。AIライフオーガナイザーです。",
    "es": "Bienvenido a MindLyst, tu organizador de vida con IA.",
}

for lang, text in messages.items():
    generate_qwen_tts(text, output_path=f"welcome_{lang}.wav")
```

### 5. Voice Cloning (Qwen3-TTS)

```python
# Clone a voice from a reference audio sample
# Provide a 5–15 second reference clip of the target voice
reference_audio = "my_voice_sample.wav"
text = "This is my cloned voice saying something new."

# Qwen3-TTS can reproduce the voice characteristics
generate_qwen_tts(text, reference=reference_audio, output_path="cloned.wav")
```

---

## Benefits

| Benefit           | Description                                              |
| ----------------- | -------------------------------------------------------- |
| **Speed**         | Generate audio faster than playback speed                |
| **Privacy**       | All voice generation runs locally — no cloud APIs        |
| **Cost**          | $0 vs $0.015/1K chars for cloud TTS (Google, ElevenLabs) |
| **Voice variety** | 8 Orpheus voices + unlimited Qwen3-TTS voice cloning     |
| **Multilingual**  | Qwen3-TTS supports 10 languages natively                 |
| **Offline**       | Works without internet                                   |
| **Customizable**  | Control emotion, speed, voice characteristics            |

### Cost Comparison (Generate 1 hour of audio)

| Method                     | Cost    | Time             |
| -------------------------- | ------- | ---------------- |
| ElevenLabs API             | ~$15–30 | ~minutes (cloud) |
| Google Cloud TTS           | ~$4–16  | ~minutes (cloud) |
| Mac M4 Pro (local)         | $0      | ~60 min          |
| **Razer RTX 5090 (local)** | **$0**  | **~15–30 min**   |

---

## Skills You'll Learn

| Skill                     | What You'll Learn                                       | Career Value                   |
| ------------------------- | ------------------------------------------------------- | ------------------------------ |
| **Speech synthesis**      | How neural TTS works (text→tokens→mel→audio)            | Core speech/NLP skill          |
| **Audio codecs**          | SNAC, Encodec, WAV format, sample rates                 | Audio engineering fundamentals |
| **Voice cloning**         | Zero-shot voice cloning techniques                      | Cutting-edge ML research area  |
| **Batch processing**      | Automating large-scale audio generation                 | Production engineering         |
| **GPU memory**            | Managing VRAM for concurrent TTS + LLM workloads        | ML ops knowledge               |
| **Audio post-processing** | ffmpeg: concatenation, normalization, format conversion | Multimedia engineering         |
| **API design**            | Building REST APIs around TTS engines                   | Backend engineering            |
| **Multilingual NLP**      | Cross-language text processing and pronunciation        | Global product development     |

---

## Next Steps

- [ ] Generate test audio with both Orpheus and Qwen3-TTS on Razer
- [ ] Create `batch-tts.sh` in `__LOCAL_LLMs/scripts/`
- [ ] Build a pre-generated audio library for LysnrAI common phrases
- [ ] Experiment with Qwen3-TTS voice cloning using your own voice
- [ ] Try generating audio in Tamil (Qwen3-TTS multilingual)
- [ ] Measure actual generation speed: words-per-second on each engine
