Compare commits

...
Author SHA1 Message Date
James Pine 273483ffcf fix TorchScript error in frozen builds and update docs for TADA
Remove @torch.jit.script from the DAC shim's snake() function —
TorchScript calls inspect.getsource() which fails in PyInstaller
binaries (no .py source files).

Update all user-facing docs: 4 → 5 TTS engines, add TADA row to
every engine comparison table, mark TADA as Shipped in the upcoming
engines list, update architecture diagrams and tech stack tables.
2026-03-17 03:28:58 -07:00
James Pine 5774a168a9 fix TADA 3B model name: tada-3b -> tada-3b-ml 2026-03-17 03:17:53 -07:00
James Pine 6bf40bd2d0 fix tokenizer patch corrupting AutoTokenizer for other engines
Replace the monkey-patch on AutoTokenizer.from_pretrained (which broke
the classmethod descriptor and caused 'Tokenizer not loaded' errors
when loading Qwen after TADA) with two targeted config patches:
- Set AlignerConfig.tokenizer_name to the local ungated tokenizer path
- Pre-load TadaConfig, inject tokenizer_name, pass config= to from_pretrained

No global state is modified; other engines are unaffected.
2026-03-17 03:15:57 -07:00
James Pine 12cda2e090 fix torchcodec error by using soundfile instead of torchaudio.load
torchaudio 2.10+ switched its default audio loading backend to
torchcodec, which isn't installed. Replace torchaudio.load() with
soundfile.read() in create_voice_prompt(). TADA's internal use of
torchaudio.functional.resample() is unaffected (pure PyTorch math,
no torchcodec dependency).
2026-03-17 02:25:05 -07:00
James Pine 7a90290a76 fix gated Llama tokenizer error by redirecting to ungated mirror
TADA hardcodes 'meta-llama/Llama-3.2-1B' as its tokenizer source in
both the Aligner and TadaForCausalLM.from_pretrained(). That repo is
gated and requires accepting Meta's license on HuggingFace.

Monkey-patch AutoTokenizer.from_pretrained during model loading to
redirect Llama tokenizer requests to 'unsloth/Llama-3.2-1B', an
ungated mirror with identical tokenizer files. The patch is scoped
to model loading only and restored immediately after.
2026-03-17 02:22:26 -07:00
James Pine b02ce8e2f3 replace descript-audio-codec with lightweight DAC shim
The real descript-audio-codec package pulls in descript-audiotools,
which transitively requires onnx, tensorboard, protobuf, matplotlib,
pystoi, and other heavy dependencies. onnx fails to build from source
on macOS due to CMake version incompatibility.

TADA only uses Snake1d (a 7-line PyTorch module) from DAC. This commit
adds a shim in backend/utils/dac_shim.py that registers fake dac.*
modules in sys.modules with just the Snake1d class, completely
eliminating the DAC/audiotools dependency chain.
2026-03-17 02:16:33 -07:00
James Pine 4e7772a21d add HumeAI TADA TTS engine (1B English + 3B Multilingual)
Integrates HumeAI's TADA (Text-Acoustic Dual Alignment) speech-language
model as a new TTS engine. TADA uses a novel 1:1 token-audio alignment
that produces coherent speech over long sequences (700s+).

Two model variants:
- tada-1b: English-only, ~4GB, built on Llama 3.2 1B
- tada-3b-ml: 10 languages, ~8GB, built on Llama 3.2 3B

Backend uses the Encoder for voice prompt encoding with caching, and
TadaForCausalLM with flow-matching diffusion for generation. Supports
bf16 inference on CUDA, forces CPU on macOS (MPS compatibility).

Installed with --no-deps due to torch>=2.7 pin conflict; descript-audio-codec
and torchaudio added as explicit sub-dependencies.
2026-03-17 01:55:15 -07:00
James Pine 51fb320b8c readme 2026-03-17 01:24:48 -07:00
James Pine 8ac202aa58 docs for adding new engines 2026-03-17 01:13:30 -07:00
20 changed files with 1058 additions and 61 deletions
+120
View File
@@ -0,0 +1,120 @@
---
name: add-tts-engine
description: Use this skill to add a new TTS engine to Voicebox. It walks through dependency research, backend implementation, frontend wiring, PyInstaller bundling, and frozen-build testing. Always start with Phase 0 (dependency audit) before writing any code.
---
# Add TTS Engine
## Goal
Integrate a new text-to-speech engine into Voicebox end-to-end: dependency research, backend protocol implementation, frontend UI wiring, PyInstaller bundling, and frozen-build verification. The user should only need to test the final build locally.
## Reference Doc
The full phased guide lives at `docs/content/docs/developer/tts-engines.mdx`. **Read this file in its entirety before starting.** It contains:
- Phase 0: Dependency research (mandatory before writing code)
- Phase 1: Backend implementation (`TTSBackend` protocol)
- Phase 2: Route and service integration (usually zero changes)
- Phase 3: Frontend integration (5 files)
- Phase 4: Dependencies (`requirements.txt`, justfile, CI, Docker)
- Phase 5: PyInstaller bundling (`build_binary.py` + `server.py`)
- Phase 6: Common upstream workarounds
- Implementation checklist (gate between phases)
## Workflow
### 1. Read the guide
```bash
# Read the full TTS engines doc
cat docs/content/docs/developer/tts-engines.mdx
```
Internalize all phases, especially Phase 0 and Phase 5. The v0.2.3 release was three patch releases because Phase 0 was skipped.
### 2. Dependency research (Phase 0)
Clone the model library into a temporary directory and audit it. Do NOT skip this.
```bash
mkdir /tmp/engine-research && cd /tmp/engine-research
git clone <model-library-url>
```
Run the grep searches from Phase 0.2 in the guide against the cloned source and its transitive dependencies. Produce a written dependency audit covering:
1. PyPI vs non-PyPI packages
2. PyInstaller directives needed (`--collect-all`, `--copy-metadata`, `--hidden-import`)
3. Runtime data files that must be bundled
4. Native library paths that need env var overrides in frozen builds
5. Monkey-patches needed (`torch.load`, float64, MPS, HF token)
6. Sample rate
7. Model download method (`from_pretrained` vs `snapshot_download` + `from_local`)
Test model loading and generation on CPU in the throwaway venv before proceeding.
### 3. Implement (Phases 1–4)
Follow the guide's phases in order. Key files to modify:
**Backend (Phase 1):**
- Create `backend/backends/<engine>_backend.py`
- Register in `backend/backends/__init__.py` (ModelConfig + TTS_ENGINES + factory)
- Update regex in `backend/models.py`
**Frontend (Phase 3):**
- `app/src/lib/api/types.ts` — engine union type
- `app/src/lib/constants/languages.ts` — ENGINE_LANGUAGES
- `app/src/components/Generation/EngineModelSelector.tsx` — ENGINE_OPTIONS, ENGINE_DESCRIPTIONS
- `app/src/lib/hooks/useGenerationForm.ts` — Zod schema, model-name mapping
- `app/src/components/ServerSettings/ModelManagement.tsx` — MODEL_DESCRIPTIONS
**Dependencies (Phase 4):**
- `backend/requirements.txt`
- `justfile` (setup-python, setup-python-release targets)
- `.github/workflows/release.yml`
- `Dockerfile` (if applicable)
### 4. PyInstaller bundling (Phase 5)
Register the engine in `backend/build_binary.py`:
- `--hidden-import` for the backend module and model package
- `--collect-all` for packages using `inspect.getsource`, shipping data files, or native libraries
- `--copy-metadata` for packages using `importlib.metadata`
If the engine has native data paths, add `os.environ.setdefault()` in `backend/server.py` inside the `if getattr(sys, 'frozen', False):` block.
### 5. Verify in dev mode
```bash
just dev
```
Test the full chain: model download → load → generate → voice cloning.
### 6. Use the checklist
Walk through the Implementation Checklist at the bottom of `tts-engines.mdx`. Every item must be checked before handing the build to the user.
## Key Lessons (from v0.2.3)
These are the most common failure modes. Phase 0 research catches all of them:
| Pattern | Symptom in Frozen Build | Fix |
|---------|------------------------|-----|
| `@typechecked` / `inspect.getsource()` | "could not get source code" | `--collect-all <package>` |
| Package ships pretrained model files | `FileNotFoundError` for `.pth.tar`, `.yaml` | `--collect-all <package>` |
| C library with hardcoded system paths | `FileNotFoundError` for `/usr/share/...` | `--collect-all` + env var in `server.py` |
| `importlib.metadata.version()` | "No package metadata found" | `--copy-metadata <package>` |
| `torch.load` without `map_location` | CUDA device not available on CPU build | Monkey-patch `torch.load` |
| `torch.from_numpy` on float64 data | dtype mismatch RuntimeError | Cast to `.float()` |
| `token=True` in HF download calls | Auth failure without stored HF token | Use `snapshot_download(token=None)` + `from_local()` |
## Notes
- The route and service layers have zero per-engine dispatch points. `main.py` requires zero changes.
- The model config registry in `backends/__init__.py` handles all dispatch automatically.
- Use `get_torch_device()` and `model_load_progress()` from `backends/base.py` — don't reimplement device detection or progress tracking.
- Always test with a **clean HuggingFace cache** (no pre-downloaded models from dev).
- Do NOT push or create a release. Hand the build to the user for local testing.
+2
View File
@@ -62,6 +62,7 @@ jobs:
pip install pyinstaller
pip install -r backend/requirements.txt
pip install --no-deps chatterbox-tts
pip install --no-deps hume-tada
- name: Install MLX dependencies (Apple Silicon only)
if: matrix.backend == 'mlx'
@@ -188,6 +189,7 @@ jobs:
pip install pyinstaller
pip install -r backend/requirements.txt
pip install --no-deps chatterbox-tts
pip install --no-deps hume-tada
- name: Install PyTorch with CUDA 12.6
run: |
+2
View File
@@ -35,6 +35,8 @@ RUN pip install --no-cache-dir --upgrade pip
COPY backend/requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt
RUN pip install --no-cache-dir --prefix=/install --no-deps chatterbox-tts
RUN pip install --no-cache-dir --prefix=/install --no-deps hume-tada
RUN pip install --no-cache-dir --prefix=/install \
git+https://github.com/QwenLM/Qwen3-TTS.git
+12 -5
View File
@@ -59,10 +59,10 @@
## What is Voicebox?
Voicebox is a **local-first voice cloning studio** — a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
Voicebox is a **local-first voice cloning studio** — a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 5 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
- **Complete privacy** — models and voice data stay on your machine
- **4 TTS engines** — Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo
- **5 TTS engines** — Qwen3-TTS, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo, and HumeAI TADA
- **23 languages** — from English to Arabic, Japanese, Hindi, Swahili, and more
- **Post-processing effects** — pitch shift, reverb, delay, chorus, compression, and filters
- **Expressive speech** — paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo
@@ -93,7 +93,7 @@ Voicebox is a **local-first voice cloning studio** — a free and open-source al
### Multi-Engine Voice Cloning
Four TTS engines with different strengths, switchable per-generation:
Five TTS engines with different strengths, switchable per-generation:
| Engine | Languages | Strengths |
| --------------------------- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
@@ -101,6 +101,7 @@ Four TTS engines with different strengths, switchable per-generation:
| **LuxTTS** | English | Lightweight (~1GB VRAM), 48kHz output, 150x realtime on CPU |
| **Chatterbox Multilingual** | 23 | Broadest language coverage — Arabic, Danish, Finnish, Greek, Hebrew, Hindi, Malay, Norwegian, Polish, Swahili, Swedish, Turkish and more |
| **Chatterbox Turbo** | English | Fast 350M model with paralinguistic emotion/sound tags |
| **TADA** (1B / 3B) | 10 | HumeAI speech-language model — 700s+ coherent audio, text-acoustic dual alignment |
### Emotions & Paralinguistic Tags
@@ -230,7 +231,7 @@ Full API documentation available at `http://localhost:17493/docs`.
| Frontend | React, TypeScript, Tailwind CSS |
| State | Zustand, React Query |
| Backend | FastAPI (Python) |
| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo |
| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo, TADA |
| Effects | Pedalboard (Spotify) |
| Transcription | Whisper / Whisper Turbo (PyTorch or MLX) |
| Inference | MLX (Apple Silicon) / PyTorch (CUDA/ROCm/XPU/CPU) |
@@ -245,7 +246,7 @@ Full API documentation available at `http://localhost:17493/docs`.
| ----------------------- | ---------------------------------------------- |
| **Real-time Streaming** | Stream audio as it generates, word by word |
| **Voice Design** | Create new voices from text descriptions |
| **More Models** | XTTS, Bark, and other open-source voice models |
| **More Models** | XTTS, Bark, and other open-source voice models |
| **Plugin Architecture** | Extend with custom models and effects |
| **Mobile Companion** | Control Voicebox from your phone |
@@ -276,6 +277,12 @@ just build # Build CPU server binary + Tauri app
just build-local # (Windows) Build CPU + CUDA server binaries + Tauri app
```
### Adding New Voice Models
The multi-engine architecture makes adding new TTS engines straightforward. A [step-by-step guide](docs/content/docs/developer/tts-engines.mdx) covers the full process: dependency research, backend protocol implementation, frontend wiring, and PyInstaller bundling.
The guide is optimized for AI coding agents. An [agent skill](.agents/skills/add-tts-engine/SKILL.md) can pick up a model name and handle the entire integration autonomously — you just test the build locally.
### Project Structure
```
@@ -20,6 +20,8 @@ const ENGINE_OPTIONS = [
{ value: 'luxtts', label: 'LuxTTS' },
{ value: 'chatterbox', label: 'Chatterbox' },
{ value: 'chatterbox_turbo', label: 'Chatterbox Turbo' },
{ value: 'tada:1B', label: 'TADA 1B' },
{ value: 'tada:3B', label: 'TADA 3B Multilingual' },
] as const;
const ENGINE_DESCRIPTIONS: Record<string, string> = {
@@ -27,6 +29,7 @@ const ENGINE_DESCRIPTIONS: Record<string, string> = {
luxtts: 'Fast, English-focused',
chatterbox: '23 languages, incl. Hebrew',
chatterbox_turbo: 'English, [laugh] [cough] tags',
tada: 'HumeAI, 700s+ coherent audio',
};
/** Engines that only support English and should force language to 'en' on select. */
@@ -34,6 +37,7 @@ const ENGLISH_ONLY_ENGINES = new Set(['luxtts', 'chatterbox_turbo']);
function getSelectValue(engine: string, modelSize?: string): string {
if (engine === 'qwen') return `qwen:${modelSize || '1.7B'}`;
if (engine === 'tada') return `tada:${modelSize || '1B'}`;
return engine;
}
@@ -48,6 +52,20 @@ function handleEngineChange(form: UseFormReturn<GenerationFormValues>, value: st
if (!available.some((l) => l.value === currentLang)) {
form.setValue('language', available[0]?.value ?? 'en');
}
} else if (value.startsWith('tada:')) {
const [, modelSize] = value.split(':');
form.setValue('engine', 'tada');
form.setValue('modelSize', modelSize as '1B' | '3B');
// TADA 1B is English-only; 3B is multilingual
if (modelSize === '1B') {
form.setValue('language', 'en');
} else {
const currentLang = form.getValues('language');
const available = getLanguageOptionsForEngine('tada');
if (!available.some((l) => l.value === currentLang)) {
form.setValue('language', available[0]?.value ?? 'en');
}
}
} else {
form.setValue('engine', value as GenerationFormValues['engine']);
form.setValue('modelSize', undefined as unknown as '1.7B' | '0.6B');
@@ -62,6 +62,10 @@ const MODEL_DESCRIPTIONS: Record<string, string> = {
'Production-grade open source TTS by Resemble AI. Supports 23 languages with voice cloning and emotion exaggeration control.',
'chatterbox-turbo':
'Streamlined 350M parameter TTS by Resemble AI. High-quality English speech with less compute and VRAM than larger models.',
'tada-1b':
'HumeAI TADA 1B — English speech-language model built on Llama 3.2 1B. Generates 700s+ of coherent audio with synchronized text-acoustic alignment.',
'tada-3b-ml':
'HumeAI TADA 3B Multilingual — built on Llama 3.2 3B. Supports 10 languages with high-fidelity voice cloning via text-acoustic dual alignment.',
'whisper-base':
'Smallest Whisper model (74M parameters). Fast transcription with moderate accuracy.',
'whisper-small':
@@ -391,7 +395,8 @@ export function ModelManagement() {
(m) =>
m.model_name.startsWith('qwen-tts') ||
m.model_name.startsWith('luxtts') ||
m.model_name.startsWith('chatterbox'),
m.model_name.startsWith('chatterbox') ||
m.model_name.startsWith('tada'),
) ?? [];
const whisperModels = modelStatus?.models.filter((m) => m.model_name.startsWith('whisper')) ?? [];
+2 -2
View File
@@ -42,8 +42,8 @@ export interface GenerationRequest {
text: string;
language: LanguageCode;
seed?: number;
model_size?: '1.7B' | '0.6B';
engine?: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo';
model_size?: '1.7B' | '0.6B' | '1B' | '3B';
engine?: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo' | 'tada';
instruct?: string;
max_chunk_chars?: number;
crossfade_ms?: number;
+1
View File
@@ -66,6 +66,7 @@ export const ENGINE_LANGUAGES: Record<string, readonly LanguageCode[]> = {
'zh',
],
chatterbox_turbo: ['en'],
tada: ['en', 'ar', 'zh', 'de', 'es', 'fr', 'it', 'ja', 'pl', 'pt'],
} as const;
/** Helper: get language options for a given engine. */
+17 -9
View File
@@ -15,9 +15,9 @@ const generationSchema = z.object({
text: z.string().min(1, '').max(50000),
language: z.enum(LANGUAGE_CODES as [LanguageCode, ...LanguageCode[]]),
seed: z.number().int().optional(),
modelSize: z.enum(['1.7B', '0.6B']).optional(),
modelSize: z.enum(['1.7B', '0.6B', '1B', '3B']).optional(),
instruct: z.string().max(500).optional(),
engine: z.enum(['qwen', 'luxtts', 'chatterbox', 'chatterbox_turbo']).optional(),
engine: z.enum(['qwen', 'luxtts', 'chatterbox', 'chatterbox_turbo', 'tada']).optional(),
});
export type GenerationFormValues = z.infer<typeof generationSchema>;
@@ -79,7 +79,11 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
? 'chatterbox-tts'
: engine === 'chatterbox_turbo'
? 'chatterbox-turbo'
: `qwen-tts-${data.modelSize}`;
: engine === 'tada'
? data.modelSize === '3B'
? 'tada-3b-ml'
: 'tada-1b'
: `qwen-tts-${data.modelSize}`;
const displayName =
engine === 'luxtts'
? 'LuxTTS'
@@ -87,9 +91,13 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
? 'Chatterbox TTS'
: engine === 'chatterbox_turbo'
? 'Chatterbox Turbo'
: data.modelSize === '1.7B'
? 'Qwen TTS 1.7B'
: 'Qwen TTS 0.6B';
: engine === 'tada'
? data.modelSize === '3B'
? 'TADA 3B Multilingual'
: 'TADA 1B'
: data.modelSize === '1.7B'
? 'Qwen TTS 1.7B'
: 'Qwen TTS 0.6B';
// Check if model needs downloading
try {
@@ -104,7 +112,7 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
console.error('Failed to check model status:', error);
}
const isQwen = engine === 'qwen';
const hasModelSizes = engine === 'qwen' || engine === 'tada';
const effectsChain = options.getEffectsChain?.();
// This now returns immediately with status="generating"
const result = await generation.mutateAsync({
@@ -112,9 +120,9 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
text: data.text,
language: data.language,
seed: data.seed,
model_size: isQwen ? data.modelSize : undefined,
model_size: hasModelSizes ? data.modelSize : undefined,
engine,
instruct: isQwen ? data.instruct || undefined : undefined,
instruct: engine === 'qwen' ? data.instruct || undefined : undefined,
max_chunk_chars: maxChunkChars,
crossfade_ms: crossfadeMs,
normalize: normalizeAudio,
+27 -2
View File
@@ -166,6 +166,7 @@ TTS_ENGINES = {
"luxtts": "LuxTTS",
"chatterbox": "Chatterbox TTS",
"chatterbox_turbo": "Chatterbox Turbo",
"tada": "TADA",
}
@@ -259,6 +260,24 @@ def _get_non_qwen_tts_configs() -> list[ModelConfig]:
needs_trim=True,
languages=["en"],
),
ModelConfig(
model_name="tada-1b",
display_name="TADA 1B (English)",
engine="tada",
hf_repo_id="HumeAI/tada-1b",
model_size="1B",
size_mb=4000,
languages=["en"],
),
ModelConfig(
model_name="tada-3b-ml",
display_name="TADA 3B Multilingual",
engine="tada",
hf_repo_id="HumeAI/tada-3b-ml",
model_size="3B",
size_mb=8000,
languages=["en", "ar", "zh", "de", "es", "fr", "it", "ja", "pl", "pt"],
),
]
@@ -339,10 +358,12 @@ def engine_has_model_sizes(engine: str) -> bool:
async def load_engine_model(engine: str, model_size: str = "default") -> None:
"""Load a model for the given engine, handling the Qwen model_size special case."""
"""Load a model for the given engine, handling engines with multiple model sizes."""
backend = get_tts_backend_for_engine(engine)
if engine == "qwen":
await backend.load_model_async(model_size)
elif engine == "tada":
await backend.load_model(model_size)
else:
await backend.load_model()
@@ -358,7 +379,7 @@ async def ensure_model_cached_or_raise(engine: str, model_size: str = "default")
cfg = c
break
if engine == "qwen":
if engine in ("qwen", "tada"):
if not backend._is_model_cached(model_size):
raise HTTPException(
status_code=400,
@@ -490,6 +511,10 @@ def get_tts_backend_for_engine(engine: str) -> TTSBackend:
from .chatterbox_turbo_backend import ChatterboxTurboTTSBackend
backend = ChatterboxTurboTTSBackend()
elif engine == "tada":
from .hume_backend import HumeTadaBackend
backend = HumeTadaBackend()
else:
raise ValueError(f"Unknown TTS engine: {engine}. Supported: {list(TTS_ENGINES.keys())}")
+347
View File
@@ -0,0 +1,347 @@
"""
HumeAI TADA TTS backend implementation.
Wraps HumeAI's TADA (Text-Acoustic Dual Alignment) model for
high-quality voice cloning. Two model variants:
- tada-1b: English-only, ~2B params (Llama 3.2 1B base)
- tada-3b-ml: Multilingual, ~4B params (Llama 3.2 3B base)
Both use a shared encoder/codec (HumeAI/tada-codec). The encoder
produces 1:1 aligned token embeddings from reference audio, and the
causal LM generates speech via flow-matching diffusion.
24kHz output, bf16 inference on CUDA, fp32 on CPU.
"""
import asyncio
import logging
import threading
from typing import ClassVar, List, Optional, Tuple
import numpy as np
from . import TTSBackend
from .base import (
is_model_cached,
get_torch_device,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
logger = logging.getLogger(__name__)
# HuggingFace repos
TADA_CODEC_REPO = "HumeAI/tada-codec"
TADA_1B_REPO = "HumeAI/tada-1b"
TADA_3B_ML_REPO = "HumeAI/tada-3b-ml"
TADA_MODEL_REPOS = {
"1B": TADA_1B_REPO,
"3B": TADA_3B_ML_REPO,
}
# Key weight files for cache detection
_TADA_MODEL_WEIGHT_FILES = [
"model.safetensors",
]
_TADA_CODEC_WEIGHT_FILES = [
"encoder/model.safetensors",
]
class HumeTadaBackend:
"""HumeAI TADA TTS backend for high-quality voice cloning."""
_load_lock: ClassVar[threading.Lock] = threading.Lock()
def __init__(self):
self.model = None
self.encoder = None
self.model_size = "1B" # default to 1B
self._device = None
self._model_load_lock = asyncio.Lock()
def _get_device(self) -> str:
# Force CPU on macOS — MPS has issues with flow matching
# and large vocab lm_head (>65536 output channels)
return get_torch_device(force_cpu_on_mac=True)
def is_loaded(self) -> bool:
return self.model is not None
def _get_model_path(self, model_size: str = "1B") -> str:
return TADA_MODEL_REPOS.get(model_size, TADA_1B_REPO)
def _is_model_cached(self, model_size: str = "1B") -> bool:
repo = TADA_MODEL_REPOS.get(model_size, TADA_1B_REPO)
model_cached = is_model_cached(repo, required_files=_TADA_MODEL_WEIGHT_FILES)
codec_cached = is_model_cached(TADA_CODEC_REPO, required_files=_TADA_CODEC_WEIGHT_FILES)
return model_cached and codec_cached
async def load_model(self, model_size: str = "1B") -> None:
"""Load the TADA model and encoder."""
if self.model is not None and self.model_size == model_size:
return
async with self._model_load_lock:
if self.model is not None and self.model_size == model_size:
return
# Unload existing model if switching sizes
if self.model is not None:
self.unload_model()
self.model_size = model_size
await asyncio.to_thread(self._load_model_sync, model_size)
def _load_model_sync(self, model_size: str = "1B"):
"""Synchronous model loading with progress tracking."""
model_name = f"tada-{model_size.lower()}"
is_cached = self._is_model_cached(model_size)
repo = TADA_MODEL_REPOS.get(model_size, TADA_1B_REPO)
with model_load_progress(model_name, is_cached):
# Install DAC shim before importing tada — tada's encoder/decoder
# import dac.nn.layers.Snake1d which requires the descript-audio-codec
# package. The real package pulls in onnx/tensorboard/matplotlib via
# descript-audiotools, so we use a lightweight shim instead.
from ..utils.dac_shim import install_dac_shim
install_dac_shim()
import torch
from huggingface_hub import snapshot_download
device = self._get_device()
self._device = device
logger.info(f"Loading HumeAI TADA {model_size} on {device}...")
# Download codec (encoder + decoder) if not cached
logger.info("Downloading TADA codec...")
snapshot_download(
repo_id=TADA_CODEC_REPO,
token=None,
allow_patterns=["*.safetensors", "*.json", "*.txt", "*.bin"],
)
# Download model weights if not cached
logger.info(f"Downloading TADA {model_size} model...")
snapshot_download(
repo_id=repo,
token=None,
allow_patterns=["*.safetensors", "*.json", "*.txt", "*.bin", "*.model"],
)
# TADA hardcodes "meta-llama/Llama-3.2-1B" as the tokenizer
# source in its Aligner and TadaForCausalLM.from_pretrained().
# That repo is gated (requires Meta license acceptance).
# Download the tokenizer from an ungated mirror and get its
# local cache path so we can point TADA at it directly.
logger.info("Downloading Llama tokenizer (ungated mirror)...")
tokenizer_path = snapshot_download(
repo_id="unsloth/Llama-3.2-1B",
token=None,
allow_patterns=["tokenizer*", "special_tokens*"],
)
# Determine dtype — use bf16 on CUDA for ~50% memory savings
if device == "cuda" and torch.cuda.is_bf16_supported():
model_dtype = torch.bfloat16
else:
model_dtype = torch.float32
# Patch the Aligner config class to use the local tokenizer
# path instead of the gated "meta-llama/Llama-3.2-1B" default.
# This avoids monkey-patching AutoTokenizer.from_pretrained
# which corrupts the classmethod descriptor for other engines.
from tada.modules.aligner import AlignerConfig
AlignerConfig.tokenizer_name = tokenizer_path
# Load encoder (only needed for voice prompt encoding)
from tada.modules.encoder import Encoder
logger.info("Loading TADA encoder...")
self.encoder = Encoder.from_pretrained(
TADA_CODEC_REPO, subfolder="encoder"
).to(device)
self.encoder.eval()
# Load the causal LM (includes decoder for wav generation).
# TadaForCausalLM.from_pretrained() calls
# getattr(config, "tokenizer_name", "meta-llama/Llama-3.2-1B")
# which hits the gated repo. Pre-load the config from HF,
# inject the local tokenizer path, then pass it in.
from tada.modules.tada import TadaForCausalLM, TadaConfig
logger.info(f"Loading TADA {model_size} model...")
config = TadaConfig.from_pretrained(repo)
config.tokenizer_name = tokenizer_path
self.model = TadaForCausalLM.from_pretrained(
repo, config=config, torch_dtype=model_dtype
).to(device)
self.model.eval()
logger.info(f"HumeAI TADA {model_size} loaded successfully on {device}")
def unload_model(self) -> None:
"""Unload model and encoder to free memory."""
if self.model is not None:
del self.model
self.model = None
if self.encoder is not None:
del self.encoder
self.encoder = None
self._device = None
import torch
if torch.cuda.is_available():
torch.cuda.empty_cache()
logger.info("HumeAI TADA unloaded")
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio using TADA's encoder.
TADA's encoder performs forced alignment between audio and text tokens,
producing an EncoderOutput with 1:1 token-audio alignment. If no
reference_text is provided, the encoder uses built-in ASR (English only).
We serialize the EncoderOutput to a dict for caching.
"""
await self.load_model(self.model_size)
cache_key = (
"tada_" + get_cache_key(audio_path, reference_text)
) if use_cache else None
if cache_key:
cached = get_cached_voice_prompt(cache_key)
if cached is not None and isinstance(cached, dict):
return cached, True
def _encode_sync():
import torch
import soundfile as sf
device = self._device
# Load audio with soundfile (torchaudio 2.10+ requires torchcodec)
audio_np, sr = sf.read(str(audio_path), dtype="float32")
audio = torch.from_numpy(audio_np).float()
if audio.ndim == 1:
audio = audio.unsqueeze(0) # (samples,) -> (1, samples)
else:
audio = audio.T # (samples, channels) -> (channels, samples)
audio = audio.to(device)
# Encode with forced alignment
text_arg = [reference_text] if reference_text else None
prompt = self.encoder(
audio, text=text_arg, sample_rate=sr
)
# Serialize EncoderOutput to a dict of CPU tensors for caching
prompt_dict = {}
for field_name in prompt.__dataclass_fields__:
val = getattr(prompt, field_name)
if isinstance(val, torch.Tensor):
prompt_dict[field_name] = val.detach().cpu()
elif isinstance(val, list):
prompt_dict[field_name] = val
elif isinstance(val, (int, float)):
prompt_dict[field_name] = val
else:
prompt_dict[field_name] = val
return prompt_dict
encoded = await asyncio.to_thread(_encode_sync)
if cache_key:
cache_voice_prompt(cache_key, encoded)
return encoded, False
async def combine_voice_prompts(
self,
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
return await _combine_voice_prompts(audio_paths, reference_texts, sample_rate=24000)
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text using HumeAI TADA.
Args:
text: Text to synthesize
voice_prompt: Serialized EncoderOutput dict from create_voice_prompt()
language: Language code (en, ar, de, es, fr, it, ja, pl, pt, zh)
seed: Random seed for reproducibility
instruct: Not supported by TADA (ignored)
Returns:
Tuple of (audio_array, sample_rate=24000)
"""
await self.load_model(self.model_size)
def _generate_sync():
import torch
from tada.modules.encoder import EncoderOutput
if seed is not None:
torch.manual_seed(seed)
if torch.cuda.is_available():
torch.cuda.manual_seed(seed)
device = self._device
# Reconstruct EncoderOutput from the cached dict
restored = {}
for k, v in voice_prompt.items():
if isinstance(v, torch.Tensor):
# Move to device and match model dtype for float tensors
if v.is_floating_point():
model_dtype = next(self.model.parameters()).dtype
restored[k] = v.to(device=device, dtype=model_dtype)
else:
restored[k] = v.to(device=device)
else:
restored[k] = v
prompt = EncoderOutput(**restored)
# For non-English with the 3B-ML model, we could reload the
# encoder with the language-specific aligner. However, the
# generation itself is language-agnostic — only the encoder's
# aligner changes. Since we encode at create_voice_prompt time,
# the language is already baked in. For simplicity, we don't
# reload the encoder here.
logger.info(f"[TADA] Generating ({language}), text length: {len(text)}")
output = self.model.generate(
prompt=prompt,
text=text,
)
# output.audio is a list of tensors (one per batch item)
if output.audio and output.audio[0] is not None:
audio_tensor = output.audio[0]
audio = audio_tensor.detach().cpu().numpy().squeeze().astype(np.float32)
else:
logger.warning("[TADA] Generation produced no audio")
audio = np.zeros(24000, dtype=np.float32)
return audio, 24000
return await asyncio.to_thread(_generate_sync)
+36
View File
@@ -186,6 +186,42 @@ def build_server(cuda=False):
# needed by LuxTTS for text-to-phoneme conversion
"--collect-all",
"piper_phonemize",
# HumeAI TADA — speech-language model using Llama + flow matching
"--hidden-import",
"backend.backends.hume_backend",
"--hidden-import",
"tada",
"--hidden-import",
"tada.modules",
"--hidden-import",
"tada.modules.tada",
"--hidden-import",
"tada.modules.encoder",
"--hidden-import",
"tada.modules.decoder",
"--hidden-import",
"tada.modules.aligner",
"--hidden-import",
"tada.modules.acoustic_spkr_verf",
"--hidden-import",
"tada.nn",
"--hidden-import",
"tada.nn.vibevoice",
"--hidden-import",
"tada.utils",
"--hidden-import",
"tada.utils.gray_code",
"--hidden-import",
"tada.utils.text",
# DAC shim — provides dac.nn.layers.Snake1d without the real
# descript-audio-codec package (which pulls onnx/tensorboard via
# descript-audiotools). The shim is in backend/utils/dac_shim.py.
"--hidden-import",
"backend.utils.dac_shim",
"--hidden-import",
"torchaudio",
"--collect-submodules",
"tada",
]
)
+2 -2
View File
@@ -66,9 +66,9 @@ class GenerationRequest(BaseModel):
text: str = Field(..., min_length=1, max_length=50000)
language: str = Field(default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it|he|ar|da|el|fi|hi|ms|nl|no|pl|sv|sw|tr)$")
seed: Optional[int] = Field(None, ge=0)
model_size: Optional[str] = Field(default="1.7B", pattern="^(1\\.7B|0\\.6B)$")
model_size: Optional[str] = Field(default="1.7B", pattern="^(1\\.7B|0\\.6B|1B|3B)$")
instruct: Optional[str] = Field(None, max_length=500)
engine: Optional[str] = Field(default="qwen", pattern="^(qwen|luxtts|chatterbox|chatterbox_turbo)$")
engine: Optional[str] = Field(default="qwen", pattern="^(qwen|luxtts|chatterbox|chatterbox_turbo|tada)$")
max_chunk_chars: int = Field(
default=800, ge=100, le=5000, description="Max characters per chunk for long text splitting"
)
+7
View File
@@ -33,6 +33,13 @@ s3tokenizer
spacy-pkuseg
pyloudnorm
# HumeAI TADA sub-dependencies (hume-tada itself is installed
# --no-deps in the setup script because it pins torch>=2.7,<2.8.
# descript-audio-codec is NOT installed — it pulls onnx/tensorboard
# via descript-audiotools. A lightweight shim in utils/dac_shim.py
# provides the only class TADA uses: Snake1d.)
torchaudio
# Audio processing
librosa>=0.10.0
soundfile>=0.12.0
+95
View File
@@ -0,0 +1,95 @@
"""
Minimal shim for descript-audio-codec (DAC).
TADA only imports Snake1d from dac.nn.layers and dac.model.dac.
The real DAC package pulls in descript-audiotools which depends on
onnx, tensorboard, protobuf, matplotlib, pystoi, etc. — none of
which are needed for TADA's runtime use of Snake1d.
This shim provides the exact Snake1d implementation (MIT-licensed,
from https://github.com/descriptinc/descript-audio-codec) so we can
avoid the entire audiotools dependency chain.
If the real DAC package is installed, this module is never used —
Python's import system will find the site-packages version first.
Install this shim only when descript-audio-codec is NOT installed.
"""
import sys
import types
import torch
import torch.nn as nn
# ── Snake activation (from dac/nn/layers.py) ────────────────────────
# NOTE: The original DAC code uses @torch.jit.script here for a 1.4x
# speedup. We omit it because TorchScript calls inspect.getsource()
# which fails inside a PyInstaller frozen binary (no .py source files).
def snake(x: torch.Tensor, alpha: torch.Tensor) -> torch.Tensor:
shape = x.shape
x = x.reshape(shape[0], shape[1], -1)
x = x + (alpha + 1e-9).reciprocal() * torch.sin(alpha * x).pow(2)
x = x.reshape(shape)
return x
class Snake1d(nn.Module):
def __init__(self, channels: int):
super().__init__()
self.alpha = nn.Parameter(torch.ones(1, channels, 1))
def forward(self, x: torch.Tensor) -> torch.Tensor:
return snake(x, self.alpha)
# ── Register as dac.nn.layers and dac.model.dac ─────────────────────
def install_dac_shim() -> None:
"""Register fake dac package modules in sys.modules.
Only installs the shim if 'dac' is not already importable
(i.e. the real descript-audio-codec is not installed).
"""
try:
import dac # noqa: F401 — real package exists, do nothing
return
except ImportError:
pass
# Create the module tree: dac -> dac.nn -> dac.nn.layers
# -> dac.model -> dac.model.dac
dac_pkg = types.ModuleType("dac")
dac_pkg.__path__ = [] # make it a package
dac_pkg.__package__ = "dac"
dac_nn = types.ModuleType("dac.nn")
dac_nn.__path__ = []
dac_nn.__package__ = "dac.nn"
dac_nn_layers = types.ModuleType("dac.nn.layers")
dac_nn_layers.__package__ = "dac.nn"
dac_nn_layers.Snake1d = Snake1d
dac_nn_layers.snake = snake
dac_model = types.ModuleType("dac.model")
dac_model.__path__ = []
dac_model.__package__ = "dac.model"
dac_model_dac = types.ModuleType("dac.model.dac")
dac_model_dac.__package__ = "dac.model"
dac_model_dac.Snake1d = Snake1d
# Wire up submodules
dac_pkg.nn = dac_nn
dac_pkg.model = dac_model
dac_nn.layers = dac_nn_layers
dac_model.dac = dac_model_dac
# Register in sys.modules
sys.modules["dac"] = dac_pkg
sys.modules["dac.nn"] = dac_nn
sys.modules["dac.nn.layers"] = dac_nn_layers
sys.modules["dac.model"] = dac_model
sys.modules["dac.model.dac"] = dac_model_dac
+341 -29
View File
@@ -3,8 +3,12 @@ title: "TTS Engines"
description: "How to add new text-to-speech engines to Voicebox"
---
> **For humans:** This doc is optimized for AI agents to implement new TTS engines autonomously. It's structured as a phased workflow with explicit gates and a checklist so an agent can do the full integration — dependency research, backend, frontend, bundling — and hand you a draft release or prod build to test locally. It's also a useful reference if you're doing it yourself.
Adding an engine touches ~10 files across 4 layers. The backend protocol work is straightforward — the real time sink is dependency hell, upstream library bugs, and PyInstaller bundling.
**Do not start writing code until you complete Phase 0.** The v0.2.3 release was three patch releases of PyInstaller fixes because dependency research was skipped. Every issue — `inspect.getsource()` failures, missing native data files, metadata lookups, dtype mismatches — was discoverable by reading the model library's source code before integration began.
## Architecture Overview
The backend is split into layers:
@@ -18,6 +22,124 @@ The backend is split into layers:
New engines only need to touch `backends/` and `models.py` on the backend side — the route and service layers use a model config registry that handles dispatch automatically.
## Phase 0: Dependency Research
**This phase is mandatory.** Clone the model library and its key dependencies into a temporary directory and inspect them before writing any integration code. The goal is to produce a dependency audit that identifies every PyInstaller-incompatible pattern, every native data file, and every upstream bug you'll need to work around.
### 0.1 Clone and Inspect the Model Library
```bash
# Create a throwaway workspace
mkdir /tmp/engine-research && cd /tmp/engine-research
# Clone the model library
git clone https://github.com/org/model-library.git
cd model-library
```
**Read these files first, in order:**
1. **`setup.py` / `setup.cfg` / `pyproject.toml`** — Check pinned dependency versions. If the library pins `torch==2.6.0` or `numpy<1.26`, you'll need `--no-deps` installation and manual sub-dependency listing (this is what happened with `chatterbox-tts`).
2. **`__init__.py` and the main model class** — Trace the import chain. Look for:
- `from_pretrained()` — does it call `huggingface_hub` internally? Does it pass `token=True` (which crashes without a stored HF token)?
- `from_local()` — does it exist? You may need manual `snapshot_download()` + `from_local()` to bypass download bugs.
- Device handling — does it default to CUDA? Does it support MPS? Many libraries crash on MPS with unsupported operators.
3. **All `import` statements** — Recursively trace what the library imports. You're looking for:
- `inspect.getsource()` anywhere in the chain (search all `.py` files)
- `typeguard` / `@typechecked` decorators (these call `inspect.getsource()` at import time)
- `importlib.metadata.version()` or `pkg_resources.get_distribution()` (need `--copy-metadata`)
- `lazy_loader` (needs `--collect-all` to bundle `.pyi` stubs)
### 0.2 Scan for PyInstaller-Incompatible Patterns
Run these searches against the cloned library **and** its transitive dependencies:
```bash
# inspect.getsource — will crash in frozen binary without --collect-all
grep -r "inspect.getsource\|getsource(" .
# typeguard / @typechecked — calls inspect.getsource at import time
grep -r "@typechecked\|from typeguard" .
# importlib.metadata — needs --copy-metadata
grep -r "importlib.metadata\|pkg_resources.get_distribution\|pkg_resources.require" .
# Data files loaded at runtime — need --collect-all or --collect-data
grep -r "Path(__file__).parent\|os.path.dirname(__file__)\|resources_path\|pkg_resources.resource_filename" .
# Native library paths — may need env var override in frozen builds
grep -r "/usr/share\|/usr/lib\|/usr/local\|espeak\|phonemize" .
# torch.load without map_location — will crash on CPU-only builds
grep -r "torch.load(" . | grep -v "map_location"
# HuggingFace token bugs
grep -r 'token=True\|token=os.getenv' .
# Float64/Float32 assumptions — librosa returns float64, many models assume float32
grep -r "torch.from_numpy\|\.double()\|float64" .
```
### 0.3 Install and Trace in a Throwaway Venv
```bash
# Create isolated venv
python -m venv /tmp/engine-venv
source /tmp/engine-venv/bin/activate
# Install the package (try normally first)
pip install model-package
# Check if it conflicts with our stack
pip install model-package torch==2.10 transformers==4.57.3 numpy>=1.26
# If this fails, you need --no-deps:
pip install --no-deps model-package
# Get the full dependency tree
pip show model-package # Check Requires: field
pip show -f model-package # List all installed files (look for data files)
# Check for non-PyPI dependencies
pip install model-package 2>&1 | grep -i "no matching distribution"
```
### 0.4 Test Model Loading on CPU
Before writing any integration code, verify the model works on CPU in a plain Python script:
```python
import torch
# Force CPU to catch map_location bugs early
model = ModelClass.from_pretrained("org/model", device="cpu")
# Test with a float32 audio array (not float64)
import numpy as np
audio = np.random.randn(16000).astype(np.float32)
output = model.generate("Hello world", audio)
print(f"Output shape: {output.shape}, dtype: {output.dtype}, sample rate: {model.sample_rate}")
```
If this crashes, you've found a bug you'll need to monkey-patch. Common ones:
- `RuntimeError: expected scalar type Float but found Double` → needs float32 cast
- `RuntimeError: map_location` → needs `torch.load` patch
- `RuntimeError: Unsupported operator aten::...` → needs MPS skip
### 0.5 Produce a Dependency Audit
Before proceeding to Phase 1, write down:
1. **PyPI vs non-PyPI deps** — which packages need `--find-links`, `git+https://`, or `--no-deps`?
2. **PyInstaller directives needed** — which packages need `--collect-all`, `--copy-metadata`, `--hidden-import`?
3. **Runtime data files** — which packages ship data files (YAML, pretrained weights, phoneme tables, shader libraries) that must be bundled?
4. **Native library paths** — which packages look for data at system paths that won't exist in a frozen binary?
5. **Monkey-patches needed** — `torch.load` map_location, float64→float32 casts, MPS skip, HF token bypass, etc.
6. **Sample rate** — what does the engine output? (24kHz, 44.1kHz, 48kHz)
7. **Model download method** — `from_pretrained()` with library-managed download, or manual `snapshot_download()` + `from_local()`?
This audit becomes your implementation plan for Phases 1, 4, and 5.
## Phase 1: Backend Implementation
### 1.1 Create the Backend File
@@ -155,54 +277,172 @@ In `app/src/components/ServerSettings/ModelManagement.tsx`:
## Phase 4: Dependencies
Use the dependency audit from Phase 0 to drive this phase. You should already know what packages are needed, which conflict, and which require special installation.
### 4.1 Python Dependencies
Add to `backend/requirements.txt`. Watch for:
Add to `backend/requirements.txt`. There are three installation patterns, depending on what Phase 0 revealed:
**Pinned dependency conflicts** — If the model package pins old versions, install with `--no-deps`:
**Normal PyPI packages:**
```
some-model-package>=1.0.0
```
**Pinned dependency conflicts (`--no-deps`)** — If the model package pins old versions of torch/numpy/transformers, install with `--no-deps` and list sub-dependencies manually. This is the pattern used for `chatterbox-tts`:
```bash
# In justfile / CI setup:
pip install --no-deps chatterbox-tts
# In requirements.txt — list each actual sub-dependency:
conformer>=0.3.2
diffusers>=0.31.0
omegaconf>=2.3.0
resemble-perth>=0.0.2
s3tokenizer>=0.1.6
```
Then list sub-dependencies manually in `requirements.txt`.
To identify sub-deps: `pip show chatterbox-tts` → `Requires:` field, then cross-reference against existing `requirements.txt` to avoid duplicates.
**Non-PyPI packages:**
```
linacodec @ git+https://github.com/user/repo.git
**Non-PyPI packages** — Some libraries only exist on GitHub or require custom indexes:
```
# Git-only packages (no PyPI release)
linacodec @ git+https://github.com/ysharma3501/LinaCodec.git
Zipvoice @ git+https://github.com/ysharma3501/LuxTTS.git
**Custom package indexes:**
```
# Custom package indexes (C extensions with platform-specific wheels)
--find-links https://k2-fsa.github.io/icefall/piper_phonemize.html
piper-phonemize>=1.2.0
```
### 4.2 Identifying Hidden Sub-Dependencies
### 4.2 Dependency Conflict Resolution
1. Install the package normally in a throwaway venv
2. Run `pip show <package>` to get its `Requires:` list
3. Cross-reference against existing requirements.txt
4. Test that the engine loads and generates
Check for conflicts with the existing stack before adding anything:
## Phase 5: PyInstaller Bundling
```bash
# Our current stack pins (approximate):
# Python 3.12+, torch>=2.10, transformers>=4.57, numpy>=1.26
This is where most of the pain lives. Common issues:
# Test compatibility
pip install model-package torch==2.10 transformers==4.57.3 numpy>=1.26
| Issue | Symptom | Fix |
|-------|---------|-----|
| `inspect.getsource()` at import | "could not get source code" | `--collect-all <package>` |
| Data files (yaml, .pth.tar) | FileNotFoundError at runtime | `--collect-all <package>` |
| Native data paths (espeak-ng) | Library looks at `/usr/share/...` | Set env var in frozen builds |
| `importlib.metadata` lookups | "No package metadata found" | `--copy-metadata <package>` |
| Dynamic imports | ModuleNotFoundError | `--hidden-import <module>` |
# If it fails, check what the package pins:
pip show model-package | grep Requires
# Look at setup.py/pyproject.toml for version constraints
```
### Testing Frozen Builds
**Known incompatible patterns in the wild:**
- `torch==2.6.0` — many older packages pin this
- `numpy<1.26` — conflicts with Python 3.12+
- `transformers==4.46.3` — many packages pin old transformers
- `onnxruntime` pinned versions — often conflict with torch
You can't skip this. Models that work in `python -m uvicorn` will break in the PyInstaller binary.
### 4.3 Update Installation Scripts
Dependencies must be added in multiple places:
| File | What to add |
|------|------------|
| `backend/requirements.txt` | Package and version constraint |
| `justfile` | `--no-deps` install line if needed (in `setup-python` and `setup-python-release` targets) |
| `.github/workflows/release.yml` | Same `--no-deps` line in CI build steps |
| `Dockerfile` | Same install commands for Docker builds |
## Phase 5: PyInstaller Bundling (`build_binary.py`)
This is where most of the pain lives. **The v0.2.3 release was entirely dedicated to fixing bundling issues** — every new engine that shipped in v0.2.1 (LuxTTS, Chatterbox, Chatterbox Turbo) worked in dev but failed in production builds. Don't skip this phase.
### 5.1 Register Your Engine in `build_binary.py`
Every new engine needs entries in `backend/build_binary.py`. This file drives PyInstaller and is the single most common source of "works in dev, breaks in prod" bugs. You need to decide which PyInstaller directives your engine's dependencies require:
| Directive | What It Does | When You Need It |
|-----------|-------------|-----------------|
| `--hidden-import <module>` | Includes a module PyInstaller can't detect via static analysis | Dynamic imports, lazy imports, plugin architectures |
| `--collect-all <package>` | Bundles source `.py` files, data files, AND native libraries | Packages that call `inspect.getsource()` at import time (e.g. `inflect` via `typeguard`'s `@typechecked`), or that ship pretrained model files (e.g. `perth` ships `.pth.tar` + `hparams.yaml`) |
| `--collect-data <package>` | Bundles only data files (not source or native libs) | Packages with YAML configs, vocab files, etc. |
| `--collect-submodules <package>` | Bundles all submodules | Packages with deep module trees that PyInstaller misses |
| `--copy-metadata <package>` | Copies `importlib.metadata` info | Packages that call `importlib.metadata.version()` or `pkg_resources.get_distribution()` at runtime. Already required for: `requests`, `transformers`, `huggingface-hub`, `tokenizers`, `safetensors`, `tqdm` |
**Example: adding hidden imports and collect-all for a new engine:**
```python
# In build_binary.py, inside the args list:
"--hidden-import",
"backend.backends.your_engine_backend",
"--hidden-import",
"your_engine_package",
"--hidden-import",
"your_engine_package.inference",
"--collect-all",
"some_dependency_that_uses_inspect_getsource",
"--copy-metadata",
"some_dependency_that_checks_its_own_version",
```
### 5.2 Lessons from v0.2.3 — Real Failures and Their Fixes
These are actual production failures from shipping new engines. Every one of these passed `python -m uvicorn` in dev:
| Engine | Failure | Root Cause | Fix |
|--------|---------|-----------|-----|
| LuxTTS | `"could not get source code"` on import | `inflect` uses `typeguard`'s `@typechecked` which calls `inspect.getsource()` — needs `.py` source files, not just bytecode | `--collect-all inflect` |
| LuxTTS | `espeak-ng-data` not found | `piper_phonemize` C library looks for data at `/usr/share/espeak-ng-data/` which doesn't exist in the bundle | `--collect-all piper_phonemize` + set `ESPEAK_DATA_PATH` env var at runtime (see 5.3) |
| LuxTTS | `inspect.getsource` error in Vocos codec | `linacodec` and `zipvoice` use source introspection | `--collect-all linacodec` + `--collect-all zipvoice` |
| Chatterbox | `FileNotFoundError` for watermark model | `perth` ships pretrained model files (`hparams.yaml`, `.pth.tar`) that PyInstaller doesn't bundle by default | `--collect-all perth` |
| All engines | `importlib.metadata` failures | Frozen binary doesn't include package metadata for `huggingface-hub`, `transformers`, etc. | `--copy-metadata` for each affected package |
| All engines | Download progress bars stuck at 0% | `huggingface_hub` silently disables tqdm progress bars based on logger level in frozen builds — our progress tracker never receives byte updates | Force-enable tqdm's internal counter in `HFProgressTracker` |
| All engines | `NameError: name 'obj' is not defined` on macOS | Python 3.12.0 has a [CPython bug](https://github.com/pyinstaller/pyinstaller/issues/7992) that corrupts bytecode when PyInstaller rewrites code objects | Upgrade to Python 3.12.13+ |
| All engines | `resource_tracker` subprocess crash | `multiprocessing` in frozen binaries needs `freeze_support()` called before anything else | Added to `server.py` entry point |
### 5.3 Runtime Frozen-Build Handling (`server.py`)
Some fixes can't live in `build_binary.py` — they need runtime detection. The entry point `backend/server.py` handles these before any heavy imports:
```python
# 1. freeze_support() — MUST be called before any multiprocessing use
import multiprocessing
multiprocessing.freeze_support()
# 2. Native data paths — redirect C libraries to bundled data
if getattr(sys, 'frozen', False):
_meipass = getattr(sys, '_MEIPASS', os.path.dirname(sys.executable))
_espeak_data = os.path.join(_meipass, 'piper_phonemize', 'espeak-ng-data')
if os.path.isdir(_espeak_data):
os.environ.setdefault('ESPEAK_DATA_PATH', _espeak_data)
# 3. stdout/stderr safety — PyInstaller --noconsole on Windows sets these to None
if not _is_writable(sys.stdout):
sys.stdout = open(os.devnull, 'w')
```
If your engine's dependencies include native libraries that look for data at system paths (like espeak-ng does), you'll need to add a similar `os.environ.setdefault()` block here.
### 5.4 CUDA vs CPU Build Branching
`build_binary.py` produces two different binaries:
- **`voicebox-server`** (CPU) — excludes all `nvidia.*` packages to avoid bundling ~3 GB of CUDA DLLs
- **`voicebox-server-cuda`** — includes `torch.cuda` and `torch.backends.cudnn`
On Windows, if the build environment has CUDA torch installed but you're building the CPU binary, the script temporarily swaps to CPU-only torch and restores CUDA torch afterward. This prevents PyInstaller from accidentally bundling CUDA libraries into the CPU build.
New engine imports go in the **common section** (not the CUDA or MLX conditional blocks) unless your engine has platform-specific dependencies.
### 5.5 MLX Conditional Inclusion
Apple Silicon builds conditionally include MLX hidden imports and `--collect-all mlx` / `--collect-all mlx_audio`. If your engine has an MLX-specific backend variant, add its imports inside the `if is_apple_silicon() and not cuda:` block.
### 5.6 Testing Frozen Builds
You can't skip this. Models that work in `python -m uvicorn` will break in the PyInstaller binary. The v0.2.3 release required **three patch releases** (v0.2.1 → v0.2.2 → v0.2.3) to get all engines working in production.
1. Build: `just build`
2. Run and try download + load + generate
3. Check stderr for the actual error
4. Fix, rebuild, repeat
2. Launch the binary directly (not via `python -m`)
3. Test the **full chain**: download → load → generate → progress tracking
4. Check stderr for the actual error (logs go to stderr for Tauri sidecar capture)
5. Fix, rebuild, repeat
**Common gotcha:** testing only generation with a pre-cached model from your dev install. Always test with a clean model cache to verify downloads work too.
## Phase 6: Common Upstream Workarounds
@@ -250,8 +490,80 @@ Based on the current model landscape, these are candidates for future integratio
| **Fish Speech** | 50+ | Medium | Word-level control via inline text | Ready |
| **Kokoro-82M** | English | 82M | CPU realtime, Apache 2.0 | Ready |
| **XTTS-v2** | 17+ | Medium | Zero-shot cloning | Ready |
| **HumeAI TADA** | EN (1B), Multi (3B) | Medium | 700s+ coherent audio, synced transcripts | Needs vetting |
| **HumeAI TADA** | EN (1B), Multi (3B) | Medium | 700s+ coherent audio, synced transcripts | Shipped |
| **MOSS-TTS** | Multilingual | Medium | Text-to-voice design, multi-speaker dialogue | Needs vetting |
| **Pocket TTS** | English | ~100M | CPU-first, >1× realtime | Needs vetting |
The multi-engine architecture is now in place, making new model integration straightforward (~1 day for a well-documented model with a PyPI package).
## Implementation Checklist
Use this as a gate between phases. Do not proceed to the next phase until every item in the current phase is checked.
### Phase 0: Dependency Research
- [ ] Cloned model library source into a temp directory
- [ ] Read `setup.py` / `pyproject.toml` — noted pinned dependency versions
- [ ] Traced all imports from the model class through to leaf dependencies
- [ ] Searched for `inspect.getsource`, `@typechecked`, `typeguard` in the full dependency tree
- [ ] Searched for `importlib.metadata`, `pkg_resources.get_distribution` in the dependency tree
- [ ] Searched for `Path(__file__).parent`, `os.path.dirname(__file__)`, hardcoded system paths
- [ ] Searched for `torch.load` calls missing `map_location`
- [ ] Searched for `torch.from_numpy` without `.float()` cast
- [ ] Searched for `token=True` or `token=os.getenv("HF_TOKEN")` in HuggingFace calls
- [ ] Tested model loading and generation on CPU in a throwaway venv
- [ ] Tested with a clean HuggingFace cache (no pre-downloaded models)
- [ ] Produced a written dependency audit documenting all findings
### Phase 1: Backend Implementation
- [ ] Created `backend/backends/<engine>_backend.py` implementing `TTSBackend` protocol
- [ ] Chose voice prompt pattern (pre-computed tensors vs deferred file paths)
- [ ] Implemented all monkey-patches identified in Phase 0
- [ ] Used `get_torch_device()` from `backends/base.py` for device selection
- [ ] Used `model_load_progress()` from `backends/base.py` for download/load tracking
- [ ] Tested: model downloads correctly
- [ ] Tested: model loads on CPU
- [ ] Tested: generation produces valid audio
- [ ] Tested: voice cloning from reference audio works
- [ ] Registered `ModelConfig` in `backends/__init__.py`
- [ ] Added to `TTS_ENGINES` dict
- [ ] Added factory branch in `get_tts_backend_for_engine()`
- [ ] Updated engine regex in `backend/models.py`
### Phase 2–3: Route, Service, and Frontend
- [ ] Confirmed zero changes needed in routes/services (or documented why custom behavior is needed)
- [ ] Added engine to TypeScript union type in `app/src/lib/api/types.ts`
- [ ] Added language map entry in `app/src/lib/constants/languages.ts`
- [ ] Added to `ENGINE_OPTIONS` and `ENGINE_DESCRIPTIONS` in `EngineModelSelector.tsx`
- [ ] Added to Zod schema and model-name mapping in `useGenerationForm.ts`
- [ ] Added description in `ModelManagement.tsx`
### Phase 4: Dependencies
- [ ] Added packages to `backend/requirements.txt`
- [ ] If `--no-deps` needed: listed sub-dependencies explicitly
- [ ] If git-only packages: added `@ git+https://...` entries
- [ ] If custom index needed: added `--find-links` line
- [ ] Updated `justfile` setup targets
- [ ] Updated `.github/workflows/release.yml` build steps
- [ ] Updated `Dockerfile` if applicable
- [ ] Verified `pip install` succeeds in a clean venv with existing requirements
### Phase 5: PyInstaller Bundling
- [ ] Added `--hidden-import` entries in `build_binary.py` for:
- [ ] `backend.backends.<engine>_backend`
- [ ] The model package and its key submodules
- [ ] Added `--collect-all` for any packages that:
- [ ] Use `inspect.getsource()` / `@typechecked`
- [ ] Ship pretrained model data files (`.pth.tar`, `.yaml`, etc.)
- [ ] Ship native data files (phoneme tables, shader libraries, etc.)
- [ ] Added `--copy-metadata` for any packages that use `importlib.metadata`
- [ ] If engine has native data paths: added `os.environ.setdefault()` in `server.py`
- [ ] Built frozen binary with `just build`
- [ ] Tested in frozen binary with **clean model cache** (not pre-cached from dev):
- [ ] Model download works with real-time progress
- [ ] Model loading works
- [ ] Generation produces valid audio
- [ ] No errors in stderr logs
### Phase 6: Final Verification
- [ ] Engine works in dev mode (`just dev`)
- [ ] Engine works in frozen binary (`just build` → run binary directly)
- [ ] Tested on target platform (macOS for MLX, Windows/Linux for CUDA)
- [ ] No regressions in existing engines
+2 -2
View File
@@ -3,12 +3,12 @@ title: "Voicebox Documentation"
description: "Voicebox is a local-first voice cloning studio -- a free and open-source alternative to ElevenLabs."
---
Voicebox is a **local-first voice cloning studio** -- a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
Voicebox is a **local-first voice cloning studio** -- a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 5 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
![Voicebox App Screenshot](/images/app-screenshot-1.webp)
- **Complete privacy** -- models and voice data stay on your machine
- **4 TTS engines** -- Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo
- **5 TTS engines** -- Qwen3-TTS, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo, and HumeAI TADA
- **23 languages** -- from English to Arabic, Japanese, Hindi, Swahili, and more
- **Post-processing effects** -- pitch shift, reverb, delay, chorus, compression, and filters
- **Expressive speech** -- paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo
+5 -4
View File
@@ -5,10 +5,10 @@ description: "Voicebox is a local-first voice cloning studio -- a free and open-
## What is Voicebox?
Voicebox is a **local-first voice cloning studio** -- a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
Voicebox is a **local-first voice cloning studio** -- a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 5 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
- **Complete privacy** -- models and voice data stay on your machine
- **4 TTS engines** -- Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo
- **5 TTS engines** -- Qwen3-TTS, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo, and HumeAI TADA
- **23 languages** -- from English to Arabic, Japanese, Hindi, Swahili, and more
- **Post-processing effects** -- pitch shift, reverb, delay, chorus, compression, and filters
- **Expressive speech** -- paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo
@@ -20,7 +20,7 @@ Voicebox is a **local-first voice cloning studio** -- a free and open-source alt
## TTS Engines
Four engines with different strengths, switchable per-generation:
Five engines with different strengths, switchable per-generation:
| Engine | Languages | Strengths |
|--------|-----------|-----------|
@@ -28,6 +28,7 @@ Four engines with different strengths, switchable per-generation:
| **LuxTTS** | English | Lightweight (~1GB VRAM), 48kHz output, 150x realtime on CPU |
| **Chatterbox Multilingual** | 23 | Broadest language coverage |
| **Chatterbox Turbo** | English | Fast 350M model with paralinguistic emotion/sound tags |
| **TADA** (1B / 3B) | 10 | HumeAI speech-language model -- 700s+ coherent audio |
## GPU Support
@@ -56,7 +57,7 @@ Four engines with different strengths, switchable per-generation:
| Frontend | React, TypeScript, Tailwind CSS |
| State | Zustand, React Query |
| Backend | FastAPI (Python) |
| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo |
| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo, TADA |
| Effects | Pedalboard (Spotify) |
| Transcription | Whisper / Whisper Turbo (PyTorch or MLX) |
| Inference | MLX (Apple Silicon) / PyTorch (CUDA/ROCm/XPU/CPU) |
+13 -5
View File
@@ -36,6 +36,10 @@
│ │ │ │ Qwen3-TTS│ │LuxTTS │ │Chatterbox │ │ │ │
│ │ │ │(Py/MLX) │ │ │ │(MTL+Turbo)│ │ │ │
│ │ │ └──────────┘ └───────┘ └───────────┘ │ │ │
│ │ │ ┌──────────┐ │ │ │
│ │ │ │ TADA │ │ │ │
│ │ │ │(1B / 3B) │ │ │ │
│ │ │ └──────────┘ │ │ │
│ │ └─────────────────────────────────────────┘ │ │
│ │ ┌───────────┐ ┌─────────┐ │ │
│ │ │ STTBackend│ │ Profiles│ │ │
@@ -59,6 +63,7 @@
| LuxTTS | `backend/backends/luxtts_backend.py` | LuxTTS — fast, CPU-friendly |
| Chatterbox MTL | `backend/backends/chatterbox_backend.py` | Chatterbox Multilingual — 23 languages |
| Chatterbox Turbo | `backend/backends/chatterbox_turbo_backend.py` | Chatterbox Turbo — English, paralinguistic tags |
| TADA | `backend/backends/hume_backend.py` | HumeAI TADA — 1B English + 3B Multilingual |
| Platform detect | `backend/platform_detect.py` | Apple Silicon → MLX, else → PyTorch |
| API types | `backend/models.py` | Pydantic request/response models |
| HF progress | `backend/utils/hf_progress.py` | HFProgressTracker (tqdm patching for download progress) |
@@ -78,7 +83,7 @@
```
POST /generate
1. Look up voice profile from DB
2. Resolve engine from request (qwen | luxtts | chatterbox | chatterbox_turbo)
2. Resolve engine from request (qwen | luxtts | chatterbox | chatterbox_turbo | tada)
3. Get backend: get_tts_backend_for_engine(engine) # thread-safe singleton per engine
4. Check model cache → if missing, trigger background download, return HTTP 202
5. Load model (lazy): tts_backend.load_model(model_size)
@@ -104,7 +109,8 @@ POST /generate
- LuxTTS integration — fast, CPU-friendly English TTS (PR #254)
- Chatterbox Multilingual TTS — 23 languages including Hebrew (PR #257)
- Instruct parameter UI exists but is non-functional across all backends (see #224, Known Limitations)
- Single flat model dropdown (Qwen 1.7B, Qwen 0.6B, LuxTTS, Chatterbox, Chatterbox Turbo)
- HumeAI TADA integration — 1B English + 3B Multilingual speech-language model (PR #296)
- Single flat model dropdown (Qwen 1.7B, Qwen 0.6B, LuxTTS, Chatterbox, Chatterbox Turbo, TADA 1B, TADA 3B)
- Centralized model config registry (`ModelConfig` dataclass) — no per-engine dispatch maps in `main.py`
- Shared `EngineModelSelector` component — engine/model dropdown defined once, used in both generation forms
@@ -136,6 +142,8 @@ POST /generate
| LuxTTS | `luxtts` | English | ~300 MB | CPU-friendly, 48 kHz, fast | None |
| Chatterbox | `chatterbox-tts` | 23 (incl. Hebrew, Arabic, Hindi, etc.) | ~3.2 GB | Zero-shot cloning, multilingual | Partial — `exaggeration` float (0-1) for expressiveness |
| Chatterbox Turbo | `chatterbox-turbo` | English | ~1.5 GB | Paralinguistic tags ([laugh], [cough]), 350M params, low latency | Partial — inline tags only, no separate instruct param |
| TADA 1B | `tada-1b` | English | ~4 GB | HumeAI speech-language model, 700s+ coherent audio | None |
| TADA 3B Multilingual | `tada-3b-ml` | 10 (en, ar, zh, de, es, fr, it, ja, pl, pt) | ~8 GB | Multilingual, text-acoustic dual alignment | None |
### Multi-Engine Architecture (Shipped)
@@ -143,7 +151,7 @@ The singleton TTS backend blocker described in the previous version of this doc
- **Thread-safe backend registry** (`_tts_backends` dict + `_tts_backends_lock`) with double-checked locking
- **Per-engine backend instances** — each engine gets its own singleton, loaded lazily
- **Engine field on GenerationRequest** — frontend sends `engine: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo'`
- **Engine field on GenerationRequest** — frontend sends `engine: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo' | 'tada'`
- **Per-engine language filtering** — `ENGINE_LANGUAGES` map in frontend, backend regex accepts all languages
- **Per-engine voice prompts** — `create_voice_prompt_for_profile()` dispatches to the correct backend
- **Trim post-processing** — `trim_tts_output()` for Chatterbox engines (cuts trailing silence/hallucination)
@@ -337,7 +345,7 @@ Notable requests:
| **CosyVoice2-0.5B** | 3-10s zero-shot | Very fast | 24 kHz | Multilingual | Low | **Yes** — `inference_instruct2()`, works with cloning | Ready | Best instruct candidate |
| **Fish Speech** | 10-30s few-shot | Real-time | 24-44 kHz | 50+ | Medium | **Yes** — inline text descriptions, word-level control | Ready | Multi-engine arch in place |
| **MOSS-TTS Family** | Zero-shot | — | — | Multilingual | Medium | **Yes** — text prompts for style + timbre design | Needs vetting | Apache 2.0, multi-speaker dialogue |
| **HumeAI TADA 1B/3B** | Zero-shot | 5× faster than LLM-TTS | — | EN (1B), Multilingual (3B) | Medium | Partial — automatic prosody from text context | Needs vetting | MIT, 700s+ coherent, synced transcript output |
| **HumeAI TADA 1B/3B** | Zero-shot | 5× faster than LLM-TTS | 24 kHz | EN (1B), Multilingual (3B) | Medium | Partial — automatic prosody from text context | **Shipped** | PR #296, MIT, 700s+ coherent |
| **VoxCPM 1.5** | Zero-shot (seconds) | ~0.15 RTF streaming | — | Bilingual (EN/ZH) | Medium | Partial — automatic context-aware prosody | Needs vetting | Apache 2.0, tokenizer-free continuous diffusion |
| **Kokoro-82M** | 3s instant | CPU realtime | 24 kHz | English | Tiny (82M) | Partial — automatic style inference | Ready | Apache 2.0, multi-engine arch in place |
| **XTTS-v2** | 6s zero-shot | Mid-GPU | 24 kHz | 17+ | Medium | Partial — style transfer from ref audio only | Ready | Multi-engine arch in place |
@@ -475,7 +483,7 @@ The generation form now uses a flat model dropdown with engine-based routing. Pe
| `/history/{id}/export` | GET | Export generation ZIP |
| `/history/{id}/export-audio` | GET | Export audio only |
| `/transcribe` | POST | Transcribe audio (Whisper) |
| `/models/status` | GET | All model statuses (Qwen, LuxTTS, Chatterbox, Chatterbox Turbo, Whisper) |
| `/models/status` | GET | All model statuses (Qwen, LuxTTS, Chatterbox, Chatterbox Turbo, TADA, Whisper) |
| `/models/download` | POST | Trigger model download |
| `/models/download/cancel` | POST | Cancel/dismiss download |
| `/models/{name}` | DELETE | Delete downloaded model |
+3
View File
@@ -46,6 +46,8 @@ setup-python:
{{ pip }} install -r {{ backend_dir }}/requirements.txt
# Chatterbox pins numpy<1.26 / torch==2.6 which break on Python 3.12+
{{ pip }} install --no-deps chatterbox-tts
# HumeAI TADA pins torch>=2.7,<2.8 which conflicts with our torch>=2.1
{{ pip }} install --no-deps hume-tada
# Apple Silicon: install MLX backend
if [ "$(uname -m)" = "arm64" ] && [ "$(uname)" = "Darwin" ]; then
echo "Detected Apple Silicon — installing MLX dependencies..."
@@ -74,6 +76,7 @@ setup-python:
}
& "{{ pip }}" install -r {{ backend_dir }}/requirements.txt
& "{{ pip }}" install --no-deps chatterbox-tts
& "{{ pip }}" install --no-deps hume-tada
& "{{ pip }}" install git+https://github.com/QwenLM/Qwen3-TTS.git
& "{{ pip }}" install pyinstaller ruff pytest pytest-asyncio -q
Write-Host "Python environment ready."