Integrates HumeAI's TADA (Text-Acoustic Dual Alignment) speech-language
model as a new TTS engine. TADA uses a novel 1:1 token-audio alignment
that produces coherent speech over long sequences (700s+).
Two model variants:
- tada-1b: English-only, ~4GB, built on Llama 3.2 1B
- tada-3b-ml: 10 languages, ~8GB, built on Llama 3.2 3B
Backend uses the Encoder for voice prompt encoding with caching, and
TadaForCausalLM with flow-matching diffusion for generation. Supports
bf16 inference on CUDA, forces CPU on macOS (MPS compatibility).
Installed with --no-deps due to torch>=2.7 pin conflict; descript-audio-codec
and torchaudio added as explicit sub-dependencies.
- New ChatterboxTurboTTSBackend wrapping ChatterboxTurboTTS (ResembleAI/chatterbox-turbo)
- English-only 350M model with paralinguistic tag support ([laugh], [cough], [chuckle])
- Bypasses upstream token=True bug by calling snapshot_download(token=None) + from_local()
- Same CPU-on-macOS forcing and torch.load monkey-patching as multilingual backend
- Full engine integration: generate, stream, model status/download/delete endpoints
- Language dropdown now shows only languages supported by the selected engine
- Per-engine language maps: Qwen (10), LuxTTS (en), Chatterbox (23), Turbo (en)
- Auto-switches to English when selecting English-only engines
- Backend language regex expanded to accept all 23 Chatterbox languages
- New ChatterboxTTSBackend wrapping ChatterboxMultilingualTTS (ResembleAI/chatterbox)
- Supports 23 languages including Hebrew, forces CPU on macOS (MPS issue)
- Monkey-patches torch.load for CPU loading, forces eager attention for compatibility
- trim_tts_output utility cuts trailing silence/hallucination from Chatterbox output
- Full engine integration: /generate, /generate/stream, model status/download/delete
- Hebrew (he) added to supported languages in frontend and backend validation
- Single flat model dropdown extended with Chatterbox option in both generation UIs
- ModelManagement UI groups LuxTTS and Chatterbox under 'Other Voice Models' section
- Add threading lock to get_tts_backend_for_engine() to prevent race
condition where concurrent requests could create duplicate backend
instances (double-checked locking pattern)
- Fix LuxTTS generate: call .detach().cpu() before .numpy() so it
works on GPU/MPS devices, not just CPU
- Store background download tasks in a module-level set to prevent
garbage collection before completion (asyncio.create_task fire-and-
forget pattern)
- Deduplicate cache_key computation in LuxTTS create_voice_prompt
- Prefix unused sr variable with underscore
Introduce LuxTTS (ZipVoice) alongside Qwen TTS, enabling users to choose
between engines at generation time. LuxTTS offers fast, English-focused
voice cloning at 48kHz with ~1GB VRAM.
Backend:
- Add LuxTTSBackend with encode_prompt/generate_speech integration
- Multi-engine registry (get_tts_backend_for_engine) replacing singleton
- Engine-prefixed voice prompt cache keys to avoid collisions
- Engine field on GenerationRequest (default 'qwen' for backward compat)
- Engine dispatch in /generate and /generate/stream endpoints
- LuxTTS in model status, download, and delete maps
Frontend:
- TTS Engine selector dropdown in GenerationForm (Qwen TTS / LuxTTS)
- Conditionally hide Model Size and Delivery Instructions for LuxTTS
- Engine field added to TypeScript types and Zod schema
- LuxTTS section in Model Management page
- Added support for MLX backend on Apple Silicon, enabling optimized performance for TTS and STT tasks.
- Implemented platform detection to dynamically select between MLX and PyTorch based on the runtime environment.
- Updated build process to include MLX-specific dependencies and configurations for macOS.
- Refactored backend code to improve model loading and inference logic, accommodating backend-specific requirements.
- Enhanced documentation to clarify backend selection and performance benefits for different platforms.
- Streamlined installation instructions and troubleshooting guidance for MLX-related issues.
- Added support for MLX backend on Apple Silicon, enabling optimized performance for TTS and STT tasks.
- Updated release workflow to include MLX-specific dependencies and configurations for macOS platforms.
- Refactored backend code to dynamically select between MLX and PyTorch based on the runtime environment.
- Enhanced model loading and inference logic to accommodate backend-specific requirements, including updated model IDs and hidden imports.
- Improved health check and model status reporting to reflect the active backend type.
- Streamlined caching mechanisms to support both backend types, ensuring compatibility and performance.
James Pine