mirror of
https://github.com/jamiepine/voicebox.git
synced 2026-09-15 21:00:42 -07:00
c8cb12f1bc
* fix(build): bundle kokoro source files for transformers runtime introspection transformers opens .py source files at runtime to check attention/MoE implementation via regex (e.g. _can_set_attn_implementation). PyInstaller's --hidden-import only bundles .pyc bytecode, so kokoro/modules.py was missing from the bundle causing a FileNotFoundError on Kokoro model load. Switch from individual --hidden-import entries to --collect-all kokoro in both build_binary.py and voicebox-server.spec. The kokoro package is 172K so no meaningful bundle size impact. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(build): use SPECPATH for runtime hook instead of hardcoded absolute path The linter expanded runtime_hooks=[] to an absolute /Users/... path which would break CI and other dev machines. Use os.path.join(SPECPATH, ...) to mirror the relative approach in build_binary.py. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(build): runtime hook to work around PyInstaller + Python 3.12 import breakages Four distinct bundling-specific crashes blocked Kokoro and Qwen CustomVoice from loading in the frozen binary: 1. torch._dynamo import triggered via class-body decorators (@torch._dynamo.allow_in_graph on PreTrainedModel, @torch.compiler.disable in flex_attention) pulls in torch._numpy._ufuncs which crashes on module load with NameError: name 'name' is not defined. 2. AlbertModel (Kokoro) triggers @auto_docstring -> modeling_auto -> GenerationMixin -> candidate_generator -> sklearn -> scipy, which hits the same class of bug in scipy.stats._distn_infrastructure (NameError: name 'obj' is not defined). 3. AutoModel (Qwen) pulls the same sklearn -> scipy chain directly. 4. librosa (required by most TTS engines) -> scipy.signal -> scipy.stats hits the _distn_infrastructure crash regardless of the transformers stubs above. The root cause of (1) and (4) is that PyInstaller's frozen importer runs module-level `for X in [<list-comp using dir()>]:` loops with an empty iterable, leaving the loop variable unbound. Trailing `del obj` / unrelated references then crash. Fix: a single runtime hook (pyi_rth_torch_compiler_disable.py) installs: - sys.modules stubs for torch._dynamo and torch._dynamo.config, plus a meta-path finder for torch._dynamo.* submodules — voicebox never uses torch.compile/dynamo for inference, so a permissive no-op stub (callable as decorator, falsey as predicate, context-manager-safe for TransformGetItemToIndex) is drop-in safe. - meta-path finder stubs for transformers.utils.auto_docstring and transformers.generation.candidate_generator — both import-chain short-circuits; docstrings and speculative decoding aren't used for TTS. - meta-path finder for scipy.stats._distn_infrastructure that reads the real .py source via the wrapped loader's get_source(), replaces the bundling-broken `del obj` with `globals().pop('obj', None)`, and compile+exec's the patched source. This keeps the real scipy module intact so librosa and everything downstream works normally. Supporting changes: - backend/pyi_hooks/hook-scipy.stats._distn_infrastructure.py sets module_collection_mode = "pyz+py" so the .py source is actually in the bundle for the runtime patcher to read. - build_binary.py and voicebox-server.spec register the runtime hook and the new hooks dir. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(build): force transformers torch<2.6 mask path and bundle spacy_pkuseg - patch transformers.masking_utils to set _is_torch_greater_or_equal_than_2_6 = False, forcing sdpa_mask_older_torch and avoiding the vmap .item() crash that breaks Qwen CustomVoice generation (our torch._dynamo stub can't reproduce TransformGetItemToIndex's graph transform). - add PyInstaller hook to bundle transformers.masking_utils .py source so the runtime finder can source-patch it. - --collect-all spacy_pkuseg so Chatterbox Multilingual can load its Chinese segmenter (dicts/default.pkl + native .so extensions). - add per-finder install diagnostics + _HOOK_VERSION marker to make future bundle-only regressions easier to triage. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(build): pass PyInstaller hook paths relative so .spec is portable Absolute paths ended up in the auto-regenerated voicebox-server.spec because build_binary.py prefixed every --runtime-hook and --additional-hooks-dir with str(backend_dir / ...). That broke builds on any machine whose checkout wasn't at /Users/jamie/... and anyone invoking pyinstaller voicebox-server.spec directly. os.chdir(backend_dir) already runs before PyInstaller (same reason server.py works as a bare filename), so the backend_dir prefix is unnecessary. Drop it so the generated spec references pyi_hooks/, pyi_rth_numpy_compat.py, pyi_rth_torch_compiler_disable.py as repo- relative paths. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Voicebox
The open-source voice synthesis studio.
Clone voices. Generate speech. Apply effects. Build voice-powered apps.
All running locally on your machine.
voicebox.sh • Docs • Download • Features • API
Click the image above to watch the demo video on voicebox.sh
What is Voicebox?
Voicebox is a local-first voice cloning studio — a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 5 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
- Complete privacy — models and voice data stay on your machine
- 5 TTS engines — Qwen3-TTS, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo, and HumeAI TADA
- 23 languages — from English to Arabic, Japanese, Hindi, Swahili, and more
- Post-processing effects — pitch shift, reverb, delay, chorus, compression, and filters
- Expressive speech — paralinguistic tags like
[laugh],[sigh],[gasp]via Chatterbox Turbo - Unlimited length — auto-chunking with crossfade for scripts, articles, and chapters
- Stories editor — multi-track timeline for conversations, podcasts, and narratives
- API-first — REST API for integrating voice synthesis into your own projects
- Native performance — built with Tauri (Rust), not Electron
- Runs everywhere — macOS (MLX/Metal), Windows (CUDA), Linux, AMD ROCm, Intel Arc, Docker
Download
| Platform | Download |
|---|---|
| macOS (Apple Silicon) | Download DMG |
| macOS (Intel) | Download DMG |
| Windows | Download MSI |
| Docker | docker compose up |
Linux — Pre-built binaries are not yet available. See voicebox.sh/linux-install for build-from-source instructions.
Features
Multi-Engine Voice Cloning
Five TTS engines with different strengths, switchable per-generation:
| Engine | Languages | Strengths |
|---|---|---|
| Qwen3-TTS (0.6B / 1.7B) | 10 | High-quality multilingual cloning, delivery instructions ("speak slowly", "whisper") |
| LuxTTS | English | Lightweight (~1GB VRAM), 48kHz output, 150x realtime on CPU |
| Chatterbox Multilingual | 23 | Broadest language coverage — Arabic, Danish, Finnish, Greek, Hebrew, Hindi, Malay, Norwegian, Polish, Swahili, Swedish, Turkish and more |
| Chatterbox Turbo | English | Fast 350M model with paralinguistic emotion/sound tags |
| TADA (1B / 3B) | 10 | HumeAI speech-language model — 700s+ coherent audio, text-acoustic dual alignment |
Emotions & Paralinguistic Tags
Type / in the text input to insert expressive tags that the model synthesizes inline with speech (Chatterbox Turbo):
[laugh] [chuckle] [gasp] [cough] [sigh] [groan] [sniff] [shush] [clear throat]
Post-Processing Effects
8 audio effects powered by Spotify's pedalboard library. Apply after generation, preview in real time, build reusable presets.
| Effect | Description |
|---|---|
| Pitch Shift | Up or down by up to 12 semitones |
| Reverb | Configurable room size, damping, wet/dry mix |
| Delay | Echo with adjustable time, feedback, and mix |
| Chorus / Flanger | Modulated delay for metallic or lush textures |
| Compressor | Dynamic range compression |
| Gain | Volume adjustment (-40 to +40 dB) |
| High-Pass Filter | Remove low frequencies |
| Low-Pass Filter | Remove high frequencies |
Ships with 4 built-in presets (Robotic, Radio, Echo Chamber, Deep Voice) and supports custom presets. Effects can be assigned per-profile as defaults.
Unlimited Generation Length
Text is automatically split at sentence boundaries and each chunk is generated independently, then crossfaded together. Works with all engines.
- Configurable auto-chunking limit (100–5,000 chars)
- Crossfade slider (0–200ms) for smooth transitions
- Max text length: 50,000 characters
- Smart splitting respects abbreviations, CJK punctuation, and
[tags]
Generation Versions
Every generation supports multiple versions with provenance tracking:
- Original — clean TTS output, always preserved
- Effects versions — apply different effects chains from any source version
- Takes — regenerate with a new seed for variation
- Source tracking — each version records its lineage
- Favorites — star generations for quick access
Async Generation Queue
Generation is non-blocking. Submit and immediately start typing the next one.
- Serial execution queue prevents GPU contention
- Real-time SSE status streaming
- Failed generations can be retried
- Stale generations from crashes auto-recover on startup
Voice Profile Management
- Create profiles from audio files or record directly in-app
- Import/export profiles to share or back up
- Multi-sample support for higher quality cloning
- Per-profile default effects chains
- Organize with descriptions and language tags
Stories Editor
Multi-voice timeline editor for conversations, podcasts, and narratives.
- Multi-track composition with drag-and-drop
- Inline audio trimming and splitting
- Auto-playback with synchronized playhead
- Version pinning per track clip
Recording & Transcription
- In-app recording with waveform visualization
- System audio capture (macOS and Windows)
- Automatic transcription powered by Whisper (including Whisper Turbo)
- Export recordings in multiple formats
Model Management
- Per-model unload to free GPU memory without deleting downloads
- Custom models directory via
VOICEBOX_MODELS_DIR - Model folder migration with progress tracking
- Download cancel/clear UI
GPU Support
| Platform | Backend | Notes |
|---|---|---|
| macOS (Apple Silicon) | MLX (Metal) | 4-5x faster via Neural Engine |
| Windows / Linux (NVIDIA) | PyTorch (CUDA) | Auto-downloads CUDA binary from within the app |
| Linux (AMD) | PyTorch (ROCm) | Auto-configures HSA_OVERRIDE_GFX_VERSION |
| Windows (any GPU) | DirectML | Universal Windows GPU support |
| Intel Arc | IPEX/XPU | Intel discrete GPU acceleration |
| Any | CPU | Works everywhere, just slower |
API
Voicebox exposes a full REST API for integrating voice synthesis into your own apps.
# Generate speech
curl -X POST http://localhost:17493/generate \
-H "Content-Type: application/json" \
-d '{"text": "Hello world", "profile_id": "abc123", "language": "en"}'
# List voice profiles
curl http://localhost:17493/profiles
# Create a profile
curl -X POST http://localhost:17493/profiles \
-H "Content-Type: application/json" \
-d '{"name": "My Voice", "language": "en"}'
Use cases: game dialogue, podcast production, accessibility tools, voice assistants, content automation.
Full API documentation available at http://localhost:17493/docs.
Tech Stack
| Layer | Technology |
|---|---|
| Desktop App | Tauri (Rust) |
| Frontend | React, TypeScript, Tailwind CSS |
| State | Zustand, React Query |
| Backend | FastAPI (Python) |
| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo, TADA |
| Effects | Pedalboard (Spotify) |
| Transcription | Whisper / Whisper Turbo (PyTorch or MLX) |
| Inference | MLX (Apple Silicon) / PyTorch (CUDA/ROCm/XPU/CPU) |
| Database | SQLite |
| Audio | WaveSurfer.js, librosa |
Roadmap
| Feature | Description |
|---|---|
| Real-time Streaming | Stream audio as it generates, word by word |
| Voice Design | Create new voices from text descriptions |
| More Models | XTTS, Bark, and other open-source voice models |
| Plugin Architecture | Extend with custom models and effects |
| Mobile Companion | Control Voicebox from your phone |
Development
See CONTRIBUTING.md for detailed setup and contribution guidelines.
Quick Start
git clone https://github.com/jamiepine/voicebox.git
cd voicebox
just setup # creates Python venv, installs all deps
just dev # starts backend + desktop app
Install just: brew install just or cargo install just. Run just --list to see all commands.
Prerequisites: Bun, Rust, Python 3.11+, Tauri Prerequisites, and Xcode on macOS.
Building Locally
just build # Build CPU server binary + Tauri app
just build-local # (Windows) Build CPU + CUDA server binaries + Tauri app
Adding New Voice Models
The multi-engine architecture makes adding new TTS engines straightforward. A step-by-step guide covers the full process: dependency research, backend protocol implementation, frontend wiring, and PyInstaller bundling.
The guide is optimized for AI coding agents. An agent skill can pick up a model name and handle the entire integration autonomously — you just test the build locally.
Project Structure
voicebox/
├── app/ # Shared React frontend
├── tauri/ # Desktop app (Tauri + Rust)
├── web/ # Web deployment
├── backend/ # Python FastAPI server
├── landing/ # Marketing website
└── scripts/ # Build & release scripts
Contributing
Contributions welcome! See CONTRIBUTING.md for guidelines.
- Fork the repo
- Create a feature branch
- Make your changes
- Submit a PR
Security
Found a security vulnerability? Please report it responsibly. See SECURITY.md for details.
License
MIT License — see LICENSE for details.