Labyricorn/voicebox
1
0
Fork 0
mirror of https://github.com/jamiepine/voicebox.git synced 2026-09-26 13:45:16 -07:00
Code Issues Packages Projects Releases Wiki Activity

Refactor TTS provider management and enhance documentation

Browse Source 
- Renamed `bundled-mlx` to `apple-mlx` for clarity in provider types.
- Updated the ProviderSettings component to reflect the new provider naming.
- Improved logging for provider startup and error handling in the backend.
- Added scripts for building and installing PyTorch CPU and CUDA providers locally.
- Enhanced the documentation to include details on TTS provider architecture and development setup.
This commit is contained in:
Jamie Pine
2026-02-01 01:23:20 -08:00
parent 3b14f81741
commit 580179eba3
15 changed files with 526 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+46 -25
View File
@@ -78,12 +78,12 @@ Download a voice model, clone any voice from a few seconds of audio, and compose
Voicebox is available now for macOS and Windows.
| Platform | Download |
|----------|----------|
| macOS (Apple Silicon) | [voicebox_aarch64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_aarch64.app.tar.gz) |
| macOS (Intel) | [voicebox_x64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_x64.app.tar.gz) |
| Windows (MSI) | [voicebox_0.1.0_x64_en-US.msi](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_0.1.0_x64_en-US.msi) |
| Windows (Setup) | [voicebox_0.1.0_x64-setup.exe](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_0.1.0_x64-setup.exe) |
| Platform | Download |
| --------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| macOS (Apple Silicon) | [voicebox_aarch64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_aarch64.app.tar.gz) |
| macOS (Intel) | [voicebox_x64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_x64.app.tar.gz) |
| Windows (MSI) | [voicebox_0.1.0_x64_en-US.msi](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_0.1.0_x64_en-US.msi) |
| Windows (Setup) | [voicebox_0.1.0_x64-setup.exe](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_0.1.0_x64-setup.exe) |
> **Linux builds coming soon** — Currently blocked by GitHub runner disk space limitations.
@@ -176,17 +176,17 @@ Full API documentation available at `http://localhost:8000/docs` when running.
## Tech Stack
| Layer | Technology |
|-------|------------|
| Desktop App | Tauri (Rust) |
| Frontend | React, TypeScript, Tailwind CSS |
| State | Zustand, React Query |
| Backend | FastAPI (Python) |
| Voice Model | Qwen3-TTS (PyTorch or MLX) |
| Transcription | Whisper (PyTorch or MLX) |
| Layer | Technology |
| ---------------- | --------------------------------------------------- |
| Desktop App | Tauri (Rust) |
| Frontend | React, TypeScript, Tailwind CSS |
| State | Zustand, React Query |
| Backend | FastAPI (Python) |
| Voice Model | Qwen3-TTS (PyTorch or MLX) |
| Transcription | Whisper (PyTorch or MLX) |
| Inference Engine | MLX (Apple Silicon) / PyTorch (Windows/Linux/Intel) |
| Database | SQLite |
| Audio | WaveSurfer.js, librosa |
| Database | SQLite |
| Audio | WaveSurfer.js, librosa |
**Why this stack?**
@@ -194,6 +194,26 @@ Full API documentation available at `http://localhost:8000/docs` when running.
- **FastAPI** — Async Python with automatic OpenAPI schema generation
- **Type-safe end-to-end** — Generated TypeScript client from OpenAPI spec
### TTS Provider Architecture
Voicebox uses a modular provider system to support different inference backends:
- **`apple-mlx`** — Bundled with macOS Apple Silicon builds
- Uses MLX with native Metal acceleration (4-5x faster)
- Works out of the box, no download required
- **`pytorch-cpu`** — Universal CPU provider (bundled or downloaded)
- Bundled with macOS Intel builds
- Downloaded on first use for Windows/Linux (~300MB)
- **`pytorch-cuda`** — Optional NVIDIA GPU-accelerated provider
- Windows/Linux only (~2.4GB)
- 4-5x faster inference on CUDA-capable GPUs
macOS builds work out of the box with bundled providers. Windows and Linux users download a provider on first launch. The app automatically detects your hardware and recommends the best option. All downloadable providers are distributed via Cloudflare R2 for fast, global delivery.
---
## Roadmap
@@ -202,13 +222,13 @@ Voicebox is the beginning of something bigger. Here's what's coming:
### Coming Soon
| Feature | Description |
|---------|-------------|
| **Real-time Synthesis** | Stream audio as it generates, word by word |
| **Conversation Mode** | Multi-speaker dialogues with automatic turn-taking |
| **Voice Effects** | Pitch shift, reverb, M3GAN-style effects |
| **Timeline Editor** | Audio studio with word-level precision editing |
| **More Models** | XTTS, Bark, and other open-source voice models |
| Feature | Description |
| ----------------------- | -------------------------------------------------- |
| **Real-time Synthesis** | Stream audio as it generates, word by word |
| **Conversation Mode** | Multi-speaker dialogues with automatic turn-taking |
| **Voice Effects** | Pitch shift, reverb, M3GAN-style effects |
| **Timeline Editor** | Audio studio with word-level precision editing |
| **More Models** | XTTS, Bark, and other open-source voice models |
### Future Vision
@@ -260,9 +280,10 @@ cd backend && pip install -r requirements.txt && cd ..
bun run dev
```
**Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org).
**Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org).
**Performance:**
**Performance:**
- **Apple Silicon (M1/M2/M3)**: Uses MLX backend with native Metal acceleration for 4-5x faster inference
- **Windows/Linux/Intel Mac**: Uses PyTorch backend (CUDA GPU recommended, CPU supported but slower)