Labyricorn/voicebox
1
0
Fork 0
mirror of https://github.com/jamiepine/voicebox.git synced 2026-10-02 16:45:15 -07:00
Code Issues Packages Projects Releases Wiki Activity

Refactor TTS provider management and enhance documentation

Browse Source 
- Renamed `bundled-mlx` to `apple-mlx` for clarity in provider types.
- Updated the ProviderSettings component to reflect the new provider naming.
- Improved logging for provider startup and error handling in the backend.
- Added scripts for building and installing PyTorch CPU and CUDA providers locally.
- Enhanced the documentation to include details on TTS provider architecture and development setup.
This commit is contained in:
Jamie Pine
2026-02-01 01:23:20 -08:00
parent 3b14f81741
commit 580179eba3
15 changed files with 526 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CONTRIBUTING.md
+149 -9
View File
@@ -14,16 +14,19 @@ Thank you for your interest in contributing to Voicebox! This document provides
### Prerequisites ### Prerequisites
- **[Bun](https://bun.sh)** - Fast JavaScript runtime and package manager - **[Bun](https://bun.sh)** - Fast JavaScript runtime and package manager
```bash ```bash
curl -fsSL https://bun.sh/install | bash curl -fsSL https://bun.sh/install | bash
``` ```
- **[Python 3.11+](https://python.org)** - For backend development - **[Python 3.11+](https://python.org)** - For backend development
```bash ```bash
python --version # Should be 3.11 or higher python --version # Should be 3.11 or higher
``` ```
- **[Rust](https://rustup.rs)** - For Tauri desktop app (installed automatically by Tauri CLI) - **[Rust](https://rustup.rs)** - For Tauri desktop app (installed automatically by Tauri CLI)
```bash ```bash
rustc --version # Check if installed rustc --version # Check if installed
``` ```
@@ -37,41 +40,46 @@ Thank you for your interest in contributing to Voicebox! This document provides
**Manual setup (required for Windows):** **Manual setup (required for Windows):**
1. **Fork and clone the repository** 1. **Fork and clone the repository**
```bash ```bash
git clone https://github.com/YOUR_USERNAME/voicebox.git git clone https://github.com/YOUR_USERNAME/voicebox.git
cd voicebox cd voicebox
``` ```
2. **Install JavaScript dependencies** 2. **Install JavaScript dependencies**
```bash ```bash
bun install bun install
``` ```
This installs dependencies for: This installs dependencies for:
- `app/` - Shared React frontend - `app/` - Shared React frontend
- `tauri/` - Tauri desktop wrapper - `tauri/` - Tauri desktop wrapper
- `web/` - Web deployment wrapper - `web/` - Web deployment wrapper
3. **Set up Python backend** 3. **Set up Python backend**
```bash ```bash
cd backend cd backend
# Create virtual environment # Create virtual environment
python -m venv venv python -m venv venv
# Activate virtual environment # Activate virtual environment
source venv/bin/activate # On macOS/Linux source venv/bin/activate # On macOS/Linux
# or # or
venv\Scripts\activate # On Windows venv\Scripts\activate # On Windows
# Install Python dependencies # Install Python dependencies
pip install -r requirements.txt pip install -r requirements.txt
# Install MLX dependencies (Apple Silicon only - for faster inference) # Install MLX dependencies (Apple Silicon only - for faster inference)
# On Apple Silicon, this enables native Metal acceleration # On Apple Silicon, this enables native Metal acceleration
if [[ $(uname -m) == "arm64" ]]; then if [[ $(uname -m) == "arm64" ]]; then
pip install -r requirements-mlx.txt pip install -r requirements-mlx.txt
fi fi
# Install Qwen3-TTS (required for voice synthesis) # Install Qwen3-TTS (required for voice synthesis)
pip install git+https://github.com/QwenLM/Qwen3-TTS.git pip install git+https://github.com/QwenLM/Qwen3-TTS.git
``` ```
@@ -81,19 +89,24 @@ Thank you for your interest in contributing to Voicebox! This document provides
Development requires two terminals: one for the Python backend, one for the Tauri app. Development requires two terminals: one for the Python backend, one for the Tauri app.
**Terminal 1: Backend server** (start this first) **Terminal 1: Backend server** (start this first)
```bash ```bash
cd backend cd backend
source venv/bin/activate # Activate venv if not already active source venv/bin/activate # Activate venv if not already active
bun run dev:server bun run dev:server
# Or manually: uvicorn main:app --reload --port 17493 # Or manually: uvicorn main:app --reload --port 17493
``` ```
Backend will be available at `http://localhost:17493` Backend will be available at `http://localhost:17493`
**Terminal 2: Desktop app** **Terminal 2: Desktop app**
```bash ```bash
bun run dev bun run dev
``` ```
This will: This will:
- Create a placeholder sidecar binary (for Tauri compilation) - Create a placeholder sidecar binary (for Tauri compilation)
- Start Vite dev server on port 5173 - Start Vite dev server on port 5173
- Launch Tauri window pointing to localhost:5173 - Launch Tauri window pointing to localhost:5173
@@ -104,26 +117,132 @@ Thank you for your interest in contributing to Voicebox! This document provides
> The bundled server binary is only used in production builds. > The bundled server binary is only used in production builds.
**Optional: Web app** **Optional: Web app**
```bash ```bash
bun run dev:web bun run dev:web
``` ```
Web app will be available at `http://localhost:5174` Web app will be available at `http://localhost:5174`
### Model Downloads ### Model Downloads
Models are automatically downloaded from HuggingFace Hub on first use: Models are automatically downloaded from HuggingFace Hub on first use:
- **Whisper** (transcription): Auto-downloads on first transcription - **Whisper** (transcription): Auto-downloads on first transcription
- **Qwen3-TTS** (voice cloning): Auto-downloads on first generation (~2-4GB) - **Qwen3-TTS** (voice cloning): Auto-downloads on first generation (~2-4GB)
First-time usage will be slower due to model downloads, but subsequent runs will use cached models. First-time usage will be slower due to model downloads, but subsequent runs will use cached models.
### TTS Provider Development
Voicebox uses a modular provider system to support different inference backends. Understanding this architecture is important when working on TTS features.
#### Provider Types
**Bundled Providers** — Included with the app binary:
- `apple-mlx` — Bundled with macOS Apple Silicon builds (`.dmg` for aarch64)
- Uses MLX for native Metal acceleration
- Configured in `.github/workflows/release.yml` with `backend: "mlx"`
**Hybrid Provider:**
- `pytorch-cpu` — Can be bundled OR downloaded depending on platform
- **Bundled** with macOS Intel builds (`.dmg` for x64)
- Configured in `.github/workflows/release.yml` with `backend: "pytorch"`
- **Downloaded** on first use for Windows/Linux builds (~300MB)
- Falls back to bundled version if external binary not found
**External-Only Providers:**
- `pytorch-cuda` — NVIDIA GPU-accelerated provider (~2.4GB)
- Windows/Linux only (no NVIDIA GPUs on macOS)
- Downloaded on demand, not bundled
- Optional for users with CUDA-capable GPUs
#### Provider Architecture
```
backend/providers/
├── __init__.py # ProviderManager - lifecycle management
├── base.py # TTSProvider protocol
├── bundled.py # BundledProvider - wraps built-in backends
├── local.py # LocalProvider - wraps external subprocess
├── installer.py # Download and install external providers
└── types.py # Provider type definitions
providers/
├── pytorch-cpu/ # External PyTorch CPU provider
│ ├── main.py # FastAPI server
│ ├── build.py # PyInstaller build script
│ └── build_and_install.py # Build and install locally
└── pytorch-cuda/ # External PyTorch CUDA provider
├── main.py
├── build.py
└── build_and_install.py
```
**How it works:**
1. **Bundled providers** run in-process within the main backend
2. **External providers** run as separate subprocess servers
3. **LocalProvider** communicates with external providers via HTTP
4. **ProviderManager** handles starting/stopping and health checks
#### Building Providers Locally
When developing provider features, you'll need to build and test external providers:
**Build a single provider:**
```bash
cd providers/pytorch-cpu
python build_and_install.py
```
**Build all providers:**
```bash
bun run build:providers
```
This script:
- Builds the provider binary with PyInstaller
- Detects your platform (Windows/macOS/Linux)
- Copies to the correct location:
- macOS: `~/Library/Application Support/voicebox/providers/`
- Windows: `%APPDATA%\voicebox\providers\`
- Linux: `~/.local/share/voicebox/providers/`
- Sets executable permissions on Unix
**Testing provider changes:**
1. Make changes to `providers/pytorch-cpu/main.py`
2. Run `bun run build:providers`
3. Restart the Voicebox app
4. Select the provider in Settings → TTS Provider
#### Provider Binary Distribution
For production releases, provider binaries are:
1. Built by GitHub Actions for all platforms
2. Uploaded to Cloudflare R2 at `downloads.voicebox.sh/providers/v{VERSION}/`
3. Downloaded on-demand by users based on their platform and GPU
See `.github/workflows/release.yml` for the build matrix.
### Building ### Building
**Build everything (recommended):** **Build everything (recommended):**
```bash ```bash
bun run build bun run build
``` ```
This automatically: This automatically:
1. Builds the Python server binary (`./scripts/build-server.sh`) 1. Builds the Python server binary (`./scripts/build-server.sh`)
2. Builds the Tauri desktop app (`cd tauri && bun run tauri build`) 2. Builds the Tauri desktop app (`cd tauri && bun run tauri build`)
@@ -132,13 +251,23 @@ Creates platform-specific installers (`.dmg`, `.msi`, `.AppImage`) in `tauri/src
**Note:** The build process detects your platform and includes the appropriate backend (MLX for Apple Silicon, PyTorch for others). **Note:** The build process detects your platform and includes the appropriate backend (MLX for Apple Silicon, PyTorch for others).
**Build server binary only:** **Build server binary only:**
```bash ```bash
bun run build:server bun run build:server
# or # or
./scripts/build-server.sh ./scripts/build-server.sh
``` ```
Creates platform-specific binary in `tauri/src-tauri/binaries/` Creates platform-specific binary in `tauri/src-tauri/binaries/`
**Build provider binaries (for development):**
```bash
bun run build:providers
```
Builds all external provider binaries and installs them to the system provider directory. See [TTS Provider Development](#tts-provider-development) for details.
**Building with local Qwen3-TTS development version:** **Building with local Qwen3-TTS development version:**
If you're actively developing or modifying the Qwen3-TTS library, set the `QWEN_TTS_PATH` environment variable to point to your local clone: If you're actively developing or modifying the Qwen3-TTS library, set the `QWEN_TTS_PATH` environment variable to point to your local clone:
@@ -151,34 +280,41 @@ bun run build:server
This makes PyInstaller use your local qwen-tts version instead of the pip-installed package. Useful when testing changes to the TTS library before they're published to PyPI or when using an editable install (`pip install -e`). This makes PyInstaller use your local qwen-tts version instead of the pip-installed package. Useful when testing changes to the TTS library before they're published to PyPI or when using an editable install (`pip install -e`).
**Build web app:** **Build web app:**
```bash ```bash
cd web cd web
bun run build bun run build
``` ```
Output in `web/dist/` Output in `web/dist/`
### Generate OpenAPI Client ### Generate OpenAPI Client
After starting the backend server: After starting the backend server:
```bash ```bash
./scripts/generate-api.sh ./scripts/generate-api.sh
``` ```
This downloads the OpenAPI schema and generates the TypeScript client in `app/src/lib/api/` This downloads the OpenAPI schema and generates the TypeScript client in `app/src/lib/api/`
### Convert Assets to Web Formats ### Convert Assets to Web Formats
To optimize images and videos for the web, run: To optimize images and videos for the web, run:
```bash ```bash
bun run convert:assets bun run convert:assets
``` ```
This script: This script:
- Converts PNG → WebP (better compression, same quality) - Converts PNG → WebP (better compression, same quality)
- Converts MOV → WebM (VP9 codec, smaller file size) - Converts MOV → WebM (VP9 codec, smaller file size)
- Processes files in `landing/public/` and `docs/public/` - Processes files in `landing/public/` and `docs/public/`
- **Deletes original files** after successful conversion - **Deletes original files** after successful conversion
**Requirements:** Install `webp` and `ffmpeg`: **Requirements:** Install `webp` and `ffmpeg`:
```bash ```bash
brew install webp ffmpeg brew install webp ffmpeg
``` ```
@@ -225,6 +361,7 @@ git push origin feature/your-feature-name
``` ```
Then create a pull request on GitHub with: Then create a pull request on GitHub with:
- Clear description of changes - Clear description of changes
- Screenshots (for UI changes) - Screenshots (for UI changes)
- Reference to related issues - Reference to related issues
@@ -370,21 +507,23 @@ Currently, testing is primarily manual. When adding tests:
Releases are managed by maintainers: Releases are managed by maintainers:
1. **Bump version using bumpversion:** 1. **Bump version using bumpversion:**
```bash ```bash
# Install bumpversion (if not already installed) # Install bumpversion (if not already installed)
pip install bumpversion pip install bumpversion
# Bump patch version (0.1.0 -> 0.1.1) # Bump patch version (0.1.0 -> 0.1.1)
bumpversion patch bumpversion patch
# Or bump minor version (0.1.0 -> 0.2.0) # Or bump minor version (0.1.0 -> 0.2.0)
bumpversion minor bumpversion minor
# Or bump major version (0.1.0 -> 1.0.0) # Or bump major version (0.1.0 -> 1.0.0)
bumpversion major bumpversion major
``` ```
This automatically: This automatically:
- Updates version numbers in all files (`tauri.conf.json`, `Cargo.toml`, all `package.json` files, `backend/main.py`) - Updates version numbers in all files (`tauri.conf.json`, `Cargo.toml`, all `package.json` files, `backend/main.py`)
- Creates a git commit with the version bump - Creates a git commit with the version bump
- Creates a git tag (e.g., `v0.1.1`, `v0.2.0`) - Creates a git tag (e.g., `v0.1.1`, `v0.2.0`)
@@ -392,6 +531,7 @@ Releases are managed by maintainers:
2. **Update CHANGELOG.md** with release notes 2. **Update CHANGELOG.md** with release notes
3. **Push commits and tags:** 3. **Push commits and tags:**
```bash ```bash
git push git push
git push --tags git push --tags
README.md
+46 -25
View File
@@ -78,12 +78,12 @@ Download a voice model, clone any voice from a few seconds of audio, and compose
Voicebox is available now for macOS and Windows. Voicebox is available now for macOS and Windows.
| Platform | Download | | Platform | Download |
|----------|----------| | --------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| macOS (Apple Silicon) | [voicebox_aarch64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_aarch64.app.tar.gz) | | macOS (Apple Silicon) | [voicebox_aarch64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_aarch64.app.tar.gz) |
| macOS (Intel) | [voicebox_x64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_x64.app.tar.gz) | | macOS (Intel) | [voicebox_x64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_x64.app.tar.gz) |
| Windows (MSI) | [voicebox_0.1.0_x64_en-US.msi](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_0.1.0_x64_en-US.msi) | | Windows (MSI) | [voicebox_0.1.0_x64_en-US.msi](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_0.1.0_x64_en-US.msi) |
| Windows (Setup) | [voicebox_0.1.0_x64-setup.exe](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_0.1.0_x64-setup.exe) | | Windows (Setup) | [voicebox_0.1.0_x64-setup.exe](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_0.1.0_x64-setup.exe) |
> **Linux builds coming soon** — Currently blocked by GitHub runner disk space limitations. > **Linux builds coming soon** — Currently blocked by GitHub runner disk space limitations.
@@ -176,17 +176,17 @@ Full API documentation available at `http://localhost:8000/docs` when running.
## Tech Stack ## Tech Stack
| Layer | Technology | | Layer | Technology |
|-------|------------| | ---------------- | --------------------------------------------------- |
| Desktop App | Tauri (Rust) | | Desktop App | Tauri (Rust) |
| Frontend | React, TypeScript, Tailwind CSS | | Frontend | React, TypeScript, Tailwind CSS |
| State | Zustand, React Query | | State | Zustand, React Query |
| Backend | FastAPI (Python) | | Backend | FastAPI (Python) |
| Voice Model | Qwen3-TTS (PyTorch or MLX) | | Voice Model | Qwen3-TTS (PyTorch or MLX) |
| Transcription | Whisper (PyTorch or MLX) | | Transcription | Whisper (PyTorch or MLX) |
| Inference Engine | MLX (Apple Silicon) / PyTorch (Windows/Linux/Intel) | | Inference Engine | MLX (Apple Silicon) / PyTorch (Windows/Linux/Intel) |
| Database | SQLite | | Database | SQLite |
| Audio | WaveSurfer.js, librosa | | Audio | WaveSurfer.js, librosa |
**Why this stack?** **Why this stack?**
@@ -194,6 +194,26 @@ Full API documentation available at `http://localhost:8000/docs` when running.
- **FastAPI** — Async Python with automatic OpenAPI schema generation - **FastAPI** — Async Python with automatic OpenAPI schema generation
- **Type-safe end-to-end** — Generated TypeScript client from OpenAPI spec - **Type-safe end-to-end** — Generated TypeScript client from OpenAPI spec
### TTS Provider Architecture
Voicebox uses a modular provider system to support different inference backends:
- **`apple-mlx`** — Bundled with macOS Apple Silicon builds
- Uses MLX with native Metal acceleration (4-5x faster)
- Works out of the box, no download required
- **`pytorch-cpu`** — Universal CPU provider (bundled or downloaded)
- Bundled with macOS Intel builds
- Downloaded on first use for Windows/Linux (~300MB)
- **`pytorch-cuda`** — Optional NVIDIA GPU-accelerated provider
- Windows/Linux only (~2.4GB)
- 4-5x faster inference on CUDA-capable GPUs
macOS builds work out of the box with bundled providers. Windows and Linux users download a provider on first launch. The app automatically detects your hardware and recommends the best option. All downloadable providers are distributed via Cloudflare R2 for fast, global delivery.
--- ---
## Roadmap ## Roadmap
@@ -202,13 +222,13 @@ Voicebox is the beginning of something bigger. Here's what's coming:
### Coming Soon ### Coming Soon
| Feature | Description | | Feature | Description |
|---------|-------------| | ----------------------- | -------------------------------------------------- |
| **Real-time Synthesis** | Stream audio as it generates, word by word | | **Real-time Synthesis** | Stream audio as it generates, word by word |
| **Conversation Mode** | Multi-speaker dialogues with automatic turn-taking | | **Conversation Mode** | Multi-speaker dialogues with automatic turn-taking |
| **Voice Effects** | Pitch shift, reverb, M3GAN-style effects | | **Voice Effects** | Pitch shift, reverb, M3GAN-style effects |
| **Timeline Editor** | Audio studio with word-level precision editing | | **Timeline Editor** | Audio studio with word-level precision editing |
| **More Models** | XTTS, Bark, and other open-source voice models | | **More Models** | XTTS, Bark, and other open-source voice models |
### Future Vision ### Future Vision
@@ -260,9 +280,10 @@ cd backend && pip install -r requirements.txt && cd ..
bun run dev bun run dev
``` ```
**Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org). **Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org).
**Performance:**
**Performance:**
- **Apple Silicon (M1/M2/M3)**: Uses MLX backend with native Metal acceleration for 4-5x faster inference - **Apple Silicon (M1/M2/M3)**: Uses MLX backend with native Metal acceleration for 4-5x faster inference
- **Windows/Linux/Intel Mac**: Uses PyTorch backend (CUDA GPU recommended, CPU supported but slower) - **Windows/Linux/Intel Mac**: Uses PyTorch backend (CUDA GPU recommended, CPU supported but slower)
app/src/components/ServerSettings/ProviderSettings.tsx
+3 -3
View File
@@ -25,7 +25,7 @@ const isMacOS = () => navigator.platform.toLowerCase().includes('mac');
type ProviderType = type ProviderType =
| 'auto' | 'auto'
| 'bundled-mlx' | 'apple-mlx'
| 'bundled-pytorch' | 'bundled-pytorch'
| 'pytorch-cpu' | 'pytorch-cpu'
| 'pytorch-cuda' | 'pytorch-cuda'
@@ -286,14 +286,14 @@ export function ProviderSettings() {
{/* MLX bundled (macOS Apple Silicon only) */} {/* MLX bundled (macOS Apple Silicon only) */}
{isMacOS() && ( {isMacOS() && (
<div className="flex items-center space-x-3 py-2"> <div className="flex items-center space-x-3 py-2">
<RadioGroupItem value="bundled-mlx" id="mlx" /> <RadioGroupItem value="apple-mlx" id="mlx" />
<Label htmlFor="mlx" className="flex-1 cursor-pointer"> <Label htmlFor="mlx" className="flex-1 cursor-pointer">
<div className="font-medium">MLX (Apple Silicon)</div> <div className="font-medium">MLX (Apple Silicon)</div>
<div className="text-sm text-muted-foreground"> <div className="text-sm text-muted-foreground">
Bundled with the app - optimized for M-series chips Bundled with the app - optimized for M-series chips
</div> </div>
</Label> </Label>
{currentProvider === 'bundled-mlx' && ( {currentProvider === 'apple-mlx' && (
<Badge variant="outline" className="ml-2"> <Badge variant="outline" className="ml-2">
Active Active
</Badge> </Badge>
backend/main.py
+1 -1
View File
@@ -1514,7 +1514,7 @@ async def list_providers():
# Get info for all known provider types # Get info for all known provider types
all_providers = [ all_providers = [
"bundled-mlx", "apple-mlx",
"bundled-pytorch", "bundled-pytorch",
"pytorch-cpu", "pytorch-cpu",
"pytorch-cuda", "pytorch-cuda",
backend/providers/__init__.py
+69 -8
View File
@@ -3,6 +3,7 @@ Provider management system for TTS providers.
""" """
from typing import Optional from typing import Optional
import asyncio
import platform import platform
from pathlib import Path from pathlib import Path
@@ -50,7 +51,7 @@ class ProviderManager:
Args: Args:
provider_type: Type of provider to start provider_type: Type of provider to start
""" """
if provider_type == "bundled-mlx": if provider_type == "apple-mlx":
# Use bundled MLX provider # Use bundled MLX provider
self.active_provider = self._get_default_provider() self.active_provider = self._get_default_provider()
elif provider_type in ["pytorch-cpu", "pytorch-cuda"]: elif provider_type in ["pytorch-cpu", "pytorch-cuda"]:
@@ -61,8 +62,15 @@ class ProviderManager:
# Find a free port # Find a free port
port = self._get_free_port() port = self._get_free_port()
# Start provider subprocess # Start provider subprocess with stdout/stderr capture
from ..config import get_data_dir from ..config import get_data_dir
import logging
logger = logging.getLogger(__name__)
logger.info(f"Starting provider {provider_type} on port {port}")
logger.info(f"Provider binary: {provider_path}")
logger.info(f"Data directory: {get_data_dir()}")
process = subprocess.Popen( process = subprocess.Popen(
[ [
str(provider_path), str(provider_path),
@@ -71,16 +79,49 @@ class ProviderManager:
], ],
stdout=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, stderr=subprocess.PIPE,
text=True,
bufsize=1,
) )
# Wait for provider to be ready # Wait for provider to be ready
base_url = f"http://127.0.0.1:{port}" base_url = f"http://127.0.0.1:{port}"
await self._wait_for_provider_health(base_url, timeout=30) try:
await self._wait_for_provider_health(base_url, timeout=30)
except TimeoutError as e:
# Capture subprocess output for debugging
stdout_lines = []
stderr_lines = []
# Try to read available output
import select
try:
if process.stdout and select.select([process.stdout], [], [], 0)[0]:
stdout_lines = process.stdout.readlines()
if process.stderr and select.select([process.stderr], [], [], 0)[0]:
stderr_lines = process.stderr.readlines()
except Exception:
# select might not work on all platforms
pass
logger.error(f"Provider failed to start. Stdout: {stdout_lines}")
logger.error(f"Provider failed to start. Stderr: {stderr_lines}")
# Terminate the process
process.terminate()
try:
process.wait(timeout=5)
except subprocess.TimeoutExpired:
process.kill()
raise
# Create LocalProvider instance # Create LocalProvider instance
self.active_provider = LocalProvider(base_url) self.active_provider = LocalProvider(base_url)
self._provider_process = process self._provider_process = process
self._provider_port = port self._provider_port = port
# Start background task to log subprocess output
asyncio.create_task(self._log_subprocess_output(process))
else: else:
# No external binary, use bundled provider (if available) # No external binary, use bundled provider (if available)
if provider_type == "pytorch-cpu": if provider_type == "pytorch-cpu":
@@ -131,7 +172,7 @@ class ProviderManager:
if system == "Darwin" and machine == "arm64": if system == "Darwin" and machine == "arm64":
# Apple Silicon gets MLX # Apple Silicon gets MLX
installed.append("bundled-mlx") installed.append("apple-mlx")
# PyTorch CPU is available on all platforms (check if bundled or downloaded) # PyTorch CPU is available on all platforms (check if bundled or downloaded)
# For now, assume it's bundled on macOS Intel, Windows, Linux # For now, assume it's bundled on macOS Intel, Windows, Linux
@@ -163,7 +204,7 @@ class ProviderManager:
Returns: Returns:
Provider information dictionary Provider information dictionary
""" """
if provider_type in ["bundled-mlx", "bundled-pytorch"]: if provider_type in ["apple-mlx", "bundled-pytorch"]:
return { return {
"type": provider_type, "type": provider_type,
"name": "Bundled Provider", "name": "Bundled Provider",
@@ -203,7 +244,7 @@ class ProviderManager:
"""Wait for provider to become healthy.""" """Wait for provider to become healthy."""
import httpx import httpx
import asyncio import asyncio
start_time = asyncio.get_event_loop().time() start_time = asyncio.get_event_loop().time()
while True: while True:
try: try:
@@ -213,12 +254,32 @@ class ProviderManager:
return return
except Exception: except Exception:
pass pass
if asyncio.get_event_loop().time() - start_time > timeout: if asyncio.get_event_loop().time() - start_time > timeout:
raise TimeoutError(f"Provider did not become healthy within {timeout} seconds") raise TimeoutError(f"Provider did not become healthy within {timeout} seconds")
await asyncio.sleep(0.5) await asyncio.sleep(0.5)
async def _log_subprocess_output(self, process: subprocess.Popen) -> None:
"""Log subprocess stdout and stderr."""
import logging
logger = logging.getLogger(__name__)
async def read_stream(stream, prefix):
if stream:
loop = asyncio.get_event_loop()
while True:
line = await loop.run_in_executor(None, stream.readline)
if not line:
break
logger.info(f"{prefix}: {line.rstrip()}")
await asyncio.gather(
read_stream(process.stdout, "Provider stdout"),
read_stream(process.stderr, "Provider stderr"),
return_exceptions=True,
)
# Global provider manager instance # Global provider manager instance
_provider_manager: Optional[ProviderManager] = None _provider_manager: Optional[ProviderManager] = None
backend/providers/types.py
+1 -1
View File
@@ -8,7 +8,7 @@ from enum import Enum
class ProviderType(str, Enum): class ProviderType(str, Enum):
"""Available provider types.""" """Available provider types."""
BUNDLED_MLX = "bundled-mlx" BUNDLED_MLX = "apple-mlx"
BUNDLED_PYTORCH = "bundled-pytorch" BUNDLED_PYTORCH = "bundled-pytorch"
PYTORCH_CPU = "pytorch-cpu" PYTORCH_CPU = "pytorch-cpu"
PYTORCH_CUDA = "pytorch-cuda" PYTORCH_CUDA = "pytorch-cuda"
package.json
+1
View File
@@ -18,6 +18,7 @@
"build:web": "cd web && bun run build", "build:web": "cd web && bun run build",
"build:landing": "cd landing && bun run build", "build:landing": "cd landing && bun run build",
"build:release": "./scripts/prepare-release.sh", "build:release": "./scripts/prepare-release.sh",
"build:providers": "python scripts/build-providers-local.py",
"generate:api": "./scripts/generate-api.sh", "generate:api": "./scripts/generate-api.sh",
"generate:keys": "cd tauri && bun tauri signer generate -w ~/.tauri/voicebox.key", "generate:keys": "cd tauri && bun tauri signer generate -w ~/.tauri/voicebox.key",
"build:server": "./scripts/build-server.sh", "build:server": "./scripts/build-server.sh",
providers/pytorch-cpu/build.py
-1
View File
@@ -61,7 +61,6 @@ def build_provider():
'--exclude-module', 'torch.utils.tensorboard', '--exclude-module', 'torch.utils.tensorboard',
'--exclude-module', 'tensorboard', '--exclude-module', 'tensorboard',
'--exclude-module', 'triton', '--exclude-module', 'triton',
'--exclude-module', 'torch.distributed',
'--exclude-module', 'torch._dynamo', '--exclude-module', 'torch._dynamo',
'--exclude-module', 'torch._inductor', '--exclude-module', 'torch._inductor',
'--exclude-module', 'torch.testing', '--exclude-module', 'torch.testing',
providers/pytorch-cpu/build_and_install.py
+57
View File
@@ -0,0 +1,57 @@
"""
Build PyTorch CPU provider and install to local provider directory.
"""
import platform
import shutil
from pathlib import Path
from build import build_provider
def get_providers_dir() -> Path:
"""Get the directory where providers are stored."""
system = platform.system()
if system == "Windows":
appdata = Path.home() / "AppData" / "Roaming"
elif system == "Darwin":
appdata = Path.home() / "Library" / "Application Support"
else: # Linux
appdata = Path.home() / ".local" / "share"
providers_dir = appdata / "voicebox" / "providers"
providers_dir.mkdir(parents=True, exist_ok=True)
return providers_dir
def main():
"""Build and install provider."""
provider_dir = Path(__file__).parent
# Build the provider
print("Building PyTorch CPU provider...")
build_provider()
# Determine binary name
binary_name = "tts-provider-pytorch-cpu"
if platform.system() == "Windows":
binary_name += ".exe"
# Source and destination paths
source = provider_dir / "dist" / binary_name
destination = get_providers_dir() / binary_name
# Copy to provider directory
print(f"Installing to {destination}...")
shutil.copy2(source, destination)
# Make executable on Unix systems
if platform.system() != "Windows":
destination.chmod(0o755)
print(f"✓ Provider installed successfully to {destination}")
if __name__ == "__main__":
main()
providers/pytorch-cpu/tts-provider-pytorch-cpu.spec
+48
View File
@@ -0,0 +1,48 @@
# -*- mode: python ; coding: utf-8 -*-
from PyInstaller.utils.hooks import collect_data_files
from PyInstaller.utils.hooks import collect_submodules
from PyInstaller.utils.hooks import copy_metadata
datas = []
hiddenimports = ['backend', 'backend.backends', 'backend.backends.pytorch_backend', 'backend.config', 'backend.utils.audio', 'backend.utils.cache', 'backend.utils.progress', 'backend.utils.hf_progress', 'backend.utils.tasks', 'torch', 'transformers', 'qwen_tts', 'qwen_tts.inference', 'qwen_tts.inference.qwen3_tts_model', 'qwen_tts.inference.qwen3_tts_tokenizer', 'qwen_tts.core', 'qwen_tts.cli', 'pkg_resources.extern', 'fastapi', 'uvicorn', 'soundfile', 'numpy', 'librosa']
datas += collect_data_files('qwen_tts')
datas += copy_metadata('qwen-tts')
hiddenimports += collect_submodules('qwen_tts')
hiddenimports += collect_submodules('jaraco')
a = Analysis(
['main.py'],
pathex=['/Users/jamespine/Projects/voicebox'],
binaries=[],
datas=datas,
hiddenimports=hiddenimports,
hookspath=[],
hooksconfig={},
runtime_hooks=[],
excludes=['torch.utils.tensorboard', 'tensorboard', 'triton', 'torch._dynamo', 'torch._inductor', 'torch.testing', 'torch.utils.benchmark', 'IPython', 'matplotlib', 'PIL', 'cv2', 'torchvision', 'torchaudio'],
noarchive=False,
optimize=0,
)
pyz = PYZ(a.pure)
exe = EXE(
pyz,
a.scripts,
a.binaries,
a.datas,
[],
name='tts-provider-pytorch-cpu',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
upx_exclude=[],
runtime_tmpdir=None,
console=True,
disable_windowed_traceback=False,
argv_emulation=False,
target_arch=None,
codesign_identity=None,
entitlements_file=None,
)
providers/pytorch-cuda/build.py
-1
View File
@@ -63,7 +63,6 @@ def build_provider():
'--exclude-module', 'torch.utils.tensorboard', '--exclude-module', 'torch.utils.tensorboard',
'--exclude-module', 'tensorboard', '--exclude-module', 'tensorboard',
'--exclude-module', 'triton', '--exclude-module', 'triton',
'--exclude-module', 'torch.distributed',
'--exclude-module', 'torch._dynamo', '--exclude-module', 'torch._dynamo',
'--exclude-module', 'torch._inductor', '--exclude-module', 'torch._inductor',
'--exclude-module', 'torch.testing', '--exclude-module', 'torch.testing',
providers/pytorch-cuda/build_and_install.py
+57
View File
@@ -0,0 +1,57 @@
"""
Build PyTorch CUDA provider and install to local provider directory.
"""
import platform
import shutil
from pathlib import Path
from build import build_provider
def get_providers_dir() -> Path:
"""Get the directory where providers are stored."""
system = platform.system()
if system == "Windows":
appdata = Path.home() / "AppData" / "Roaming"
elif system == "Darwin":
appdata = Path.home() / "Library" / "Application Support"
else: # Linux
appdata = Path.home() / ".local" / "share"
providers_dir = appdata / "voicebox" / "providers"
providers_dir.mkdir(parents=True, exist_ok=True)
return providers_dir
def main():
"""Build and install provider."""
provider_dir = Path(__file__).parent
# Build the provider
print("Building PyTorch CUDA provider...")
build_provider()
# Determine binary name
binary_name = "tts-provider-pytorch-cuda"
if platform.system() == "Windows":
binary_name += ".exe"
# Source and destination paths
source = provider_dir / "dist" / binary_name
destination = get_providers_dir() / binary_name
# Copy to provider directory
print(f"Installing to {destination}...")
shutil.copy2(source, destination)
# Make executable on Unix systems
if platform.system() != "Windows":
destination.chmod(0o755)
print(f"✓ Provider installed successfully to {destination}")
if __name__ == "__main__":
main()
providers/pytorch-cuda/tts-provider-pytorch-cuda.spec
+48
View File
@@ -0,0 +1,48 @@
# -*- mode: python ; coding: utf-8 -*-
from PyInstaller.utils.hooks import collect_data_files
from PyInstaller.utils.hooks import collect_submodules
from PyInstaller.utils.hooks import copy_metadata
datas = []
hiddenimports = ['backend', 'backend.backends', 'backend.backends.pytorch_backend', 'backend.config', 'backend.utils.audio', 'backend.utils.cache', 'backend.utils.progress', 'backend.utils.hf_progress', 'backend.utils.tasks', 'torch', 'torch.cuda', 'torch.backends.cudnn', 'transformers', 'qwen_tts', 'qwen_tts.inference', 'qwen_tts.inference.qwen3_tts_model', 'qwen_tts.inference.qwen3_tts_tokenizer', 'qwen_tts.core', 'qwen_tts.cli', 'pkg_resources.extern', 'fastapi', 'uvicorn', 'soundfile', 'numpy', 'librosa']
datas += collect_data_files('qwen_tts')
datas += copy_metadata('qwen-tts')
hiddenimports += collect_submodules('qwen_tts')
hiddenimports += collect_submodules('jaraco')
a = Analysis(
['main.py'],
pathex=['/Users/jamespine/Projects/voicebox'],
binaries=[],
datas=datas,
hiddenimports=hiddenimports,
hookspath=[],
hooksconfig={},
runtime_hooks=[],
excludes=['torch.utils.tensorboard', 'tensorboard', 'triton', 'torch._dynamo', 'torch._inductor', 'torch.testing', 'torch.utils.benchmark', 'IPython', 'matplotlib', 'PIL', 'cv2', 'torchvision', 'torchaudio'],
noarchive=False,
optimize=0,
)
pyz = PYZ(a.pure)
exe = EXE(
pyz,
a.scripts,
a.binaries,
a.datas,
[],
name='tts-provider-pytorch-cuda',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
upx_exclude=[],
runtime_tmpdir=None,
console=True,
disable_windowed_traceback=False,
argv_emulation=False,
target_arch=None,
codesign_identity=None,
entitlements_file=None,
)
scripts/build-providers-local.py
Executable
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env python3
"""
Build and install all TTS providers locally for development.
"""
import subprocess
import sys
from pathlib import Path
def main():
"""Build and install all providers."""
project_root = Path(__file__).parent.parent
providers_dir = project_root / "providers"
providers = ["pytorch-cpu", "pytorch-cuda"]
for provider in providers:
provider_path = providers_dir / provider
script_path = provider_path / "build_and_install.py"
if not script_path.exists():
print(f"⚠ Skipping {provider}: build_and_install.py not found")
continue
print(f"\n{'=' * 60}")
print(f"Building and installing {provider}...")
print(f"{'=' * 60}\n")
try:
subprocess.run(
[sys.executable, str(script_path)],
cwd=provider_path,
check=True,
)
except subprocess.CalledProcessError as e:
print(f"✗ Failed to build {provider}: {e}")
sys.exit(1)
print(f"\n{'=' * 60}")
print("✓ All providers built and installed successfully!")
print(f"{'=' * 60}\n")
if __name__ == "__main__":
main()
tauri/src-tauri/gen/Assets.car
BIN
View File
Binary file not shown.