+
+
+ {isPending
+ ? 'Generating...'
+ : !selectedProfileId
+ ? 'Select a voice profile first'
+ : 'Generate speech'}
+
+
+
+
+ Fine tune instructions
+
+
diff --git a/docs/plans/TTS_PROVIDER_ARCHITECTURE.md b/docs/plans/TTS_PROVIDER_ARCHITECTURE.md
new file mode 100644
index 00000000..8d35a7e5
--- /dev/null
+++ b/docs/plans/TTS_PROVIDER_ARCHITECTURE.md
@@ -0,0 +1,964 @@
+# TTS Provider Architecture
+
+**Status:** Planned for v0.1.13
+**Created:** 2025-01-31
+**Problem:** GitHub 2GB release limit + poor UX for frequent updates requiring 2.4GB re-downloads
+
+---
+
+## Overview
+
+Split the monolithic backend into modular components:
+
+1. **Main App** (~150-200MB): Tauri + FastAPI backend + Whisper + UI/profiles/history
+2. **TTS Providers** (downloadable plugins): Separate executables for model inference
+
+This architecture solves:
+
+- ✅ GitHub 2GB release artifact limit
+- ✅ Frequent app updates without re-downloading large python binaries
+- ✅ User choice of compute backend (CPU/GPU/Cloud)
+- ✅ External provider support (OpenAI, custom servers)
+- ✅ Future extensibility
+
+---
+
+## Architecture Diagram
+
+```
+┌─────────────────────────────────────────────────────────┐
+│ Voicebox App (Tauri + Backend) ~150MB │
+│ ├─ UI Layer (React) │
+│ ├─ Backend (FastAPI) │
+│ │ ├─ Voice Profiles │
+│ │ ├─ Generation History │
+│ │ ├─ Audio Editing / Stories │
+│ │ └─ Provider Manager ◄──────────────┐ │
+│ └─ Whisper (bundled, tiny ~50MB) │ │
+└─────────────────────────────────────────┼────────────────┘
+ │
+ HTTP/IPC │
+ │
+ ┌────────────────────────────────┼─────────────────┐
+ │ │ │
+ ▼ ▼ ▼
+┌─────────────────┐ ┌─────────────────┐ ┌──────────────────┐
+│ TTS Provider: │ │ TTS Provider: │ │ TTS Provider: │
+│ PyTorch CPU │ │ PyTorch CUDA │ │ MLX (Apple) │
+│ │ │ │ │ │
+│ ~300MB │ │ ~2.4GB │ │ ~800MB │
+│ │ │ │ │ │
+│ Local inference │ │ GPU inference │ │ Metal inference │
+└─────────────────┘ └─────────────────┘ └──────────────────┘
+ │ │ │
+ └────────────────────────┴─────────────────────┘
+ │
+ ┌─────────────▼──────────────┐
+ │ Future Providers: │
+ │ • Remote Server │
+ │ • OpenAI API │
+ │ • ElevenLabs │
+ │ • Custom Docker Container │
+ └────────────────────────────┘
+```
+
+---
+
+## Problem Statement
+
+### Current Architecture Issues
+
+**Monolithic Binary:**
+
+- CPU version: ~295MB
+- CUDA version: ~2.37GB
+- GitHub releases: 2GB file size limit (BLOCKED)
+- Updates require re-downloading entire binary
+- Poor UX: update app → restart → download CUDA update → restart again
+
+**User Pain Points:**
+
+1. Cannot release CUDA version on GitHub (over 2GB)
+2. Every app update forces 2.4GB re-download for GPU users
+3. No flexibility (can't use OpenAI, remote servers, etc.)
+4. Wastes bandwidth for small bug fixes
+
+---
+
+## Solution: Pluggable TTS Providers
+
+### Component Breakdown
+
+#### 1. Main App (voicebox.exe / .app / .AppImage)
+
+**Size:** ~100-150MB
+
+**Includes:**
+
+- Tauri runtime + React UI
+- FastAPI backend (pure Python, no PyTorch)
+- Whisper model (tiny, ~50MB)
+- SQLite database
+- Profile/history/audio editing logic
+- Provider management system
+
+**Does NOT include:**
+
+- PyTorch (CPU or CUDA)
+- TTS models (Qwen3-TTS)
+- Heavy ML dependencies
+
+**Updates frequently:** UI fixes, feature additions, non-ML changes
+
+---
+
+#### 2. TTS Provider: PyTorch CPU
+
+**Binary:** `tts-provider-pytorch-cpu.exe`
+**Size:** ~200MB
+
+**Includes:**
+
+- PyTorch CPU build
+- Qwen3-TTS package
+- Transformers
+- No CUDA libraries
+
+**Download source:** Cloudflare R2
+**Updates rarely:** Only when model code changes
+
+---
+
+#### 3. TTS Provider: PyTorch CUDA
+
+**Binary:** `tts-provider-pytorch-cuda.exe`
+**Size:** ~2.4GB
+
+**Includes:**
+
+- PyTorch CUDA build (cu121)
+- Qwen3-TTS package
+- CUDA runtime, cuDNN, cuBLAS
+- Transformers
+
+**Download source:** Cloudflare R2
+**Platform:** Windows + Linux (NVIDIA GPU)
+**Updates rarely:** Only when model code or CUDA version changes
+
+---
+
+#### 4. TTS Provider: MLX
+
+**Binary:** `tts-provider-mlx`
+**Size:** ~150MB
+
+**Includes:**
+
+- MLX framework
+- MLX-optimized Qwen3-TTS
+- Metal acceleration
+
+**Platform:** macOS only (Apple Silicon)
+**Download source:** Cloudflare R2
+
+---
+
+#### 5. TTS Provider: Remote
+
+**Binary:** None (built-in config)
+**Size:** 0MB
+
+**How it works:**
+
+- User provides URL to their own TTS server
+- Backend proxies requests to that server
+- Implements API spec from `EXTERNAL_PROVIDERS.md`
+
+**Use cases:**
+
+- AMD GPU users running their own server
+- Team deployments with shared GPU server
+- Cloud hosting (Modal, RunPod, Replicate)
+
+---
+
+#### 6. TTS Provider: OpenAI
+
+**Binary:** None (API wrapper)
+**Size:** 0MB
+
+**How it works:**
+
+- User provides OpenAI API key
+- Backend wraps OpenAI Audio API
+- Voice profiles map to OpenAI voices
+
+**Benefits:**
+
+- Zero local compute
+- Pay-per-use
+- Instant setup
+
+---
+
+## Communication Protocol
+
+### Provider API Specification
+
+All TTS providers must implement these endpoints:
+
+#### POST /tts/generate
+
+Generate speech from text.
+
+**Request:**
+
+```json
+{
+ "text": "Hello world!",
+ "voice_prompt": {
+ /* voice prompt object */
+ },
+ "language": "en",
+ "seed": 12345,
+ "model_size": "1.7B"
+}
+```
+
+**Response:**
+
+```json
+{
+ "audio": "base64-encoded-audio",
+ "sample_rate": 24000,
+ "duration": 2.5
+}
+```
+
+#### POST /tts/create_voice_prompt
+
+Create voice prompt from reference audio.
+
+**Request:** (multipart/form-data)
+
+- `audio`: Audio file
+- `reference_text`: Transcript
+
+**Response:**
+
+```json
+{
+ "voice_prompt": {
+ /* serialized prompt */
+ }
+}
+```
+
+#### GET /tts/health
+
+Health check.
+
+**Response:**
+
+```json
+{
+ "status": "healthy",
+ "provider": "pytorch-cuda",
+ "version": "1.0.0",
+ "model": "Qwen3-TTS-12Hz-1.7B-Base",
+ "device": "cuda:0"
+}
+```
+
+#### GET /tts/status
+
+Model status.
+
+**Response:**
+
+```json
+{
+ "model_loaded": true,
+ "model_size": "1.7B",
+ "available_sizes": ["0.6B", "1.7B"],
+ "gpu_available": true,
+ "vram_used_mb": 1234
+}
+```
+
+---
+
+## Backend Implementation
+
+### Provider Manager
+
+**File:** `backend/providers/__init__.py`
+
+```python
+class ProviderManager:
+ """Manages TTS provider lifecycle."""
+
+ def __init__(self):
+ self.active_provider: Optional[Provider] = None
+ self.config = load_provider_config()
+
+ async def start_provider(self, provider_type: str) -> str:
+ """Start a TTS provider process."""
+ if provider_type == "pytorch-cpu":
+ return await self._start_local_provider("tts-provider-pytorch-cpu.exe")
+ elif provider_type == "pytorch-cuda":
+ return await self._start_local_provider("tts-provider-pytorch-cuda.exe")
+ elif provider_type == "mlx":
+ return await self._start_local_provider("tts-provider-mlx")
+ elif provider_type == "remote":
+ return self.config["remote_url"]
+ elif provider_type == "openai":
+ return None # No subprocess, API wrapper
+
+ async def _start_local_provider(self, binary_name: str) -> str:
+ """Start local provider subprocess."""
+ provider_path = get_provider_binary_path(binary_name)
+
+ if not provider_path.exists():
+ raise ProviderNotInstalledException(binary_name)
+
+ # Start subprocess on random port
+ port = get_free_port()
+ process = subprocess.Popen([
+ str(provider_path),
+ "--port", str(port),
+ "--data-dir", str(config.get_data_dir())
+ ])
+
+ # Wait for provider to be ready
+ await wait_for_provider_health(f"http://localhost:{port}")
+
+ self.active_provider = Provider(process, port)
+ return f"http://localhost:{port}"
+
+ async def stop_provider(self):
+ """Stop active provider."""
+ if self.active_provider:
+ self.active_provider.process.terminate()
+ self.active_provider = None
+```
+
+---
+
+### Provider Abstraction
+
+**File:** `backend/providers/base.py`
+
+```python
+class TTSProvider(ABC):
+ """Abstract base for TTS providers."""
+
+ @abstractmethod
+ async def generate(
+ self,
+ text: str,
+ voice_prompt: dict,
+ language: str,
+ seed: Optional[int]
+ ) -> tuple[np.ndarray, int]:
+ """Generate speech audio."""
+ pass
+
+ @abstractmethod
+ async def create_voice_prompt(
+ self,
+ audio_path: str,
+ reference_text: str
+ ) -> dict:
+ """Create voice prompt from reference audio."""
+ pass
+```
+
+**File:** `backend/providers/local.py`
+
+```python
+class LocalProvider(TTSProvider):
+ """Provider that communicates with local subprocess via HTTP."""
+
+ def __init__(self, base_url: str):
+ self.base_url = base_url
+ self.client = httpx.AsyncClient()
+
+ async def generate(self, text, voice_prompt, language, seed):
+ response = await self.client.post(
+ f"{self.base_url}/tts/generate",
+ json={
+ "text": text,
+ "voice_prompt": voice_prompt,
+ "language": language,
+ "seed": seed
+ }
+ )
+ data = response.json()
+ audio = np.frombuffer(base64.b64decode(data["audio"]), dtype=np.float32)
+ return audio, data["sample_rate"]
+```
+
+**File:** `backend/providers/openai.py`
+
+```python
+class OpenAIProvider(TTSProvider):
+ """Provider that wraps OpenAI Audio API."""
+
+ def __init__(self, api_key: str):
+ self.client = OpenAI(api_key=api_key)
+
+ async def generate(self, text, voice_prompt, language, seed):
+ # Map voice_prompt to OpenAI voice name
+ voice = map_profile_to_openai_voice(voice_prompt)
+
+ response = await self.client.audio.speech.create(
+ model="tts-1",
+ voice=voice,
+ input=text
+ )
+
+ # Convert to numpy array
+ audio_data = response.content
+ audio, sr = load_audio_from_bytes(audio_data)
+ return audio, sr
+```
+
+---
+
+## Provider Installation
+
+### Download Manager
+
+**File:** `backend/providers/installer.py`
+
+```python
+class ProviderInstaller:
+ """Handles provider download and installation."""
+
+ async def download_provider(self, provider_type: str):
+ """Download provider binary from R2."""
+
+ binary_name = {
+ "pytorch-cpu": "tts-provider-pytorch-cpu.exe",
+ "pytorch-cuda": "tts-provider-pytorch-cuda.exe",
+ "mlx": "tts-provider-mlx"
+ }[provider_type]
+
+ download_url = f"https://downloads.voicebox.sh/providers/v{PROVIDER_VERSION}/{binary_name}"
+
+ # Download with progress tracking (reuse existing SSE system)
+ await download_with_progress(
+ url=download_url,
+ destination=get_provider_install_path(binary_name),
+ progress_key=f"provider-{provider_type}"
+ )
+```
+
+**Provider Storage Location:**
+
+- Windows: `%APPDATA%/voicebox/providers/`
+- macOS: `~/Library/Application Support/voicebox/providers/`
+- Linux: `~/.local/share/voicebox/providers/`
+
+---
+
+## Frontend Implementation
+
+### Provider Settings UI
+
+**Component:** `app/src/components/ServerSettings/ProviderSettings.tsx`
+
+```tsx
+export function ProviderSettings() {
+ const [selectedProvider, setSelectedProvider] =
+ useState
+
+
+ {/* PyTorch CUDA */}
+
+
+
+ {/* PyTorch CPU */}
+
+
+ {!installedProviders?.includes("pytorch-cuda") && gpuAvailable && (
+
+ )}
+
+
+
+ {/* MLX (macOS only) */}
+ {isMacOS && (
+
+
+ {!installedProviders?.includes("pytorch-cpu") && (
+
+ )}
+
+
+ )}
+
+ {/* Remote */}
+
+
+ {!installedProviders?.includes("mlx") && (
+
+ )}
+
+
+
+ {/* OpenAI */}
+
+
+ {selectedProvider === "remote" && (
+
+ )}
+
+
+
+
+ {selectedProvider === "openai" && (
+
+ )}
+ Voicebox is a local-first voice cloning studio with DAW-like features - for professional voice synthesis. Think of it as the Ollama for voice{' '} - — download models, clone voices, and generate speech entirely on your machine. + for professional voice synthesis. Think of it as a{' '} + local, free and open-source alternative to ElevenLabs — download + models, clone voices, and generate speech entirely on your machine.
Unlike cloud services that lock your voice data behind subscriptions, Voicebox gives
diff --git a/tauri/src-tauri/src/audio_capture/linux.rs b/tauri/src-tauri/src/audio_capture/linux.rs
new file mode 100644
index 00000000..8af26e97
--- /dev/null
+++ b/tauri/src-tauri/src/audio_capture/linux.rs
@@ -0,0 +1,16 @@
+use crate::audio_capture::AudioCaptureState;
+
+pub async fn start_capture(
+ state: &AudioCaptureState,
+ max_duration_secs: u32,
+) -> Result<(), String> {
+ todo!("implement Linux audio capture")
+}
+
+pub async fn stop_capture(state: &AudioCaptureState) -> Result