Compare commits

...
50 Commits
Author SHA1 Message Date
Jamie Pine 38bf96ff20 fix: pin numba for release CI wheel compatibility 2026-02-23 12:18:34 -08:00
Jamie Pine 0b14cb1b2c Bump version: 0.1.12 → 0.1.13 2026-02-23 11:24:46 -08:00
Jamie Pine 4d24e69012 docs: point download links to latest release 2026-02-23 11:23:30 -08:00
Jamie PineandGitHub 90436e428d Merge pull request #77 from ManuLG/fix/broken-confirmation-modals
fix: await for confirmation before deleting voices and channels
2026-02-23 11:13:12 -08:00
Jamie PineandGitHub e4bb288904 Merge pull request #93 from iJaack/fix/mlx-apple-silicon-binary
fix(mlx): bundle native libs and broaden error handling for Apple Silicon
2026-02-23 11:12:34 -08:00
Jamie PineandGitHub 6f8bc7f23b Merge pull request #95 from CelebrityPunks/fix/model-size-selection-ignored
Fix: selecting 0.6B model still downloads and uses 1.7B
2026-02-23 11:12:12 -08:00
Jamie PineandGitHub baca111d50 Merge branch 'main' into fix/model-size-selection-ignored 2026-02-23 11:12:05 -08:00
Jamie PineandGitHub cc298fe6d8 Merge pull request #79 from martyniukyurii/fix/unicode-content-disposition
fix: handle non-ASCII filenames in Content-Disposition headers
2026-02-23 11:09:36 -08:00
Jamie PineandGitHub 46b8f6b882 Merge pull request #78 from tomasmach/fix/getUserMedia-undefined-check
fix: guard getUserMedia call against undefined mediaDevices in non-secure contexts
2026-02-23 11:09:23 -08:00
Jamie PineandGitHub 162cf4fb84 Merge pull request #122 from white1107/fix/web-tailwind-plugin
fix(web): add @tailwindcss/vite plugin to web config
2026-02-21 13:46:30 -08:00
Jamie PineandGitHub 68558243d9 Merge pull request #126 from lemassykoi/main
Create requirements.txt
2026-02-21 13:46:07 -08:00
Jamie PineandGitHub 8d5ad926f9 Merge pull request #128 from mrigankad/fix/voicebox-bugs
fix: resolve multiple issues (#96, #119, #111, #108, #121, #125, #127)
2026-02-21 13:45:19 -08:00
Jamie PineandGitHub 334f037dce Merge pull request #146 from xPolar/landing/spacebot-banner
Add Spacebot banner to landing page
2026-02-21 13:41:31 -08:00
xPolar f6522eea80 Add Spacebot banner to landing page
Adds a persistent top-of-page banner linking to spacebot.sh,
another project by the creator of Voicebox. Uses existing design
tokens for a consistent look.
2026-02-21 13:37:44 -08:00
lemassykoiandAmp 7615a08f81 ci: add Windows-only build workflow without signing
Amp-Thread-ID: https://ampcode.com/threads/T-019c7c3e-072f-7109-86a7-072a6b309891
Co-authored-by: Amp <[email protected]>
2026-02-20 23:06:45 +01:00
lemassykoiandAmp 31ea3c68a5 fix: remove silent browser fallback that bypasses save dialog path
Amp-Thread-ID: https://ampcode.com/threads/T-019c7c3e-072f-7109-86a7-072a6b309891
Co-authored-by: Amp <[email protected]>
2026-02-20 19:27:41 +01:00
Mriganka 54d72ddfd0 fix: resolve multiple issues (#96, #119, #111, #108, #121, #125, #127) 2026-02-20 23:23:38 +05:30
Clément PAPPALARDOandGitHub d4794f78e1 Create requirements.txt 2026-02-20 17:14:14 +01:00
white1107 aa7c9a9a8d fix(web): add @tailwindcss/vite plugin to web config
The web version was missing the Tailwind CSS Vite plugin, causing
CSS to not load at all. This adds the same plugin configuration
that exists in the tauri version.

Fixes #121
2026-02-20 20:11:27 +09:00
AbrahamandClaude Opus 4.6 ca6ed0998a Fix model size selection ignored when generating speech
The /generate endpoint created the voice prompt before loading the
user's requested model size. Since create_voice_prompt() internally
calls load_model_async(None), it fell back to the hardcoded default
of "1.7B", causing the 1.7B model to be downloaded even when the
user explicitly selected 0.6B.

This reorders the operations so the requested model is loaded first,
ensuring create_voice_prompt() and generate() use the correct model.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-18 09:41:44 -08:00
Eva 829d4d6d5b fix(mlx): bundle native libs and broaden error handling for Apple Silicon
The distributed macOS aarch64 binary shipped without MLX acceleration despite
the model and backend code supporting it. Two root causes:

1. **OSError not caught in platform_detect.py**
   PyInstaller bundles isolate the filesystem, so when MLX tries to load its
   Metal shader libraries (.metallib) it raises OSError, not ImportError.
   platform_detect.get_backend_type() only caught ImportError, causing a
   silent fallback to PyTorch even on Apple Silicon hardware.
   Fix: broaden the except clause to (ImportError, OSError, RuntimeError)
   and import mlx.core instead of mlx (forces native lib loading eagerly).

2. **collect_data_files used instead of collect_all for MLX**
   build_binary.py and voicebox-server.spec used --collect-data /
   collect_data_files for mlx and mlx_audio. This copies Python source and
   pure-Python data, but NOT native shared libraries (.dylib, .metallib).
   Fix: switch to --collect-all / collect_all which captures binaries too,
   then pass them to Analysis(binaries=...) in the spec.

Result: macOS Apple Silicon users now get MLX inference (~4-5x faster than
PyTorch CPU), matching the performance documented in the README.
2026-02-18 16:51:48 +01:00
YuriiandCursor 0be7975db5 fix: handle non-ASCII filenames in Content-Disposition headers
The export endpoints (export-audio, export generation, export profile,
export story) crash with `'latin-1' codec can't encode characters` when
the generated text or profile/story name contains non-ASCII characters
(e.g. Cyrillic, Chinese, Arabic).

Root cause: Python's `str.isalnum()` passes Unicode letters through to
the filename, but HTTP headers are encoded as latin-1 by the ASGI server,
which cannot represent characters outside the 0-255 range.

Fix: introduce `_safe_content_disposition()` helper that builds a
standards-compliant header with an ASCII-only `filename` fallback and a
RFC 5987 `filename*=UTF-8''...` parameter for Unicode-capable clients.

Fixes #68

Co-authored-by: Cursor <[email protected]>
2026-02-17 12:58:42 +04:00
tomasmach 40e4af828a fix: guard getUserMedia call against undefined mediaDevices in non-secure contexts 2026-02-17 09:28:23 +01:00
Manuel Lorenzo 0e57826ea5 fix: await for confirmation before deleting voices and channels 2026-02-17 00:29:52 +01:00
Spacedrive Mac Mini 2 eb2cd861b1 chore: update Cargo.lock version to 0.1.12 2026-02-10 06:59:41 -08:00
Jamie PineandGitHub 701cc647a7 Merge pull request #57 from selop/chore/readme
chore: updates repo URL in README
2026-02-06 05:08:24 -08:00
Sergej Lopatkin be6ccaf044 chore: updates repo URL in README
Updates the repository URL in the README to point to the correct fork.

Adds a prerequisite for XCode on macOS for development.
2026-02-06 13:22:59 +01:00
Jamie PineandGitHub 1040625a88 Merge pull request #44 from selop/feature/delivery-instructions
Enhances floating generate box UX
2026-02-02 17:58:19 -08:00
Sergej Lopatkin 6f4503b521 Enhances floating generate box UX
- Adds tooltips on hover for buttons of the generate box
- Replaces the message square icon with a sliders icon for the instruction mode toggle.
- Adds a tooltip to the instruction mode toggle button.
- Updates the placeholder text for the input field.
2026-02-02 22:19:54 +01:00
Sergej LopatkinandGitHub f5b6edc2e7 Merge pull request #1 from jamiepine/main
update fork
2026-02-02 22:19:30 +01:00
Jamie PineandGitHub 8197f0724c Merge pull request #40 from Spyabo/fix/audio-export-path-resolution
Fix: audio export path resolution
2026-02-02 06:54:39 -08:00
Reese Wright d40f7d2676 refactor: improve path resolution readability 2026-02-02 14:54:05 +00:00
Reese Wright 99fbcca7f4 update CHANGELOG for audio export fix 2026-02-02 14:34:39 +00:00
Reese Wright 04f9880c9a fix audio export path resolution 2026-02-02 14:26:34 +00:00
Jamie Pine b9c858295d Update Voicebox description as an alternative to ElevenLabs, rather than Ollama 2026-02-01 00:45:47 -08:00
Jamie Pine 610f64c762 fix linux compile 2026-01-31 07:44:28 -08:00
Jamie Pine 220333b3bb corrections 2026-01-31 02:15:45 -08:00
Jamie Pine e194e95512 corrections 2026-01-31 02:14:37 -08:00
Jamie Pine e796412c2c corrections 2026-01-31 02:13:42 -08:00
Jamie Pine cb541521d2 Update TTS Provider Architecture status to v0.1.13 2026-01-31 02:11:41 -08:00
Jamie Pine 2bc243f93e Add TTS Provider Architecture plan
Solves GitHub 2GB limit + frequent update UX issues by splitting app into:
- Main app (~150MB): UI + backend logic + Whisper
- TTS Providers (plugins): Separate downloadable binaries
  - pytorch-cpu (~300MB)
  - pytorch-cuda (~2.4GB)
  - mlx (~800MB, macOS)
  - remote (connect to external server)
  - openai (API wrapper)

Benefits:
- Main app under GitHub 2GB limit
- Updates don't require re-downloading providers
- User choice of compute backend
- External provider support for teams/cloud
- Future-proof extensibility
2026-01-31 02:09:42 -08:00
Jamie Pine 0209008d73 disable cuda for 0.1.12 2026-01-31 01:46:14 -08:00
Jamie Pine 9bde534860 Bump version: 0.1.11 → 0.1.12 2026-01-30 21:23:07 -08:00
Jamie PineandGitHub 97eb570b28 Merge pull request #25 from jamiepine/fix-dl-notification-when-generating-from-already-cached-model
Fix dl notification when generating from already cached model
2026-01-30 21:20:25 -08:00
Jamie PineandGitHub 7d0557a099 Merge pull request #27 from jamiepine/model-dl-fix
Enhance model caching checks and progress tracking for downloads
2026-01-30 21:19:52 -08:00
Jamie PineandGitHub 20851ccc2b Merge pull request #24 from jamiepine/fix-multi-sample
Fix multi sample
2026-01-30 17:07:53 -08:00
Jamie Pine e5f4606a6c Update CircleButton component to include default button type
- Added a default `type` prop set to 'button' in the CircleButton component to ensure proper button behavior.
- Enhanced the component's flexibility by allowing the type to be overridden through props.
2026-01-30 17:07:35 -08:00
Jamie Pine 146ef5aaeb Add delete confirmation dialogs in HistoryTable and SampleList components
- Implemented delete confirmation dialogs for both HistoryTable and SampleList components to enhance user experience and prevent accidental deletions.
- Added state management for handling the selected item to be deleted and the visibility of the delete dialog.
- Refactored delete handling functions to utilize the new dialog confirmation flow, improving code clarity and maintainability.
2026-01-30 17:06:12 -08:00
Jamie Pine 971604d14f Implement profile cache management in audio processing
- Added `clear_profile_cache` function to manage cache files for specific profiles.
- Integrated cache clearing in `add_profile_sample`, `delete_profile`, and `delete_profile_sample` functions to ensure stale audio caches are invalidated after modifications.
- Enhanced `clear_voice_prompt_cache` to also delete combined audio files, improving overall cache management.
2026-01-30 16:50:24 -08:00
Jamie PineandGitHub 7fcca09f24 Merge pull request #23 from jamiepine/audio-export-entitlement-fix
Audio export entitlement fix
2026-01-30 15:08:50 -08:00
44 changed files with 1670 additions and 220 deletions
+1 -1
View File
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.1.11
current_version = 0.1.13
commit = True
tag = True
tag_name = v{new_version}
+63
View File
@@ -0,0 +1,63 @@
name: Build Windows
on:
workflow_dispatch:
jobs:
build-windows:
permissions:
contents: write
runs-on: windows-latest
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: "pip"
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install -r backend/requirements.txt
- name: Build Python server
shell: bash
run: |
cd backend
python build_binary.py
PLATFORM=$(rustc --print host-tuple)
mkdir -p ../tauri/src-tauri/binaries
cp dist/voicebox-server.exe ../tauri/src-tauri/binaries/voicebox-server-${PLATFORM}.exe
echo "Built voicebox-server-${PLATFORM}.exe"
- name: Setup Bun
uses: oven-sh/setup-bun@v2
- name: Install Rust stable
uses: dtolnay/rust-toolchain@stable
- name: Rust cache
uses: swatinem/rust-cache@v2
with:
workspaces: "./tauri/src-tauri -> target"
- name: Install dependencies
run: bun install
- uses: tauri-apps/tauri-action@v0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
projectPath: tauri
tagName: v__VERSION__
releaseName: "voicebox v__VERSION__ (test build)"
releaseBody: "Test build for audio export fix"
releaseDraft: true
prerelease: true
args: ""
includeUpdaterJson: false
+21 -21
View File
@@ -4,7 +4,7 @@ on:
workflow_dispatch:
push:
tags:
- 'v*'
- "v*"
jobs:
release:
@@ -14,22 +14,22 @@ jobs:
fail-fast: false
matrix:
include:
- platform: 'macos-latest'
args: '--target aarch64-apple-darwin'
python-version: '3.12'
backend: 'mlx'
- platform: 'macos-15-intel'
args: '--target x86_64-apple-darwin'
python-version: '3.12'
backend: 'pytorch'
- platform: "macos-latest"
args: "--target aarch64-apple-darwin"
python-version: "3.12"
backend: "mlx"
- platform: "macos-15-intel"
args: "--target x86_64-apple-darwin"
python-version: "3.12"
backend: "pytorch"
# - platform: 'ubuntu-22.04'
# args: ''
# python-version: '3.12'
# backend: 'pytorch'
- platform: 'windows-latest'
args: ''
python-version: '3.12'
backend: 'pytorch'
- platform: "windows-latest"
args: ""
python-version: "3.12"
backend: "pytorch"
runs-on: ${{ matrix.platform }}
@@ -53,7 +53,7 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache: "pip"
- name: Install Python dependencies
run: |
@@ -66,11 +66,11 @@ jobs:
run: |
pip install -r backend/requirements-mlx.txt
- name: Install PyTorch with CUDA (Windows only)
if: matrix.platform == 'windows-latest'
run: |
pip install torch --index-url https://download.pytorch.org/whl/cu121 --force-reinstall --no-deps
pip install torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
# - name: Install PyTorch with CUDA (Windows only)
# if: matrix.platform == 'windows-latest'
# run: |
# pip install torch --index-url https://download.pytorch.org/whl/cu121 --force-reinstall --no-deps
# pip install torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
- name: Build Python server (Linux/macOS)
if: matrix.platform != 'windows-latest'
@@ -106,7 +106,7 @@ jobs:
- name: Rust cache
uses: swatinem/rust-cache@v2
with:
workspaces: './tauri/src-tauri -> target'
workspaces: "./tauri/src-tauri -> target"
- name: Install dependencies
run: bun install
@@ -142,7 +142,7 @@ jobs:
with:
projectPath: tauri
tagName: v__VERSION__
releaseName: 'voicebox v__VERSION__'
releaseName: "voicebox v__VERSION__"
releaseBody: |
## What's Changed
See the assets below to download and install this version.
+3
View File
@@ -53,6 +53,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Fixed
- Audio export failing when Tauri save dialog returns object instead of string path
### Added
- **Makefile** - Comprehensive development workflow automation with commands for setup, development, building, testing, and code quality checks
- Includes Python version detection and compatibility warnings
+8 -8
View File
@@ -59,7 +59,7 @@
## What is Voicebox?
Voicebox is a **local-first voice cloning studio** with DAW-like features for professional voice synthesis. Think of it as the **Ollama for voice** — download models, clone voices, and generate speech entirely on your machine.
Voicebox is a **local-first voice cloning studio** with DAW-like features for professional voice synthesis. Think of it as a **local, free and open-source alternative to ElevenLabs** — download models, clone voices, and generate speech entirely on your machine.
Unlike cloud services that lock your voice data behind subscriptions, Voicebox gives you:
@@ -80,10 +80,10 @@ Voicebox is available now for macOS and Windows.
| Platform | Download |
|----------|----------|
| macOS (Apple Silicon) | [voicebox_aarch64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_aarch64.app.tar.gz) |
| macOS (Intel) | [voicebox_x64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_x64.app.tar.gz) |
| Windows (MSI) | [voicebox_0.1.0_x64_en-US.msi](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_0.1.0_x64_en-US.msi) |
| Windows (Setup) | [voicebox_0.1.0_x64-setup.exe](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_0.1.0_x64-setup.exe) |
| macOS (Apple Silicon) | [Voicebox_aarch64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/latest/download/Voicebox_aarch64.app.tar.gz) |
| macOS (Intel) | [Voicebox_x64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/latest/download/Voicebox_x64.app.tar.gz) |
| Windows (MSI) | [Latest Windows MSI](https://github.com/jamiepine/voicebox/releases/latest) |
| Windows (Setup) | [Latest Windows Setup](https://github.com/jamiepine/voicebox/releases/latest) |
> **Linux builds coming soon** — Currently blocked by GitHub runner disk space limitations.
@@ -233,7 +233,7 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed setup and contribution guide
```bash
# Clone the repo
git clone https://github.com/voicebox-sh/voicebox.git
git clone https://github.com/jamiepine/voicebox.git
cd voicebox
# Setup everything
@@ -247,7 +247,7 @@ make dev
```bash
# Clone the repo
git clone https://github.com/voicebox-sh/voicebox.git
git clone https://github.com/jamiepine/voicebox.git
cd voicebox
# Install dependencies
@@ -260,7 +260,7 @@ cd backend && pip install -r requirements.txt && cd ..
bun run dev
```
**Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org).
**Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org). [XCode on macOS](https://developer.apple.com/xcode/).
**Performance:**
- **Apple Silicon (M1/M2/M3)**: Uses MLX backend with native Metal acceleration for 4-5x faster inference
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@voicebox/app",
"version": "0.1.11",
"version": "0.1.13",
"private": true,
"type": "module",
"scripts": {
+8 -6
View File
@@ -124,6 +124,13 @@ export function AudioTab() {
);
}
const handleChannelDelete = async (e, channelId) => {
e.stopPropagation();
if (await confirm('Delete this channel?')) {
deleteChannel.mutate(channelId);
}
}
const allChannels = channels || [];
const allDevices = devices || [];
const selectedChannel = selectedChannelId
@@ -241,12 +248,7 @@ export function AudioTab() {
variant="ghost"
size="sm"
className="h-8 w-8 p-0"
onClick={(e) => {
e.stopPropagation();
if (confirm('Delete this channel?')) {
deleteChannel.mutate(channel.id);
}
}}
onClick={(e) => handleChannelDelete(e, channel.id)}
>
<Trash2 className="h-4 w-4" />
</Button>
@@ -1,6 +1,6 @@
import { useMatchRoute } from '@tanstack/react-router';
import { AnimatePresence, motion } from 'framer-motion';
import { Loader2, MessageSquare, Sparkles } from 'lucide-react';
import { Loader2, SlidersHorizontal, Sparkles } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Form, FormControl, FormField, FormItem, FormMessage } from '@/components/ui/form';
@@ -187,7 +187,7 @@ export function FloatingGenerateBox({
}}
>
<motion.div
className="bg-background/30 backdrop-blur-2xl border border-accent/20 rounded-[2rem] shadow-2xl hover:bg-background/40 hover:border-accent/20 transition-all duration-300 overflow-hidden p-3"
className="bg-background/30 backdrop-blur-2xl border border-accent/20 rounded-[2rem] shadow-2xl hover:bg-background/40 hover:border-accent/20 transition-all duration-300 p-3"
transition={{ duration: 0.6, ease: 'easeInOut' }}
>
<Form {...form}>
@@ -274,7 +274,7 @@ export function FloatingGenerateBox({
field.ref(node);
}
}}
placeholder="Add delivery instructions..."
placeholder="e.g. very happy and excited"
className="resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm placeholder:text-muted-foreground/60 w-full"
style={{
minHeight: isExpanded ? '100px' : '32px',
@@ -294,18 +294,27 @@ export function FloatingGenerateBox({
</motion.div>
<div className="relative shrink-0">
<Button
type="submit"
disabled={isPending || !selectedProfileId}
className="h-10 w-10 rounded-full bg-accent hover:bg-accent/90 hover:scale-105 text-accent-foreground shadow-lg hover:shadow-accent/50 transition-all duration-200"
size="icon"
>
{isPending ? (
<Loader2 className="h-4 w-4 animate-spin" />
) : (
<Sparkles className="h-4 w-4" />
)}
</Button>
<div className="group relative">
<Button
type="submit"
disabled={isPending || !selectedProfileId}
className="h-10 w-10 rounded-full bg-accent hover:bg-accent/90 hover:scale-105 text-accent-foreground shadow-lg hover:shadow-accent/50 transition-all duration-200"
size="icon"
>
{isPending ? (
<Loader2 className="h-4 w-4 animate-spin" />
) : (
<Sparkles className="h-4 w-4" />
)}
</Button>
<span className="pointer-events-none absolute bottom-full left-1/2 -translate-x-1/2 mb-2 whitespace-nowrap rounded-md bg-popover px-3 py-1.5 text-xs text-popover-foreground border border-border opacity-0 transition-opacity group-hover:opacity-100 z-[9999]">
{isPending
? 'Generating...'
: !selectedProfileId
? 'Select a voice profile first'
: 'Generate speech'}
</span>
</div>
<AnimatePresence>
{isExpanded && (
<motion.div
@@ -315,20 +324,25 @@ export function FloatingGenerateBox({
transition={{ duration: 0.2 }}
className="absolute top-0 right-[calc(100%+0.5rem)]"
>
<Button
type="button"
variant="ghost"
size="icon"
onClick={() => setIsInstructMode(!isInstructMode)}
className={cn(
'h-10 w-10 rounded-full transition-all duration-200',
isInstructMode
? 'bg-accent text-accent-foreground border border-accent hover:bg-accent/90'
: 'bg-card border border-border hover:bg-background/50',
)}
>
<MessageSquare className="h-4 w-4" />
</Button>
<div className="group relative">
<Button
type="button"
variant="ghost"
size="icon"
onClick={() => setIsInstructMode(!isInstructMode)}
className={cn(
'h-10 w-10 rounded-full transition-all duration-200',
isInstructMode
? 'bg-accent text-accent-foreground border border-accent hover:bg-accent/90'
: 'bg-card border border-border hover:bg-background/50',
)}
>
<SlidersHorizontal className="h-4 w-4" />
</Button>
<span className="pointer-events-none absolute bottom-full left-1/2 -translate-x-1/2 mb-2 whitespace-nowrap rounded-md bg-popover px-3 py-1.5 text-xs text-popover-foreground border border-border opacity-0 transition-opacity group-hover:opacity-100 z-[9999]">
Fine tune instructions
</span>
</div>
</motion.div>
)}
</AnimatePresence>
+45 -1
View File
@@ -53,6 +53,8 @@ export function HistoryTable() {
const fileInputRef = useRef<HTMLInputElement>(null);
const [importDialogOpen, setImportDialogOpen] = useState(false);
const [selectedFile, setSelectedFile] = useState<File | null>(null);
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
const [generationToDelete, setGenerationToDelete] = useState<{ id: string; name: string } | null>(null);
const limit = 20;
const { toast } = useToast();
@@ -179,6 +181,19 @@ export function HistoryTable() {
);
};
const handleDeleteClick = (generationId: string, profileName: string) => {
setGenerationToDelete({ id: generationId, name: profileName });
setDeleteDialogOpen(true);
};
const handleDeleteConfirm = () => {
if (generationToDelete) {
deleteGeneration.mutate(generationToDelete.id);
setDeleteDialogOpen(false);
setGenerationToDelete(null);
}
};
const handleImportConfirm = () => {
if (selectedFile) {
importGeneration.mutate(selectedFile, {
@@ -320,7 +335,7 @@ export function HistoryTable() {
Export Package
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => deleteGeneration.mutate(gen.id)}
onClick={() => handleDeleteClick(gen.id, gen.profile_name)}
disabled={deleteGeneration.isPending}
className="text-destructive focus:text-destructive"
>
@@ -351,6 +366,35 @@ export function HistoryTable() {
</>
)}
<Dialog open={deleteDialogOpen} onOpenChange={setDeleteDialogOpen}>
<DialogContent>
<DialogHeader>
<DialogTitle>Delete Generation</DialogTitle>
<DialogDescription>
Are you sure you want to delete this generation from "{generationToDelete?.name}"? This action cannot be undone.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button
variant="outline"
onClick={() => {
setDeleteDialogOpen(false);
setGenerationToDelete(null);
}}
>
Cancel
</Button>
<Button
variant="destructive"
onClick={handleDeleteConfirm}
disabled={deleteGeneration.isPending}
>
{deleteGeneration.isPending ? 'Deleting...' : 'Delete'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
<Dialog open={importDialogOpen} onOpenChange={setImportDialogOpen}>
<DialogContent>
<DialogHeader>
@@ -58,6 +58,7 @@ export function AudioSampleRecording({
// Request microphone access when component mounts
useEffect(() => {
if (!showWaveform) return;
if (!navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) return;
let stream: MediaStream | null = null;
@@ -43,7 +43,7 @@ import {
} from '@/lib/hooks/useProfiles';
import { useSystemAudioCapture } from '@/lib/hooks/useSystemAudioCapture';
import { useTranscription } from '@/lib/hooks/useTranscription';
import { formatAudioDuration, getAudioDuration } from '@/lib/utils/audio';
import { convertToWav, formatAudioDuration, getAudioDuration } from '@/lib/utils/audio';
import { usePlatform } from '@/platform/PlatformContext';
import { useServerStore } from '@/stores/serverStore';
import { type ProfileFormDraft, useUIStore } from '@/stores/uiStore';
@@ -505,10 +505,23 @@ export function ProfileForm() {
language: data.language,
});
// Convert non-WAV uploads to WAV so the backend can always use soundfile.
// Recorded audio is already WAV (from useAudioRecording's convertToWav call).
let fileToUpload: File = sampleFile;
if (!sampleFile.type.includes('wav') && !sampleFile.name.toLowerCase().endsWith('.wav')) {
try {
const wavBlob = await convertToWav(sampleFile);
const wavName = sampleFile.name.replace(/\.[^.]+$/, '.wav');
fileToUpload = new File([wavBlob], wavName, { type: 'audio/wav' });
} catch {
// If browser can't decode the format, send the original and let the backend try.
}
}
try {
await addSample.mutateAsync({
profileId: profile.id,
file: sampleFile,
file: fileToUpload,
referenceText: referenceText,
});
@@ -2,6 +2,14 @@ import { Check, Edit, Pause, Play, Plus, Trash2, Volume2, X } from 'lucide-react
import { useEffect, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { CircleButton } from '@/components/ui/circle-button';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { Slider } from '@/components/ui/slider';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
@@ -140,10 +148,19 @@ export function SampleList({ profileId }: SampleListProps) {
const [uploadOpen, setUploadOpen] = useState(false);
const [editingSampleId, setEditingSampleId] = useState<string | null>(null);
const [editedText, setEditedText] = useState<string>('');
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
const [sampleToDelete, setSampleToDelete] = useState<string | null>(null);
const handleDelete = (sampleId: string) => {
if (confirm('Are you sure you want to delete this sample?')) {
deleteSample.mutate(sampleId);
const handleDeleteClick = (sampleId: string) => {
setSampleToDelete(sampleId);
setDeleteDialogOpen(true);
};
const handleDeleteConfirm = () => {
if (sampleToDelete) {
deleteSample.mutate(sampleToDelete);
setDeleteDialogOpen(false);
setSampleToDelete(null);
}
};
@@ -268,7 +285,7 @@ export function SampleList({ profileId }: SampleListProps) {
<CircleButton
icon={Trash2}
title="Delete sample"
onClick={() => handleDelete(sample.id)}
onClick={() => handleDeleteClick(sample.id)}
disabled={deleteSample.isPending}
/>
</div>
@@ -306,6 +323,35 @@ export function SampleList({ profileId }: SampleListProps) {
</p>
<SampleUpload profileId={profileId} open={uploadOpen} onOpenChange={setUploadOpen} />
<Dialog open={deleteDialogOpen} onOpenChange={setDeleteDialogOpen}>
<DialogContent>
<DialogHeader>
<DialogTitle>Delete Sample</DialogTitle>
<DialogDescription>
Are you sure you want to delete this audio sample? This action cannot be undone.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button
variant="outline"
onClick={() => {
setDeleteDialogOpen(false);
setSampleToDelete(null);
}}
>
Cancel
</Button>
<Button
variant="destructive"
onClick={handleDeleteConfirm}
disabled={deleteSample.isPending}
>
{deleteSample.isPending ? 'Deleting...' : 'Delete'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</div>
);
}
+3 -3
View File
@@ -79,8 +79,8 @@ export function VoicesTab() {
setDialogOpen(true);
};
const handleDelete = (profileId: string) => {
if (confirm('Are you sure you want to delete this profile?')) {
const handleProfileDelete = async (profileId: string) => {
if (await confirm('Are you sure you want to delete this profile?')) {
deleteProfile.mutate(profileId);
}
};
@@ -147,7 +147,7 @@ export function VoicesTab() {
channels={channels || []}
onChannelChange={(channelIds) => handleChannelChange(profile.id, channelIds)}
onEdit={() => handleEdit(profile.id)}
onDelete={() => handleDelete(profile.id)}
onDelete={() => handleProfileDelete(profile.id)}
/>
))}
</TableBody>
+2 -1
View File
@@ -6,10 +6,11 @@ export interface CircleButtonProps extends React.ButtonHTMLAttributes<HTMLButton
}
const CircleButton = React.forwardRef<HTMLButtonElement, CircleButtonProps>(
({ className, icon: Icon, ...props }, ref) => {
({ className, icon: Icon, type = 'button', ...props }, ref) => {
return (
<button
ref={ref}
type={type}
className={cn(
'h-7 w-7 rounded-full flex items-center justify-center flex-shrink-0',
'hover:bg-muted transition-colors',
+26 -20
View File
@@ -20,11 +20,13 @@ export function useAudioRecording({
const streamRef = useRef<MediaStream | null>(null);
const timerRef = useRef<number | null>(null);
const startTimeRef = useRef<number | null>(null);
const cancelledRef = useRef<boolean>(false);
const startRecording = useCallback(async () => {
try {
setError(null);
chunksRef.current = [];
cancelledRef.current = false;
setDuration(0);
// Check if getUserMedia is available
@@ -87,31 +89,34 @@ export function useAudioRecording({
};
mediaRecorder.onstop = async () => {
// Snapshot the cancellation flag and recorded duration immediately —
// cancelRecording() clears chunks and sets cancelledRef synchronously
// before this async handler runs, so we must check it first.
const wasCancelled = cancelledRef.current;
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
const webmBlob = new Blob(chunksRef.current, { type: 'audio/webm' });
// Convert to WAV format to avoid needing ffmpeg on backend
try {
const wavBlob = await convertToWav(webmBlob);
// Pass the actual recorded duration
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
onRecordingComplete?.(wavBlob, recordedDuration);
} catch (err) {
console.error('Error converting audio to WAV:', err);
// Fallback to original blob if conversion fails
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
onRecordingComplete?.(webmBlob, recordedDuration);
}
// Stop all tracks
// Stop all tracks now that we have the data
streamRef.current?.getTracks().forEach((track) => {
track.stop();
});
streamRef.current = null;
// Don't fire completion callback if the recording was cancelled
if (wasCancelled) return;
// Convert to WAV format to avoid needing ffmpeg on backend
try {
const wavBlob = await convertToWav(webmBlob);
onRecordingComplete?.(wavBlob, recordedDuration);
} catch (err) {
console.error('Error converting audio to WAV:', err);
// Fallback to original blob if conversion fails
onRecordingComplete?.(webmBlob, recordedDuration);
}
};
mediaRecorder.onerror = (event) => {
@@ -167,9 +172,10 @@ export function useAudioRecording({
const cancelRecording = useCallback(() => {
if (mediaRecorderRef.current) {
cancelledRef.current = true; // Must be set before stop() triggers onstop
chunksRef.current = [];
mediaRecorderRef.current.stop();
setIsRecording(false);
chunksRef.current = [];
setDuration(0);
}
+36 -18
View File
@@ -22,6 +22,11 @@ export function formatAudioDuration(seconds: number): string {
* If the file has a recordedDuration property (from recording hooks),
* use that instead of trying to read metadata. This fixes issues on Windows
* where WebM files from MediaRecorder don't have proper duration metadata.
*
* For uploaded files we use AudioContext.decodeAudioData which fully decodes
* the audio and returns the exact duration. This is more reliable than
* HTMLMediaElement.duration which can return incorrect large values for VBR
* MP3 files that lack a proper XING/VBRI header.
*/
export async function getAudioDuration(
file: File & { recordedDuration?: number },
@@ -30,26 +35,39 @@ export async function getAudioDuration(
return file.recordedDuration;
}
return new Promise((resolve, reject) => {
const audio = new Audio();
const url = URL.createObjectURL(file);
// Use Web Audio API for accurate duration — avoids VBR MP3 metadata issues.
try {
const audioContext = new AudioContext();
try {
const arrayBuffer = await file.arrayBuffer();
const audioBuffer = await audioContext.decodeAudioData(arrayBuffer);
return audioBuffer.duration;
} finally {
await audioContext.close();
}
} catch {
// Fallback: read duration from the media element (less accurate but works for WAV).
return new Promise((resolve, reject) => {
const audio = new Audio();
const url = URL.createObjectURL(file);
audio.addEventListener('loadedmetadata', () => {
URL.revokeObjectURL(url);
if (Number.isFinite(audio.duration) && audio.duration > 0) {
resolve(audio.duration);
} else {
reject(new Error('Audio file has invalid duration metadata'));
}
audio.addEventListener('loadedmetadata', () => {
URL.revokeObjectURL(url);
if (Number.isFinite(audio.duration) && audio.duration > 0) {
resolve(audio.duration);
} else {
reject(new Error('Audio file has invalid duration metadata'));
}
});
audio.addEventListener('error', () => {
URL.revokeObjectURL(url);
reject(new Error('Failed to load audio file'));
});
audio.src = url;
});
audio.addEventListener('error', () => {
URL.revokeObjectURL(url);
reject(new Error('Failed to load audio file'));
});
audio.src = url;
});
}
}
/**
+1 -1
View File
@@ -1,3 +1,3 @@
# Backend package
__version__ = "0.1.11"
__version__ = "0.1.13"
+48 -11
View File
@@ -29,9 +29,23 @@ class PyTorchTTSBackend:
"""Get the best available device."""
if torch.cuda.is_available():
return "cuda"
elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
# MPS can have issues, use CPU for stability
return "cpu"
# Intel Arc / Intel Xe GPU via intel-extension-for-pytorch (IPEX)
try:
import intel_extension_for_pytorch # noqa: F401
if hasattr(torch, 'xpu') and torch.xpu.is_available():
return "xpu"
except ImportError:
pass
# Any GPU on Windows via DirectML (torch-directml)
try:
import torch_directml
if torch_directml.device_count() > 0:
return torch_directml.device(0)
except ImportError:
pass
# MPS (Apple Silicon) — kept for completeness but MLX backend is preferred
if hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
return "cpu" # MPS disabled for stability; MLX backend handles Apple Silicon
return "cpu"
def is_loaded(self) -> bool:
@@ -166,11 +180,21 @@ class PyTorchTTSBackend:
# Load the model (tqdm is patched, but filters out non-download progress)
try:
self.model = Qwen3TTSModel.from_pretrained(
model_path,
device_map=self.device,
torch_dtype=torch.float32 if self.device == "cpu" else torch.bfloat16,
)
# Don't pass device_map on CPU: accelerate's meta-tensor mechanism
# causes "Cannot copy out of meta tensor" when moving to CPU.
# Instead load directly then call .to(device) if needed.
if self.device == "cpu":
self.model = Qwen3TTSModel.from_pretrained(
model_path,
torch_dtype=torch.float32,
low_cpu_mem_usage=False,
)
else:
self.model = Qwen3TTSModel.from_pretrained(
model_path,
device_map=self.device,
torch_dtype=torch.bfloat16,
)
finally:
# Exit the patch context
tracker_context.__exit__(None, None, None)
@@ -358,9 +382,22 @@ class PyTorchSTTBackend:
"""Get the best available device."""
if torch.cuda.is_available():
return "cuda"
elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
# MPS support for Whisper
return "cpu" # Use CPU for stability
# Intel Arc / Intel Xe GPU via intel-extension-for-pytorch (IPEX)
try:
import intel_extension_for_pytorch # noqa: F401
if hasattr(torch, 'xpu') and torch.xpu.is_available():
return "xpu"
except ImportError:
pass
# Any GPU on Windows via DirectML (torch-directml)
try:
import torch_directml
if torch_directml.device_count() > 0:
return torch_directml.device(0)
except ImportError:
pass
if hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
return "cpu" # MPS disabled for stability
return "cpu"
def is_loaded(self) -> bool:
+7 -3
View File
@@ -83,9 +83,13 @@ def build_server():
'--hidden-import', 'mlx_audio.stt',
'--collect-submodules', 'mlx',
'--collect-submodules', 'mlx_audio',
# Collect MLX data files including Metal shader libraries (.metallib)
'--collect-data', 'mlx',
'--collect-data', 'mlx_audio',
# Use --collect-all so PyInstaller bundles both data files AND
# native shared libraries (.dylib, .metallib) for MLX.
# Previously only --collect-data was used, which caused MLX to
# raise OSError at runtime inside the bundled binary because
# the Metal shader libraries were missing.
'--collect-all', 'mlx',
'--collect-all', 'mlx_audio',
])
else:
print("Building for non-Apple Silicon platform - PyTorch only")
+9
View File
@@ -4,8 +4,17 @@ Configuration module for voicebox backend.
Handles data directory configuration for production bundling.
"""
import os
from pathlib import Path
# Allow users to override the HuggingFace model download directory.
# Set VOICEBOX_MODELS_DIR to an absolute path before starting the server.
# This sets HF_HUB_CACHE so all huggingface_hub downloads go to that path.
_custom_models_dir = os.environ.get("VOICEBOX_MODELS_DIR")
if _custom_models_dir:
os.environ["HF_HUB_CACHE"] = _custom_models_dir
print(f"[config] Model download path set to: {_custom_models_dir}")
# Default data directory (used in development)
_data_dir = Path("data")
+153 -39
View File
@@ -22,6 +22,24 @@ import uuid
import asyncio
import signal
import os
from urllib.parse import quote
def _safe_content_disposition(disposition_type: str, filename: str) -> str:
"""Build a Content-Disposition header that is safe for non-ASCII filenames.
Uses RFC 5987 ``filename*`` parameter so that browsers can decode
UTF-8 filenames while the ``filename`` fallback stays ASCII-only.
"""
ascii_name = "".join(
c for c in filename if c.isascii() and (c.isalnum() or c in " -_.")
).strip() or "download"
utf8_name = quote(filename, safe="")
return (
f'{disposition_type}; filename="{ascii_name}"; '
f"filename*=UTF-8''{utf8_name}"
)
from . import database, models, profiles, history, tts, transcribe, config, export_import, channels, stories, __version__
from .database import get_db, Generation as DBGeneration, VoiceProfile as DBVoiceProfile
@@ -77,10 +95,39 @@ async def health():
tts_model = tts.get_tts_model()
backend_type = get_backend_type()
# Check for GPU availability (CUDA or MPS)
# Check for GPU availability (CUDA, MPS, Intel Arc XPU, or DirectML)
has_cuda = torch.cuda.is_available()
has_mps = hasattr(torch.backends, 'mps') and torch.backends.mps.is_available()
gpu_available = has_cuda or has_mps
# Intel Arc / Intel Xe via intel-extension-for-pytorch (IPEX)
has_xpu = False
xpu_name = None
try:
import intel_extension_for_pytorch as ipex # noqa: F401
if hasattr(torch, 'xpu') and torch.xpu.is_available():
has_xpu = True
try:
xpu_name = torch.xpu.get_device_name(0)
except Exception:
xpu_name = "Intel GPU"
except ImportError:
pass
# DirectML backend (torch-directml) for any Windows GPU
has_directml = False
directml_name = None
try:
import torch_directml
if torch_directml.device_count() > 0:
has_directml = True
try:
directml_name = torch_directml.device_name(0)
except Exception:
directml_name = "DirectML GPU"
except ImportError:
pass
gpu_available = has_cuda or has_mps or has_xpu or has_directml or backend_type == "mlx"
gpu_type = None
if has_cuda:
@@ -89,6 +136,10 @@ async def health():
gpu_type = "MPS (Apple Silicon)"
elif backend_type == "mlx":
gpu_type = "Metal (Apple Silicon via MLX)"
elif has_xpu:
gpu_type = f"XPU ({xpu_name})"
elif has_directml:
gpu_type = f"DirectML ({directml_name})"
vram_used = None
if has_cuda:
@@ -252,12 +303,17 @@ async def add_profile_sample(
db: Session = Depends(get_db),
):
"""Add a sample to a voice profile."""
# Save uploaded file to temporary location
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
# Preserve the uploaded file's extension so librosa can detect format correctly.
# Defaulting to .wav was causing soundfile to reject MP3/WebM content as invalid WAV.
_allowed_audio_exts = {'.wav', '.mp3', '.m4a', '.ogg', '.flac', '.aac', '.webm', '.opus'}
_uploaded_ext = Path(file.filename or '').suffix.lower()
file_suffix = _uploaded_ext if _uploaded_ext in _allowed_audio_exts else '.wav'
with tempfile.NamedTemporaryFile(suffix=file_suffix, delete=False) as tmp:
content = await file.read()
tmp.write(content)
tmp_path = tmp.name
try:
sample = await profiles.add_profile_sample(
profile_id,
@@ -268,6 +324,8 @@ async def add_profile_sample(
return sample
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
except Exception as e:
raise HTTPException(status_code=500, detail=f"Failed to process audio file: {str(e)}")
finally:
# Clean up temp file
Path(tmp_path).unlink(missing_ok=True)
@@ -388,7 +446,7 @@ async def export_profile(
io.BytesIO(zip_bytes),
media_type="application/zip",
headers={
"Content-Disposition": f'attachment; filename="{filename}"'
"Content-Disposition": _safe_content_disposition("attachment", filename)
}
)
except ValueError as e:
@@ -542,47 +600,50 @@ async def generate_speech(
if not profile:
raise HTTPException(status_code=404, detail="Profile not found")
# Create voice prompt from profile
voice_prompt = await profiles.create_voice_prompt_for_profile(
data.profile_id,
db,
)
# Generate audio
# Resolve model size and load the correct model FIRST.
# This must happen before create_voice_prompt_for_profile because that
# function calls load_model_async(None), which falls back to self.model_size.
# If the model is already loaded with the right size at that point, it
# returns immediately and the voice prompt is created by the correct model.
tts_model = tts.get_tts_model()
# Load the requested model size if different from current (async to not block)
model_size = data.model_size or "1.7B"
# Check if model needs to be downloaded first
model_path = tts_model._get_model_path(model_size)
if model_path.startswith("Qwen/"):
# Model not cached - check if it exists remotely or needs download
from huggingface_hub import constants as hf_constants
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_path.replace("/", "--"))
if not repo_cache.exists():
# Start download in background
model_name = f"qwen-tts-{model_size}"
if not tts_model._is_model_cached(model_size):
# Model is not fully cached — kick off a background download and tell
# the client to retry once it's ready.
model_name = f"qwen-tts-{model_size}"
async def download_model_background():
try:
await tts_model.load_model_async(model_size)
except Exception as e:
task_manager.error_download(model_name, str(e))
async def download_model_background():
try:
await tts_model.load_model_async(model_size)
except Exception as e:
task_manager.error_download(model_name, str(e))
task_manager.start_download(model_name)
asyncio.create_task(download_model_background())
task_manager.start_download(model_name)
asyncio.create_task(download_model_background())
# Return 202 Accepted with download info
raise HTTPException(
status_code=202,
detail={
"message": f"Model {model_size} is being downloaded. Please wait and try again.",
"model_name": model_name,
"downloading": True
}
)
raise HTTPException(
status_code=202,
detail={
"message": f"Model {model_size} is being downloaded. Please wait and try again.",
"model_name": model_name,
"downloading": True,
},
)
# Load (or switch to) the requested model before building the voice prompt
await tts_model.load_model_async(model_size)
# Create voice prompt from profile (model is already loaded with correct size)
voice_prompt = await profiles.create_voice_prompt_for_profile(
data.profile_id,
db,
)
audio, sample_rate = await tts_model.generate(
data.text,
voice_prompt,
@@ -625,6 +686,59 @@ async def generate_speech(
raise HTTPException(status_code=500, detail=str(e))
@app.post("/generate/stream")
async def stream_speech(
data: models.GenerationRequest,
db: Session = Depends(get_db),
):
"""
Generate speech and stream the WAV audio directly without saving to disk.
Returns raw WAV bytes via a StreamingResponse so the client can start
playing audio before the entire file has been received. This endpoint
does NOT create a history entry — use /generate for that.
"""
profile = await profiles.get_profile(data.profile_id, db)
if not profile:
raise HTTPException(status_code=404, detail="Profile not found")
tts_model = tts.get_tts_model()
model_size = data.model_size or "1.7B"
if not tts_model._is_model_cached(model_size):
raise HTTPException(
status_code=400,
detail=f"Model {model_size} is not downloaded yet. Use /generate to trigger a download.",
)
# Load the correct model before building the voice prompt (fixes issue #96)
await tts_model.load_model_async(model_size)
voice_prompt = await profiles.create_voice_prompt_for_profile(data.profile_id, db)
audio, sample_rate = await tts_model.generate(
data.text,
voice_prompt,
data.language,
data.seed,
data.instruct,
)
wav_bytes = tts.audio_to_wav_bytes(audio, sample_rate)
async def _wav_stream():
# Yield in chunks so large responses don't block the event loop
chunk_size = 64 * 1024 # 64 KB
for i in range(0, len(wav_bytes), chunk_size):
yield wav_bytes[i : i + chunk_size]
return StreamingResponse(
_wav_stream(),
media_type="audio/wav",
headers={"Content-Disposition": 'attachment; filename="speech.wav"'},
)
# ============================================
# HISTORY ENDPOINTS
# ============================================
@@ -753,7 +867,7 @@ async def export_generation(
io.BytesIO(zip_bytes),
media_type="application/zip",
headers={
"Content-Disposition": f'attachment; filename="{filename}"'
"Content-Disposition": _safe_content_disposition("attachment", filename)
}
)
except ValueError as e:
@@ -786,7 +900,7 @@ async def export_generation_audio(
audio_path,
media_type="audio/wav",
headers={
"Content-Disposition": f'attachment; filename="{filename}"'
"Content-Disposition": _safe_content_disposition("attachment", filename)
}
)
@@ -1054,7 +1168,7 @@ async def export_story_audio(
io.BytesIO(audio_bytes),
media_type="audio/wav",
headers={
"Content-Disposition": f'attachment; filename="{filename}"'
"Content-Disposition": _safe_content_disposition("attachment", filename)
}
)
except HTTPException:
+7 -5
View File
@@ -19,15 +19,17 @@ def is_apple_silicon() -> bool:
def get_backend_type() -> Literal["mlx", "pytorch"]:
"""
Detect the best backend for the current platform.
Returns:
"mlx" on Apple Silicon (if MLX is available), "pytorch" otherwise
"mlx" on Apple Silicon (if MLX is available and functional), "pytorch" otherwise
"""
if is_apple_silicon():
try:
import mlx
import mlx.core # noqa: F401 — triggers native lib loading
return "mlx"
except ImportError:
# MLX not installed, fallback to PyTorch
except (ImportError, OSError, RuntimeError):
# MLX not installed, or native libraries failed to load inside a
# PyInstaller bundle (OSError on missing .dylib / .metallib).
# Fall through to PyTorch.
return "pytorch"
return "pytorch"
+22 -1
View File
@@ -22,7 +22,7 @@ from .database import (
)
from .utils.audio import validate_reference_audio, load_audio, save_audio
from .utils.images import validate_image, process_avatar
from .utils.cache import _get_cache_dir
from .utils.cache import _get_cache_dir, clear_profile_cache
from .tts import get_tts_model
from . import config
@@ -121,6 +121,10 @@ async def add_profile_sample(
db.commit()
db.refresh(db_sample)
# Invalidate combined audio cache for this profile
# Since a new sample was added, any cached combined audio is now stale
clear_profile_cache(profile_id)
return ProfileSampleResponse.model_validate(db_sample)
@@ -242,6 +246,9 @@ async def delete_profile(
if profile_dir.exists():
shutil.rmtree(profile_dir)
# Clean up combined audio cache files for this profile
clear_profile_cache(profile_id)
return True
@@ -263,6 +270,9 @@ async def delete_profile_sample(
if not sample:
return False
# Store profile_id before deleting
profile_id = sample.profile_id
# Delete audio file
audio_path = Path(sample.audio_path)
if audio_path.exists():
@@ -272,6 +282,10 @@ async def delete_profile_sample(
db.delete(sample)
db.commit()
# Invalidate combined audio cache for this profile
# Since the sample set changed, any cached combined audio is now stale
clear_profile_cache(profile_id)
return True
@@ -295,10 +309,17 @@ async def update_profile_sample(
if not sample:
return None
# Store profile_id before updating
profile_id = sample.profile_id
sample.reference_text = reference_text
db.commit()
db.refresh(sample)
# Invalidate combined audio cache for this profile
# Since the reference text changed, cache keys and combined text are now stale
clear_profile_cache(profile_id)
return ProfileSampleResponse.model_validate(sample)
+1
View File
@@ -18,6 +18,7 @@ qwen-tts>=0.0.5
librosa>=0.10.0
soundfile>=0.12.0
numpy>=1.24.0
numba>=0.60.0,<0.61.0
# Utilities
python-multipart>=0.0.6
-8
View File
@@ -32,11 +32,3 @@ def audio_to_wav_bytes(audio: np.ndarray, sample_rate: int) -> bytes:
sf.write(buffer, audio, sample_rate, format="WAV")
buffer.seek(0)
return buffer.read()
def audio_to_wav_bytes(audio: np.ndarray, sample_rate: int) -> bytes:
"""Convert audio array to WAV bytes."""
buffer = io.BytesIO()
sf.write(buffer, audio, sample_rate, format="WAV")
buffer.seek(0)
return buffer.read()
+35
View File
@@ -105,11 +105,46 @@ def clear_voice_prompt_cache() -> int:
deleted_count = 0
if cache_dir.exists():
# Delete prompt cache files
for cache_file in cache_dir.glob("*.prompt"):
try:
cache_file.unlink()
deleted_count += 1
except Exception as e:
print(f"Failed to delete cache file {cache_file}: {e}")
# Delete combined audio files
for audio_file in cache_dir.glob("combined_*.wav"):
try:
audio_file.unlink()
deleted_count += 1
except Exception as e:
print(f"Failed to delete combined audio file {audio_file}: {e}")
return deleted_count
def clear_profile_cache(profile_id: str) -> int:
"""
Clear cache files for a specific profile.
Args:
profile_id: Profile ID
Returns:
Number of cache files deleted
"""
cache_dir = _get_cache_dir()
deleted_count = 0
if cache_dir.exists():
# Delete combined audio files for this profile
pattern = f"combined_{profile_id}_*.wav"
for audio_file in cache_dir.glob(pattern):
try:
audio_file.unlink()
deleted_count += 1
except Exception as e:
print(f"Failed to delete combined audio file {audio_file}: {e}")
return deleted_count
+8 -3
View File
@@ -6,8 +6,13 @@ from PyInstaller.utils.hooks import copy_metadata
datas = []
hiddenimports = ['backend', 'backend.main', 'backend.config', 'backend.database', 'backend.models', 'backend.profiles', 'backend.history', 'backend.tts', 'backend.transcribe', 'backend.platform_detect', 'backend.backends', 'backend.backends.pytorch_backend', 'backend.utils.audio', 'backend.utils.cache', 'backend.utils.progress', 'backend.utils.hf_progress', 'backend.utils.validation', 'torch', 'transformers', 'fastapi', 'uvicorn', 'sqlalchemy', 'librosa', 'soundfile', 'qwen_tts', 'qwen_tts.inference', 'qwen_tts.inference.qwen3_tts_model', 'qwen_tts.inference.qwen3_tts_tokenizer', 'qwen_tts.core', 'qwen_tts.cli', 'pkg_resources.extern', 'backend.backends.mlx_backend', 'mlx', 'mlx.core', 'mlx.nn', 'mlx_audio', 'mlx_audio.tts', 'mlx_audio.stt']
datas += collect_data_files('qwen_tts')
datas += collect_data_files('mlx')
datas += collect_data_files('mlx_audio')
# Use collect_all (not collect_data_files) so native .dylib and .metallib
# files are bundled as binaries, not data. Without this, MLX raises OSError
# when loading Metal shaders inside the PyInstaller bundle.
from PyInstaller.utils.hooks import collect_all as _collect_all
_mlx_datas, _mlx_bins, _mlx_hidden = _collect_all('mlx')
_mlxa_datas, _mlxa_bins, _mlxa_hidden = _collect_all('mlx_audio')
datas += _mlx_datas + _mlxa_datas
datas += copy_metadata('qwen-tts')
hiddenimports += collect_submodules('qwen_tts')
hiddenimports += collect_submodules('jaraco')
@@ -18,7 +23,7 @@ hiddenimports += collect_submodules('mlx_audio')
a = Analysis(
['server.py'],
pathex=[],
binaries=[],
binaries=_mlx_bins + _mlxa_bins,
datas=datas,
hiddenimports=hiddenimports,
hookspath=[],
+1 -1
View File
@@ -5,7 +5,7 @@ description: "Welcome to Voicebox - the open-source voice synthesis studio"
## What is Voicebox?
Voicebox is a **local-first voice cloning studio** with DAW-like features for professional voice synthesis. Think of it as the **Ollama for voice** — download models, clone voices, and generate speech entirely on your machine.
Voicebox is a **local-first voice cloning studio** with DAW-like features for professional voice synthesis. Think of it as a **local, free and open-source alternative to ElevenLabs** — download models, clone voices, and generate speech entirely on your machine.
<Frame>
<img src="/images/app-screenshot-1.webp" alt="Voicebox App Screenshot" />
+964
View File
@@ -0,0 +1,964 @@
# TTS Provider Architecture
**Status:** Planned for v0.1.13
**Created:** 2025-01-31
**Problem:** GitHub 2GB release limit + poor UX for frequent updates requiring 2.4GB re-downloads
---
## Overview
Split the monolithic backend into modular components:
1. **Main App** (~150-200MB): Tauri + FastAPI backend + Whisper + UI/profiles/history
2. **TTS Providers** (downloadable plugins): Separate executables for model inference
This architecture solves:
- ✅ GitHub 2GB release artifact limit
- ✅ Frequent app updates without re-downloading large python binaries
- ✅ User choice of compute backend (CPU/GPU/Cloud)
- ✅ External provider support (OpenAI, custom servers)
- ✅ Future extensibility
---
## Architecture Diagram
```
┌─────────────────────────────────────────────────────────┐
│ Voicebox App (Tauri + Backend) ~150MB │
│ ├─ UI Layer (React) │
│ ├─ Backend (FastAPI) │
│ │ ├─ Voice Profiles │
│ │ ├─ Generation History │
│ │ ├─ Audio Editing / Stories │
│ │ └─ Provider Manager ◄──────────────┐ │
│ └─ Whisper (bundled, tiny ~50MB) │ │
└─────────────────────────────────────────┼────────────────┘
│
HTTP/IPC │
│
┌────────────────────────────────┼─────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌──────────────────┐
│ TTS Provider: │ │ TTS Provider: │ │ TTS Provider: │
│ PyTorch CPU │ │ PyTorch CUDA │ │ MLX (Apple) │
│ │ │ │ │ │
│ ~300MB │ │ ~2.4GB │ │ ~800MB │
│ │ │ │ │ │
│ Local inference │ │ GPU inference │ │ Metal inference │
└─────────────────┘ └─────────────────┘ └──────────────────┘
│ │ │
└────────────────────────┴─────────────────────┘
│
┌─────────────▼──────────────┐
│ Future Providers: │
│ • Remote Server │
│ • OpenAI API │
│ • ElevenLabs │
│ • Custom Docker Container │
└────────────────────────────┘
```
---
## Problem Statement
### Current Architecture Issues
**Monolithic Binary:**
- CPU version: ~295MB
- CUDA version: ~2.37GB
- GitHub releases: 2GB file size limit (BLOCKED)
- Updates require re-downloading entire binary
- Poor UX: update app → restart → download CUDA update → restart again
**User Pain Points:**
1. Cannot release CUDA version on GitHub (over 2GB)
2. Every app update forces 2.4GB re-download for GPU users
3. No flexibility (can't use OpenAI, remote servers, etc.)
4. Wastes bandwidth for small bug fixes
---
## Solution: Pluggable TTS Providers
### Component Breakdown
#### 1. Main App (voicebox.exe / .app / .AppImage)
**Size:** ~100-150MB
**Includes:**
- Tauri runtime + React UI
- FastAPI backend (pure Python, no PyTorch)
- Whisper model (tiny, ~50MB)
- SQLite database
- Profile/history/audio editing logic
- Provider management system
**Does NOT include:**
- PyTorch (CPU or CUDA)
- TTS models (Qwen3-TTS)
- Heavy ML dependencies
**Updates frequently:** UI fixes, feature additions, non-ML changes
---
#### 2. TTS Provider: PyTorch CPU
**Binary:** `tts-provider-pytorch-cpu.exe`
**Size:** ~200MB
**Includes:**
- PyTorch CPU build
- Qwen3-TTS package
- Transformers
- No CUDA libraries
**Download source:** Cloudflare R2
**Updates rarely:** Only when model code changes
---
#### 3. TTS Provider: PyTorch CUDA
**Binary:** `tts-provider-pytorch-cuda.exe`
**Size:** ~2.4GB
**Includes:**
- PyTorch CUDA build (cu121)
- Qwen3-TTS package
- CUDA runtime, cuDNN, cuBLAS
- Transformers
**Download source:** Cloudflare R2
**Platform:** Windows + Linux (NVIDIA GPU)
**Updates rarely:** Only when model code or CUDA version changes
---
#### 4. TTS Provider: MLX
**Binary:** `tts-provider-mlx`
**Size:** ~150MB
**Includes:**
- MLX framework
- MLX-optimized Qwen3-TTS
- Metal acceleration
**Platform:** macOS only (Apple Silicon)
**Download source:** Cloudflare R2
---
#### 5. TTS Provider: Remote
**Binary:** None (built-in config)
**Size:** 0MB
**How it works:**
- User provides URL to their own TTS server
- Backend proxies requests to that server
- Implements API spec from `EXTERNAL_PROVIDERS.md`
**Use cases:**
- AMD GPU users running their own server
- Team deployments with shared GPU server
- Cloud hosting (Modal, RunPod, Replicate)
---
#### 6. TTS Provider: OpenAI
**Binary:** None (API wrapper)
**Size:** 0MB
**How it works:**
- User provides OpenAI API key
- Backend wraps OpenAI Audio API
- Voice profiles map to OpenAI voices
**Benefits:**
- Zero local compute
- Pay-per-use
- Instant setup
---
## Communication Protocol
### Provider API Specification
All TTS providers must implement these endpoints:
#### POST /tts/generate
Generate speech from text.
**Request:**
```json
{
"text": "Hello world!",
"voice_prompt": {
/* voice prompt object */
},
"language": "en",
"seed": 12345,
"model_size": "1.7B"
}
```
**Response:**
```json
{
"audio": "base64-encoded-audio",
"sample_rate": 24000,
"duration": 2.5
}
```
#### POST /tts/create_voice_prompt
Create voice prompt from reference audio.
**Request:** (multipart/form-data)
- `audio`: Audio file
- `reference_text`: Transcript
**Response:**
```json
{
"voice_prompt": {
/* serialized prompt */
}
}
```
#### GET /tts/health
Health check.
**Response:**
```json
{
"status": "healthy",
"provider": "pytorch-cuda",
"version": "1.0.0",
"model": "Qwen3-TTS-12Hz-1.7B-Base",
"device": "cuda:0"
}
```
#### GET /tts/status
Model status.
**Response:**
```json
{
"model_loaded": true,
"model_size": "1.7B",
"available_sizes": ["0.6B", "1.7B"],
"gpu_available": true,
"vram_used_mb": 1234
}
```
---
## Backend Implementation
### Provider Manager
**File:** `backend/providers/__init__.py`
```python
class ProviderManager:
"""Manages TTS provider lifecycle."""
def __init__(self):
self.active_provider: Optional[Provider] = None
self.config = load_provider_config()
async def start_provider(self, provider_type: str) -> str:
"""Start a TTS provider process."""
if provider_type == "pytorch-cpu":
return await self._start_local_provider("tts-provider-pytorch-cpu.exe")
elif provider_type == "pytorch-cuda":
return await self._start_local_provider("tts-provider-pytorch-cuda.exe")
elif provider_type == "mlx":
return await self._start_local_provider("tts-provider-mlx")
elif provider_type == "remote":
return self.config["remote_url"]
elif provider_type == "openai":
return None # No subprocess, API wrapper
async def _start_local_provider(self, binary_name: str) -> str:
"""Start local provider subprocess."""
provider_path = get_provider_binary_path(binary_name)
if not provider_path.exists():
raise ProviderNotInstalledException(binary_name)
# Start subprocess on random port
port = get_free_port()
process = subprocess.Popen([
str(provider_path),
"--port", str(port),
"--data-dir", str(config.get_data_dir())
])
# Wait for provider to be ready
await wait_for_provider_health(f"http://localhost:{port}")
self.active_provider = Provider(process, port)
return f"http://localhost:{port}"
async def stop_provider(self):
"""Stop active provider."""
if self.active_provider:
self.active_provider.process.terminate()
self.active_provider = None
```
---
### Provider Abstraction
**File:** `backend/providers/base.py`
```python
class TTSProvider(ABC):
"""Abstract base for TTS providers."""
@abstractmethod
async def generate(
self,
text: str,
voice_prompt: dict,
language: str,
seed: Optional[int]
) -> tuple[np.ndarray, int]:
"""Generate speech audio."""
pass
@abstractmethod
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str
) -> dict:
"""Create voice prompt from reference audio."""
pass
```
**File:** `backend/providers/local.py`
```python
class LocalProvider(TTSProvider):
"""Provider that communicates with local subprocess via HTTP."""
def __init__(self, base_url: str):
self.base_url = base_url
self.client = httpx.AsyncClient()
async def generate(self, text, voice_prompt, language, seed):
response = await self.client.post(
f"{self.base_url}/tts/generate",
json={
"text": text,
"voice_prompt": voice_prompt,
"language": language,
"seed": seed
}
)
data = response.json()
audio = np.frombuffer(base64.b64decode(data["audio"]), dtype=np.float32)
return audio, data["sample_rate"]
```
**File:** `backend/providers/openai.py`
```python
class OpenAIProvider(TTSProvider):
"""Provider that wraps OpenAI Audio API."""
def __init__(self, api_key: str):
self.client = OpenAI(api_key=api_key)
async def generate(self, text, voice_prompt, language, seed):
# Map voice_prompt to OpenAI voice name
voice = map_profile_to_openai_voice(voice_prompt)
response = await self.client.audio.speech.create(
model="tts-1",
voice=voice,
input=text
)
# Convert to numpy array
audio_data = response.content
audio, sr = load_audio_from_bytes(audio_data)
return audio, sr
```
---
## Provider Installation
### Download Manager
**File:** `backend/providers/installer.py`
```python
class ProviderInstaller:
"""Handles provider download and installation."""
async def download_provider(self, provider_type: str):
"""Download provider binary from R2."""
binary_name = {
"pytorch-cpu": "tts-provider-pytorch-cpu.exe",
"pytorch-cuda": "tts-provider-pytorch-cuda.exe",
"mlx": "tts-provider-mlx"
}[provider_type]
download_url = f"https://downloads.voicebox.sh/providers/v{PROVIDER_VERSION}/{binary_name}"
# Download with progress tracking (reuse existing SSE system)
await download_with_progress(
url=download_url,
destination=get_provider_install_path(binary_name),
progress_key=f"provider-{provider_type}"
)
```
**Provider Storage Location:**
- Windows: `%APPDATA%/voicebox/providers/`
- macOS: `~/Library/Application Support/voicebox/providers/`
- Linux: `~/.local/share/voicebox/providers/`
---
## Frontend Implementation
### Provider Settings UI
**Component:** `app/src/components/ServerSettings/ProviderSettings.tsx`
```tsx
export function ProviderSettings() {
const [selectedProvider, setSelectedProvider] =
useState<ProviderType>("auto");
const {data: installedProviders} = useQuery({
queryKey: ["providers", "installed"],
queryFn: () => apiClient.getInstalledProviders(),
});
return (
<Card>
<CardHeader>
<CardTitle>TTS Provider</CardTitle>
<CardDescription>Choose how Voicebox generates speech</CardDescription>
</CardHeader>
<CardContent>
<RadioGroup
value={selectedProvider}
onValueChange={setSelectedProvider}
>
{/* Auto-detect */}
<div className="flex items-center space-x-2">
<RadioGroupItem value="auto" id="auto" />
<Label htmlFor="auto">
<div className="font-medium">Auto-detect (Recommended)</div>
<div className="text-sm text-muted-foreground">
Automatically choose the best available provider
</div>
</Label>
</div>
{/* PyTorch CUDA */}
<div className="flex items-center justify-between">
<div className="flex items-center space-x-2">
<RadioGroupItem
value="pytorch-cuda"
id="cuda"
disabled={!gpuAvailable}
/>
<Label htmlFor="cuda">
<div className="font-medium">PyTorch CUDA (NVIDIA GPU)</div>
<div className="text-sm text-muted-foreground">
4-5x faster inference on NVIDIA GPUs
</div>
</Label>
</div>
{!installedProviders?.includes("pytorch-cuda") && gpuAvailable && (
<Button
onClick={() => downloadProvider("pytorch-cuda")}
size="sm"
>
Download (2.4GB)
</Button>
)}
</div>
{/* PyTorch CPU */}
<div className="flex items-center justify-between">
<div className="flex items-center space-x-2">
<RadioGroupItem value="pytorch-cpu" id="cpu" />
<Label htmlFor="cpu">
<div className="font-medium">PyTorch CPU</div>
<div className="text-sm text-muted-foreground">
Works on any system, slower inference
</div>
</Label>
</div>
{!installedProviders?.includes("pytorch-cpu") && (
<Button onClick={() => downloadProvider("pytorch-cpu")} size="sm">
Download (300MB)
</Button>
)}
</div>
{/* MLX (macOS only) */}
{isMacOS && (
<div className="flex items-center justify-between">
<div className="flex items-center space-x-2">
<RadioGroupItem value="mlx" id="mlx" />
<Label htmlFor="mlx">
<div className="font-medium">MLX (Apple Silicon)</div>
<div className="text-sm text-muted-foreground">
Optimized for M1/M2/M3 chips
</div>
</Label>
</div>
{!installedProviders?.includes("mlx") && (
<Button onClick={() => downloadProvider("mlx")} size="sm">
Download (800MB)
</Button>
)}
</div>
)}
{/* Remote */}
<div className="space-y-2">
<div className="flex items-center space-x-2">
<RadioGroupItem value="remote" id="remote" />
<Label htmlFor="remote">
<div className="font-medium">Remote Server</div>
<div className="text-sm text-muted-foreground">
Connect to your own TTS server
</div>
</Label>
</div>
{selectedProvider === "remote" && (
<Input placeholder="http://your-server:8000" className="ml-6" />
)}
</div>
{/* OpenAI */}
<div className="space-y-2">
<div className="flex items-center space-x-2">
<RadioGroupItem value="openai" id="openai" />
<Label htmlFor="openai">
<div className="font-medium">OpenAI API</div>
<div className="text-sm text-muted-foreground">
Use OpenAI's TTS API (requires API key)
</div>
</Label>
</div>
{selectedProvider === "openai" && (
<Input type="password" placeholder="sk-..." className="ml-6" />
)}
</div>
</RadioGroup>
</CardContent>
</Card>
);
}
```
---
## File Structure
```
voicebox/
├── backend/
│ ├── main.py # Main FastAPI app (no TTS code)
│ ├── providers/
│ │ ├── __init__.py # ProviderManager
│ │ ├── base.py # TTSProvider ABC
│ │ ├── local.py # LocalProvider (subprocess)
│ │ ├── remote.py # RemoteProvider (HTTP)
│ │ ├── openai.py # OpenAIProvider (API wrapper)
│ │ └── installer.py # Provider download logic
│ ├── profiles.py # Voice profile management
│ ├── history.py # Generation history
│ ├── transcribe.py # Whisper (still bundled)
│ └── ... (other backend modules)
│
├── providers/
│ ├── pytorch-cpu/
│ │ ├── main.py # FastAPI server for TTS
│ │ ├── tts_backend.py # PyTorch TTS logic
│ │ ├── requirements.txt # torch (CPU), qwen-tts, transformers
│ │ └── build.spec # PyInstaller spec
│ │
│ ├── pytorch-cuda/
│ │ ├── main.py # FastAPI server for TTS
│ │ ├── tts_backend.py # PyTorch TTS logic
│ │ ├── requirements.txt # torch+cu121, qwen-tts, transformers
│ │ └── build.spec # PyInstaller spec
│ │
│ └── mlx/
│ ├── main.py # FastAPI server for TTS
│ ├── mlx_backend.py # MLX TTS logic
│ ├── requirements.txt # mlx, qwen-tts-mlx
│ └── build.spec # PyInstaller spec
│
├── app/ # Frontend (Tauri + React)
│ └── src/
│ └── components/
│ └── ServerSettings/
│ └── ProviderSettings.tsx
│
└── tauri/
└── src-tauri/
└── tauri.conf.json # No externalBin for providers
```
---
## Migration Path
### Phase 1: Refactor Backend (No User Changes)
**Goal:** Abstract TTS behind provider interface
1. Create `backend/providers/` module structure
2. Implement `TTSProvider` abstract base class
3. Create `LocalProvider` wrapper for current PyTorch code
4. Modify `backend/tts.py` to use provider abstraction
5. Keep PyTorch bundled in main app
**Result:** Code is prepared, but user experience unchanged
---
### Phase 2: Build Provider Binaries
**Goal:** Create standalone TTS provider executables
1. Create separate PyInstaller specs for each provider
2. Build provider executables:
- `tts-provider-pytorch-cpu.exe` (~300MB)
- `tts-provider-pytorch-cuda.exe` (~2.4GB)
- `tts-provider-mlx` (~800MB, macOS)
3. Test subprocess communication
4. Upload providers to Cloudflare R2
**Result:** Provider binaries exist but aren't used yet
---
### Phase 3: Remove PyTorch from Main App
**Goal:** Split main app from providers
1. Exclude PyTorch/Qwen3-TTS from main app PyInstaller spec
2. Main app now requires provider download
3. Update GitHub CI to build multiple artifacts:
- `voicebox-{version}-{platform}.exe` (~150MB)
- `tts-provider-pytorch-cpu-{version}.exe`
- `tts-provider-pytorch-cuda-{version}.exe`
- `tts-provider-mlx-{version}` (macOS)
**Result:** Main app is small, providers downloaded separately
---
### Phase 4: Add Provider UI
**Goal:** User-facing provider management
1. Create Provider Settings page
2. Implement provider download UI
3. Add provider status indicators
4. Show active provider in UI
**Result:** Users can choose and download providers
---
### Phase 5: External Providers
**Goal:** Enable remote and cloud providers
1. Implement `RemoteProvider` (HTTP client)
2. Implement `OpenAIProvider` (API wrapper)
3. Add provider configuration UI (URLs, API keys)
4. Document external provider API spec
**Result:** Full provider ecosystem
---
## Provider Versioning
### Independent Versioning
Providers have their own version numbers, independent of the main app:
- **App version:** `v0.2.0` (frequent updates)
- **Provider version:** `v1.0.0` (rare updates)
### Compatibility Matrix
**Example:**
| App Version | Min Provider Version | Max Provider Version |
| ----------- | -------------------- | -------------------- |
| v0.2.0 | v1.0.0 | v1.x.x |
| v0.3.0 | v1.0.0 | v1.x.x |
| v0.4.0 | v1.2.0 | v1.x.x |
| v1.0.0 | v2.0.0 | v2.x.x |
**Backend checks compatibility:**
```python
async def check_provider_compatibility(provider_version: str) -> bool:
"""Check if provider version is compatible with current app."""
min_version = "1.0.0"
max_version = "1.999.999"
return min_version <= provider_version < max_version
```
**UI shows warning if incompatible:**
```
⚠️ Provider version 0.9.0 is outdated. Update to v1.0.0+
```
---
## User Flows
### First-Time Setup
1. User downloads and installs Voicebox (~150MB)
2. App launches → detects no TTS provider installed
3. Shows setup wizard:
```
Choose your TTS provider:
[ ] PyTorch CUDA (2.4GB) [Download]
✓ Fastest on NVIDIA GPUs
✗ Requires NVIDIA GPU
[●] PyTorch CPU (300MB) [Download]
✓ Works on any system
✗ Slower inference
[ ] MLX (800MB) [Download]
✓ Fast on Apple Silicon
✗ macOS only (M1/M2/M3)
[ ] Remote Server
URL: ___________________
[ ] OpenAI API
API Key: ________________
```
4. User selects provider → downloads with progress bar
5. Provider installs to AppData/Application Support
6. App starts provider → ready to use
---
### App Update Flow (No Provider Change)
**Scenario:** Bug fix in UI, no backend changes
1. User gets update notification: "Voicebox v0.2.1 available"
2. Downloads update (~150MB, not 2.4GB!)
3. Installs and restarts
4. **Provider stays the same** (no re-download needed)
5. App starts using existing provider
**User experience:** Fast updates, no multi-GB downloads
---
### Provider Update Flow
**Scenario:** New Qwen3-TTS model version released
1. User opens Settings → Provider tab
2. Sees notification: "Provider update available (v1.1.0)"
3. Clicks "Update Provider"
4. Downloads new provider binary
5. Old provider binary is replaced
6. Restart app to use new provider
**Frequency:** Rare (only when TTS model/backend changes)
---
### Switching Providers
**Scenario:** User upgrades to NVIDIA GPU
1. User goes to Settings → Provider
2. Selects "PyTorch CUDA"
3. Clicks "Download" → downloads 2.4GB
4. Download completes → restarts app
5. App now uses CUDA provider
---
## Benefits
| Benefit | Details |
| ----------------------------- | --------------------------------------------------------- |
| **GitHub Releases Work** | Main app ~150MB << 2GB limit |
| **Fast Updates** | UI/feature updates don't require re-downloading providers |
| **User Choice** | CPU, CUDA, MLX, OpenAI, remote server |
| **External Provider Support** | Users can run their own TTS servers |
| **Bandwidth Savings** | Only download provider once, app updates are small |
| **Future-Proof** | Easy to add new providers (ElevenLabs, custom models) |
| **Team Deployments** | Multiple users share one remote provider |
| **Cloud-Ready** | Works with Modal, Replicate, RunPod, etc. |
---
## Open Questions
### 1. Provider Versioning
**Question:** Should providers have independent versions or match app version?
**Options:**
- A. Independent (providers: v1.x, app: v0.2.x)
- B. Matched (both use v0.2.x)
**Recommendation:** Independent versioning with compatibility matrix
---
### 2. Auto-Update Providers
**Question:** Should providers auto-update separately from app?
**Options:**
- A. Manual updates only (user clicks "Update Provider")
- B. Optional auto-update (user can enable)
- C. Always auto-update
**Recommendation:** Optional auto-update (default off)
---
### 3. Provider Discovery
**Question:** How does app find installed providers?
**Options:**
- A. Check standard paths in AppData/Application Support
- B. Registry (Windows) / plist (macOS)
- C. Config file with provider locations
**Recommendation:** Standard paths + config fallback
---
### 4. Fallback Behavior
**Question:** What if no provider is installed?
**Options:**
- A. Show setup wizard on first launch
- B. Block app until provider installed
- C. Allow app to run in "demo mode" (transcription only)
**Recommendation:** Setup wizard on first launch
---
### 5. Provider Auto-Start
**Question:** Should provider start automatically with app?
**Options:**
- A. Always start selected provider on app launch
- B. Start on-demand (when user generates speech)
- C. User preference
**Recommendation:** Auto-start (configurable in settings)
---
## Future Enhancements
- [ ] **Provider Marketplace:** Built-in directory of community providers
- [ ] **Multi-Provider Support:** Use different providers per voice/language
- [ ] **Provider Health Monitoring:** Automatic failover if provider crashes
- [ ] **Cost Tracking:** Monitor API usage for OpenAI/cloud providers
- [ ] **Performance Metrics:** Latency, throughput, VRAM usage dashboards
- [ ] **Docker Providers:** Run providers in Docker containers
- [ ] **Provider Plugins:** Load custom providers from user scripts
---
## Related Documents
- [EXTERNAL_PROVIDERS.md](./EXTERNAL_PROVIDERS.md) - External provider support plan
- [OPENAI_SUPPORT.md](./OPENAI_SUPPORT.md) - OpenAI API compatibility
- [github-2gb-limit-issue.md](../github-2gb-limit-issue.md) - Original problem
- [r2-setup.md](../r2-setup.md) - Cloudflare R2 configuration
---
## Contributing
If you want to build a custom TTS provider:
1. Implement the provider API spec (see above)
2. Test with Voicebox locally
3. Package as executable (PyInstaller, Docker, etc.)
4. Share in GitHub Discussions
**Questions?**
- GitHub Issues: [voicebox/issues](https://github.com/jamiepine/voicebox/issues)
- Discord: Coming soon
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@voicebox/landing",
"version": "0.1.11",
"version": "0.1.13",
"description": "Landing page for voicebox.sh",
"scripts": {
"dev": "bun --bun next dev --turbo",
+2
View File
@@ -1,6 +1,7 @@
import type { Metadata } from 'next';
import { Inter } from 'next/font/google';
import './globals.css';
import { Banner } from '@/components/Banner';
import { Footer } from '@/components/Footer';
import { Header } from '@/components/Header';
@@ -31,6 +32,7 @@ export default function RootLayout({ children }: { children: React.ReactNode })
<html lang="en" suppressHydrationWarning className="dark">
<body className={inter.variable}>
<div className="relative min-h-screen bg-background font-sans flex flex-col">
<Banner />
<Header />
<main className="container mx-auto px-4 sm:px-6 md:px-4 flex-1 py-4 sm:py-6 md:py-0">
{children}
+3 -2
View File
@@ -239,8 +239,9 @@ export default function Home() {
<div className="space-y-6 text-lg text-foreground/80 text-center">
<p>
Voicebox is a <strong>local-first voice cloning studio</strong> with DAW-like features
for professional voice synthesis. Think of it as the <strong>Ollama for voice</strong>{' '}
— download models, clone voices, and generate speech entirely on your machine.
for professional voice synthesis. Think of it as a{' '}
<strong>local, free and open-source alternative to ElevenLabs</strong> — download
models, clone voices, and generate speech entirely on your machine.
</p>
<p>
Unlike cloud services that lock your voice data behind subscriptions, Voicebox gives
+25
View File
@@ -0,0 +1,25 @@
import { ArrowRight } from 'lucide-react';
export function Banner() {
return (
<div className="bg-primary/[0.06] border-b border-border backdrop-blur-sm">
<div className="container mx-auto px-4">
<div className="flex items-center justify-center h-10 text-sm">
<a
href="https://spacebot.sh"
target="_blank"
rel="noopener noreferrer"
className="flex items-center gap-2 text-muted-foreground hover:text-foreground transition-colors group"
>
<span>
Also by the creator of Voicebox:{' '}
<strong className="text-foreground/90">Spacebot</strong>, an AI agent OS for teams.
Connect Discord, Slack, or Telegram in one click.
</span>
<ArrowRight className="h-3.5 w-3.5 transition-transform group-hover:translate-x-0.5" />
</a>
</div>
</div>
</div>
);
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "voicebox",
"version": "0.1.11",
"version": "0.1.13",
"private": true,
"workspaces": [
"app",
+9
View File
@@ -0,0 +1,9 @@
uvicorn
fastapi
sqlalchemy
torch
torchvision
soundfile
librosa
python-multipart
huggingface_hub
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@voicebox/tauri",
"private": true,
"version": "0.1.11",
"version": "0.1.13",
"type": "module",
"scripts": {
"dev": "vite",
+1 -1
View File
@@ -5041,7 +5041,7 @@ checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "voicebox"
version = "0.1.11"
version = "0.1.12"
dependencies = [
"base64 0.22.1",
"core-foundation-sys",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "voicebox"
version = "0.1.11"
version = "0.1.13"
description = "A production-quality desktop app for Qwen3-TTS voice cloning and generation"
authors = ["you"]
license = ""
@@ -0,0 +1,16 @@
use crate::audio_capture::AudioCaptureState;
pub async fn start_capture(
state: &AudioCaptureState,
max_duration_secs: u32,
) -> Result<(), String> {
todo!("implement Linux audio capture")
}
pub async fn stop_capture(state: &AudioCaptureState) -> Result<String, String> {
todo!("implement Linux audio capture stop")
}
pub fn is_supported() -> bool {
false
}
+4
View File
@@ -2,11 +2,15 @@
mod macos;
#[cfg(target_os = "windows")]
mod windows;
#[cfg(target_os = "linux")]
mod linux;
#[cfg(target_os = "macos")]
pub use macos::*;
#[cfg(target_os = "windows")]
pub use windows::*;
#[cfg(target_os = "linux")]
pub use linux::*;
use std::sync::{Arc, Mutex};
+2 -2
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "Voicebox",
"version": "0.1.11",
"version": "0.1.13",
"identifier": "sh.voicebox.app",
"build": {
"beforeDevCommand": "bun run dev",
@@ -12,7 +12,7 @@
"bundle": {
"active": true,
"targets": "all",
"createUpdaterArtifacts": true,
"createUpdaterArtifacts": false,
"externalBin": ["binaries/voicebox-server"],
"icon": [
"icons/32x32.png",
+18 -22
View File
@@ -2,29 +2,25 @@ import type { PlatformFilesystem, FileFilter } from '@/platform/types';
export const tauriFilesystem: PlatformFilesystem = {
async saveFile(filename: string, blob: Blob, filters?: FileFilter[]) {
try {
const { save } = await import('@tauri-apps/plugin-dialog');
const filePath = await save({
defaultPath: filename,
filters: filters || [],
});
const { save } = await import('@tauri-apps/plugin-dialog');
const { writeFile } = await import('@tauri-apps/plugin-fs');
if (filePath) {
const { writeBinaryFile } = await import('@tauri-apps/plugin-fs');
const arrayBuffer = await blob.arrayBuffer();
await writeBinaryFile(filePath, new Uint8Array(arrayBuffer));
}
} catch (error) {
console.error('Failed to use Tauri dialog, falling back to browser download:', error);
// Fall back to browser download if Tauri dialog fails
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
const filePath = await save({
defaultPath: filename,
filters: filters || [],
});
if (!filePath) return; // User cancelled the dialog
const resolvedPath = typeof filePath === 'string'
? filePath
: (filePath as { path: string }).path;
if (!resolvedPath) {
throw new Error('Failed to resolve save path from dialog');
}
const arrayBuffer = await blob.arrayBuffer();
await writeFile(resolvedPath, new Uint8Array(arrayBuffer));
},
};
+2 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@voicebox/web",
"private": true,
"version": "0.1.11",
"version": "0.1.13",
"type": "module",
"scripts": {
"dev": "vite",
@@ -21,6 +21,7 @@
"@types/react-dom": "^18.3.0",
"@typescript-eslint/eslint-plugin": "^7.0.0",
"@typescript-eslint/parser": "^7.0.0",
"@tailwindcss/vite": "^4.0.0",
"@vitejs/plugin-react": "^4.3.0",
"eslint": "^8.57.0",
"eslint-plugin-react-hooks": "^4.6.0",
+2 -1
View File
@@ -1,9 +1,10 @@
import path from 'node:path';
import react from '@vitejs/plugin-react';
import tailwindcss from '@tailwindcss/vite';
import { defineConfig } from 'vite';
export default defineConfig({
plugins: [react()],
plugins: [react(), tailwindcss()],
resolve: {
alias: {
'@': path.resolve(__dirname, '../app/src'),