Compare commits

...
Author SHA1 Message Date
Jamie Pine 38bf96ff20 fix: pin numba for release CI wheel compatibility 2026-02-23 12:18:34 -08:00
Jamie Pine 0b14cb1b2c Bump version: 0.1.12 → 0.1.13 2026-02-23 11:24:46 -08:00
Jamie Pine 4d24e69012 docs: point download links to latest release 2026-02-23 11:23:30 -08:00
Jamie PineandGitHub 90436e428d Merge pull request #77 from ManuLG/fix/broken-confirmation-modals
fix: await for confirmation before deleting voices and channels
2026-02-23 11:13:12 -08:00
Jamie PineandGitHub e4bb288904 Merge pull request #93 from iJaack/fix/mlx-apple-silicon-binary
fix(mlx): bundle native libs and broaden error handling for Apple Silicon
2026-02-23 11:12:34 -08:00
Jamie PineandGitHub 6f8bc7f23b Merge pull request #95 from CelebrityPunks/fix/model-size-selection-ignored
Fix: selecting 0.6B model still downloads and uses 1.7B
2026-02-23 11:12:12 -08:00
Jamie PineandGitHub baca111d50 Merge branch 'main' into fix/model-size-selection-ignored 2026-02-23 11:12:05 -08:00
Jamie PineandGitHub cc298fe6d8 Merge pull request #79 from martyniukyurii/fix/unicode-content-disposition
fix: handle non-ASCII filenames in Content-Disposition headers
2026-02-23 11:09:36 -08:00
Jamie PineandGitHub 46b8f6b882 Merge pull request #78 from tomasmach/fix/getUserMedia-undefined-check
fix: guard getUserMedia call against undefined mediaDevices in non-secure contexts
2026-02-23 11:09:23 -08:00
Jamie PineandGitHub 162cf4fb84 Merge pull request #122 from white1107/fix/web-tailwind-plugin
fix(web): add @tailwindcss/vite plugin to web config
2026-02-21 13:46:30 -08:00
Jamie PineandGitHub 68558243d9 Merge pull request #126 from lemassykoi/main
Create requirements.txt
2026-02-21 13:46:07 -08:00
Jamie PineandGitHub 8d5ad926f9 Merge pull request #128 from mrigankad/fix/voicebox-bugs
fix: resolve multiple issues (#96, #119, #111, #108, #121, #125, #127)
2026-02-21 13:45:19 -08:00
Jamie PineandGitHub 334f037dce Merge pull request #146 from xPolar/landing/spacebot-banner
Add Spacebot banner to landing page
2026-02-21 13:41:31 -08:00
xPolar f6522eea80 Add Spacebot banner to landing page
Adds a persistent top-of-page banner linking to spacebot.sh,
another project by the creator of Voicebox. Uses existing design
tokens for a consistent look.
2026-02-21 13:37:44 -08:00
lemassykoiandAmp 7615a08f81 ci: add Windows-only build workflow without signing
Amp-Thread-ID: https://ampcode.com/threads/T-019c7c3e-072f-7109-86a7-072a6b309891
Co-authored-by: Amp <[email protected]>
2026-02-20 23:06:45 +01:00
lemassykoiandAmp 31ea3c68a5 fix: remove silent browser fallback that bypasses save dialog path
Amp-Thread-ID: https://ampcode.com/threads/T-019c7c3e-072f-7109-86a7-072a6b309891
Co-authored-by: Amp <[email protected]>
2026-02-20 19:27:41 +01:00
Mriganka 54d72ddfd0 fix: resolve multiple issues (#96, #119, #111, #108, #121, #125, #127) 2026-02-20 23:23:38 +05:30
Clément PAPPALARDOandGitHub d4794f78e1 Create requirements.txt 2026-02-20 17:14:14 +01:00
white1107 aa7c9a9a8d fix(web): add @tailwindcss/vite plugin to web config
The web version was missing the Tailwind CSS Vite plugin, causing
CSS to not load at all. This adds the same plugin configuration
that exists in the tauri version.

Fixes #121
2026-02-20 20:11:27 +09:00
AbrahamandClaude Opus 4.6 ca6ed0998a Fix model size selection ignored when generating speech
The /generate endpoint created the voice prompt before loading the
user's requested model size. Since create_voice_prompt() internally
calls load_model_async(None), it fell back to the hardcoded default
of "1.7B", causing the 1.7B model to be downloaded even when the
user explicitly selected 0.6B.

This reorders the operations so the requested model is loaded first,
ensuring create_voice_prompt() and generate() use the correct model.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-18 09:41:44 -08:00
Eva 829d4d6d5b fix(mlx): bundle native libs and broaden error handling for Apple Silicon
The distributed macOS aarch64 binary shipped without MLX acceleration despite
the model and backend code supporting it. Two root causes:

1. **OSError not caught in platform_detect.py**
   PyInstaller bundles isolate the filesystem, so when MLX tries to load its
   Metal shader libraries (.metallib) it raises OSError, not ImportError.
   platform_detect.get_backend_type() only caught ImportError, causing a
   silent fallback to PyTorch even on Apple Silicon hardware.
   Fix: broaden the except clause to (ImportError, OSError, RuntimeError)
   and import mlx.core instead of mlx (forces native lib loading eagerly).

2. **collect_data_files used instead of collect_all for MLX**
   build_binary.py and voicebox-server.spec used --collect-data /
   collect_data_files for mlx and mlx_audio. This copies Python source and
   pure-Python data, but NOT native shared libraries (.dylib, .metallib).
   Fix: switch to --collect-all / collect_all which captures binaries too,
   then pass them to Analysis(binaries=...) in the spec.

Result: macOS Apple Silicon users now get MLX inference (~4-5x faster than
PyTorch CPU), matching the performance documented in the README.
2026-02-18 16:51:48 +01:00
YuriiandCursor 0be7975db5 fix: handle non-ASCII filenames in Content-Disposition headers
The export endpoints (export-audio, export generation, export profile,
export story) crash with `'latin-1' codec can't encode characters` when
the generated text or profile/story name contains non-ASCII characters
(e.g. Cyrillic, Chinese, Arabic).

Root cause: Python's `str.isalnum()` passes Unicode letters through to
the filename, but HTTP headers are encoded as latin-1 by the ASGI server,
which cannot represent characters outside the 0-255 range.

Fix: introduce `_safe_content_disposition()` helper that builds a
standards-compliant header with an ASCII-only `filename` fallback and a
RFC 5987 `filename*=UTF-8''...` parameter for Unicode-capable clients.

Fixes #68

Co-authored-by: Cursor <[email protected]>
2026-02-17 12:58:42 +04:00
tomasmach 40e4af828a fix: guard getUserMedia call against undefined mediaDevices in non-secure contexts 2026-02-17 09:28:23 +01:00
Manuel Lorenzo 0e57826ea5 fix: await for confirmation before deleting voices and channels 2026-02-17 00:29:52 +01:00
Spacedrive Mac Mini 2 eb2cd861b1 chore: update Cargo.lock version to 0.1.12 2026-02-10 06:59:41 -08:00
Jamie PineandGitHub 701cc647a7 Merge pull request #57 from selop/chore/readme
chore: updates repo URL in README
2026-02-06 05:08:24 -08:00
Sergej Lopatkin be6ccaf044 chore: updates repo URL in README
Updates the repository URL in the README to point to the correct fork.

Adds a prerequisite for XCode on macOS for development.
2026-02-06 13:22:59 +01:00
Jamie PineandGitHub 1040625a88 Merge pull request #44 from selop/feature/delivery-instructions
Enhances floating generate box UX
2026-02-02 17:58:19 -08:00
Sergej Lopatkin 6f4503b521 Enhances floating generate box UX
- Adds tooltips on hover for buttons of the generate box
- Replaces the message square icon with a sliders icon for the instruction mode toggle.
- Adds a tooltip to the instruction mode toggle button.
- Updates the placeholder text for the input field.
2026-02-02 22:19:54 +01:00
Sergej LopatkinandGitHub f5b6edc2e7 Merge pull request #1 from jamiepine/main
update fork
2026-02-02 22:19:30 +01:00
Jamie PineandGitHub 8197f0724c Merge pull request #40 from Spyabo/fix/audio-export-path-resolution
Fix: audio export path resolution
2026-02-02 06:54:39 -08:00
Reese Wright d40f7d2676 refactor: improve path resolution readability 2026-02-02 14:54:05 +00:00
Reese Wright 99fbcca7f4 update CHANGELOG for audio export fix 2026-02-02 14:34:39 +00:00
Reese Wright 04f9880c9a fix audio export path resolution 2026-02-02 14:26:34 +00:00
Jamie Pine b9c858295d Update Voicebox description as an alternative to ElevenLabs, rather than Ollama 2026-02-01 00:45:47 -08:00
Jamie Pine 610f64c762 fix linux compile 2026-01-31 07:44:28 -08:00
Jamie Pine 220333b3bb corrections 2026-01-31 02:15:45 -08:00
Jamie Pine e194e95512 corrections 2026-01-31 02:14:37 -08:00
Jamie Pine e796412c2c corrections 2026-01-31 02:13:42 -08:00
Jamie Pine cb541521d2 Update TTS Provider Architecture status to v0.1.13 2026-01-31 02:11:41 -08:00
Jamie Pine 2bc243f93e Add TTS Provider Architecture plan
Solves GitHub 2GB limit + frequent update UX issues by splitting app into:
- Main app (~150MB): UI + backend logic + Whisper
- TTS Providers (plugins): Separate downloadable binaries
  - pytorch-cpu (~300MB)
  - pytorch-cuda (~2.4GB)
  - mlx (~800MB, macOS)
  - remote (connect to external server)
  - openai (API wrapper)

Benefits:
- Main app under GitHub 2GB limit
- Updates don't require re-downloading providers
- User choice of compute backend
- External provider support for teams/cloud
- Future-proof extensibility
2026-01-31 02:09:42 -08:00
Jamie Pine 0209008d73 disable cuda for 0.1.12 2026-01-31 01:46:14 -08:00
Jamie Pine 9bde534860 Bump version: 0.1.11 → 0.1.12 2026-01-30 21:23:07 -08:00
Jamie PineandGitHub 97eb570b28 Merge pull request #25 from jamiepine/fix-dl-notification-when-generating-from-already-cached-model
Fix dl notification when generating from already cached model
2026-01-30 21:20:25 -08:00
Jamie PineandGitHub 7d0557a099 Merge pull request #27 from jamiepine/model-dl-fix
Enhance model caching checks and progress tracking for downloads
2026-01-30 21:19:52 -08:00
Jamie Pine 60a03c56a9 Enhance model caching checks and progress tracking for downloads
- Updated caching methods in MLX, PyTorch, and backend to ensure models are fully downloaded before being marked as cached.
- Improved progress tracking to filter out non-download progress and provide accurate feedback during model downloads.
- Enhanced HFProgressTracker to skip non-byte progress bars and ensure meaningful progress reporting.
- Refactored progress initialization to provide immediate feedback while fetching metadata from HuggingFace.
- Added error handling and logging for better debugging during cache checks and download processes.
2026-01-30 21:17:01 -08:00
Jamie Pine d3393fb940 Refactor model download progress tracking and enhance SSE handling
- Rearranged imports for consistency in useModelDownloadToast hook.
- Improved logging in useModelDownloadToast for better debugging during download events.
- Updated progress calculation to handle cases where progress exceeds 100%.
- Enhanced toast notifications to reflect download completion and error states.
- Introduced throttling in ProgressManager to optimize SSE updates and prevent overwhelming clients.
- Added new test scripts for monitoring SSE events during model downloads, ensuring accurate progress reporting.
2026-01-30 20:18:53 -08:00
Jamie Pine 07c0aba883 Refactor model download handling and improve progress tracking
- Rearranged imports for consistency across components.
- Enhanced the ModelManagement component to include detailed logging for download actions and errors.
- Updated the ModelProgress component to connect to SSE only when actively downloading, preventing connection exhaustion.
- Added a downloading state to the model status to indicate ongoing downloads.
- Improved toast notifications for model downloads with completion and error callbacks.
- Refactored the useModelDownloadToast hook to support new callbacks for download completion and error handling.
- Updated backend model status to reflect downloading state during active downloads.
2026-01-30 19:53:20 -08:00
Jamie Pine 77418a52ae Update release workflow and model references
- Added a step to install PyTorch with CUDA for Windows in the release workflow.
- Updated model references in backend/main.py to use openai/whisper models instead of mlx-community for the MLX backend.
2026-01-30 18:10:17 -08:00
Jamie Pine 46f6806e14 Update versions and implement auto-update feature
- Bumped version numbers for @voicebox/app, @voicebox/landing, @voicebox/tauri, and @voicebox/web to 0.1.11.
- Added a new `useAutoUpdater` hook to check for app updates on startup and notify users with toast messages.
- Enhanced `UpdateStatus` component to handle version retrieval errors more gracefully.
- Updated dependencies in `package.json` for Tauri plugins to support new update functionalities.
2026-01-30 18:02:28 -08:00
Jamie PineandGitHub 20851ccc2b Merge pull request #24 from jamiepine/fix-multi-sample
Fix multi sample
2026-01-30 17:07:53 -08:00
Jamie Pine 0b17073345 Add test suite for Voicebox backend
- Introduced a new directory for manual test scripts aimed at debugging and validating backend functionality.
- Added README.md detailing the purpose and usage of various test scripts, including tests for TTS generation, model downloads, and progress tracking.
- Included an __init__.py file to define the test suite structure and provide context for the tests.
2026-01-30 16:48:14 -08:00
Jamie Pine 17106b1e40 Add progress tracking and caching checks for model downloads
- Introduced methods to check if models are cached locally in MLX and PyTorch backends.
- Enhanced progress tracking during model loading to filter out non-download progress when models are cached.
- Updated HFProgressTracker to conditionally report progress based on download status.
- Added test scripts for monitoring SSE events during model downloads and verifying progress tracking functionality.
- Improved overall error handling and logging for better debugging during model download processes.
2026-01-30 16:47:54 -08:00
Jamie PineandGitHub 7fcca09f24 Merge pull request #23 from jamiepine/audio-export-entitlement-fix
Audio export entitlement fix
2026-01-30 15:08:50 -08:00
62 changed files with 3744 additions and 504 deletions
+1 -1
View File
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.1.11
current_version = 0.1.13
commit = True
tag = True
tag_name = v{new_version}
+63
View File
@@ -0,0 +1,63 @@
name: Build Windows
on:
workflow_dispatch:
jobs:
build-windows:
permissions:
contents: write
runs-on: windows-latest
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: "pip"
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install -r backend/requirements.txt
- name: Build Python server
shell: bash
run: |
cd backend
python build_binary.py
PLATFORM=$(rustc --print host-tuple)
mkdir -p ../tauri/src-tauri/binaries
cp dist/voicebox-server.exe ../tauri/src-tauri/binaries/voicebox-server-${PLATFORM}.exe
echo "Built voicebox-server-${PLATFORM}.exe"
- name: Setup Bun
uses: oven-sh/setup-bun@v2
- name: Install Rust stable
uses: dtolnay/rust-toolchain@stable
- name: Rust cache
uses: swatinem/rust-cache@v2
with:
workspaces: "./tauri/src-tauri -> target"
- name: Install dependencies
run: bun install
- uses: tauri-apps/tauri-action@v0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
projectPath: tauri
tagName: v__VERSION__
releaseName: "voicebox v__VERSION__ (test build)"
releaseBody: "Test build for audio export fix"
releaseDraft: true
prerelease: true
args: ""
includeUpdaterJson: false
+22 -16
View File
@@ -4,7 +4,7 @@ on:
workflow_dispatch:
push:
tags:
- 'v*'
- "v*"
jobs:
release:
@@ -14,22 +14,22 @@ jobs:
fail-fast: false
matrix:
include:
- platform: 'macos-latest'
args: '--target aarch64-apple-darwin'
python-version: '3.12'
backend: 'mlx'
- platform: 'macos-15-intel'
args: '--target x86_64-apple-darwin'
python-version: '3.12'
backend: 'pytorch'
- platform: "macos-latest"
args: "--target aarch64-apple-darwin"
python-version: "3.12"
backend: "mlx"
- platform: "macos-15-intel"
args: "--target x86_64-apple-darwin"
python-version: "3.12"
backend: "pytorch"
# - platform: 'ubuntu-22.04'
# args: ''
# python-version: '3.12'
# backend: 'pytorch'
- platform: 'windows-latest'
args: ''
python-version: '3.12'
backend: 'pytorch'
- platform: "windows-latest"
args: ""
python-version: "3.12"
backend: "pytorch"
runs-on: ${{ matrix.platform }}
@@ -53,7 +53,7 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache: "pip"
- name: Install Python dependencies
run: |
@@ -66,6 +66,12 @@ jobs:
run: |
pip install -r backend/requirements-mlx.txt
# - name: Install PyTorch with CUDA (Windows only)
# if: matrix.platform == 'windows-latest'
# run: |
# pip install torch --index-url https://download.pytorch.org/whl/cu121 --force-reinstall --no-deps
# pip install torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
- name: Build Python server (Linux/macOS)
if: matrix.platform != 'windows-latest'
run: |
@@ -100,7 +106,7 @@ jobs:
- name: Rust cache
uses: swatinem/rust-cache@v2
with:
workspaces: './tauri/src-tauri -> target'
workspaces: "./tauri/src-tauri -> target"
- name: Install dependencies
run: bun install
@@ -136,7 +142,7 @@ jobs:
with:
projectPath: tauri
tagName: v__VERSION__
releaseName: 'voicebox v__VERSION__'
releaseName: "voicebox v__VERSION__"
releaseBody: |
## What's Changed
See the assets below to download and install this version.
+3
View File
@@ -53,6 +53,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Fixed
- Audio export failing when Tauri save dialog returns object instead of string path
### Added
- **Makefile** - Comprehensive development workflow automation with commands for setup, development, building, testing, and code quality checks
- Includes Python version detection and compatibility warnings
+8 -8
View File
@@ -59,7 +59,7 @@
## What is Voicebox?
Voicebox is a **local-first voice cloning studio** with DAW-like features for professional voice synthesis. Think of it as the **Ollama for voice** — download models, clone voices, and generate speech entirely on your machine.
Voicebox is a **local-first voice cloning studio** with DAW-like features for professional voice synthesis. Think of it as a **local, free and open-source alternative to ElevenLabs** — download models, clone voices, and generate speech entirely on your machine.
Unlike cloud services that lock your voice data behind subscriptions, Voicebox gives you:
@@ -80,10 +80,10 @@ Voicebox is available now for macOS and Windows.
| Platform | Download |
|----------|----------|
| macOS (Apple Silicon) | [voicebox_aarch64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_aarch64.app.tar.gz) |
| macOS (Intel) | [voicebox_x64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_x64.app.tar.gz) |
| Windows (MSI) | [voicebox_0.1.0_x64_en-US.msi](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_0.1.0_x64_en-US.msi) |
| Windows (Setup) | [voicebox_0.1.0_x64-setup.exe](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_0.1.0_x64-setup.exe) |
| macOS (Apple Silicon) | [Voicebox_aarch64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/latest/download/Voicebox_aarch64.app.tar.gz) |
| macOS (Intel) | [Voicebox_x64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/latest/download/Voicebox_x64.app.tar.gz) |
| Windows (MSI) | [Latest Windows MSI](https://github.com/jamiepine/voicebox/releases/latest) |
| Windows (Setup) | [Latest Windows Setup](https://github.com/jamiepine/voicebox/releases/latest) |
> **Linux builds coming soon** — Currently blocked by GitHub runner disk space limitations.
@@ -233,7 +233,7 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed setup and contribution guide
```bash
# Clone the repo
git clone https://github.com/voicebox-sh/voicebox.git
git clone https://github.com/jamiepine/voicebox.git
cd voicebox
# Setup everything
@@ -247,7 +247,7 @@ make dev
```bash
# Clone the repo
git clone https://github.com/voicebox-sh/voicebox.git
git clone https://github.com/jamiepine/voicebox.git
cd voicebox
# Install dependencies
@@ -260,7 +260,7 @@ cd backend && pip install -r requirements.txt && cd ..
bun run dev
```
**Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org).
**Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org). [XCode on macOS](https://developer.apple.com/xcode/).
**Performance:**
- **Apple Silicon (M1/M2/M3)**: Uses MLX backend with native Metal acceleration for 4-5x faster inference
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@voicebox/app",
"version": "0.1.11",
"version": "0.1.13",
"private": true,
"type": "module",
"scripts": {
+15 -5
View File
@@ -1,13 +1,14 @@
import { useEffect, useRef, useState } from 'react';
import { RouterProvider } from '@tanstack/react-router';
import { useEffect, useRef, useState } from 'react';
import voiceboxLogo from '@/assets/voicebox-logo.png';
import ShinyText from '@/components/ShinyText';
import { TitleBarDragRegion } from '@/components/TitleBarDragRegion';
import { useAutoUpdater } from '@/hooks/useAutoUpdater';
import { TOP_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { cn } from '@/lib/utils/cn';
import { usePlatform } from '@/platform/PlatformContext';
import { router } from '@/router';
import { useServerStore } from '@/stores/serverStore';
import { usePlatform } from '@/platform/PlatformContext';
const LOADING_MESSAGES = [
'Warming up tensors...',
@@ -38,6 +39,9 @@ function App() {
const [loadingMessageIndex, setLoadingMessageIndex] = useState(0);
const serverStartingRef = useRef(false);
// Automatically check for app updates on startup and show toast notifications
useAutoUpdater({ checkOnMount: true, showToast: true });
// Sync stored setting to Rust on startup
useEffect(() => {
if (platform.metadata.isTauri) {
@@ -46,14 +50,18 @@ function App() {
console.error('Failed to sync initial setting to Rust:', error);
});
}
}, [platform]);
// Empty dependency array - platform is stable from context, only run once
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.metadata.isTauri, platform.lifecycle]);
// Setup lifecycle callbacks
useEffect(() => {
platform.lifecycle.onServerReady = () => {
setServerReady(true);
};
}, [platform]);
// Empty dependency array - platform is stable from context, only run once
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.lifecycle]);
// Setup window close handler and auto-start server when running in Tauri (production only)
useEffect(() => {
@@ -111,7 +119,9 @@ function App() {
// Window close event handles server shutdown based on setting
serverStartingRef.current = false;
};
}, [platform]);
// Empty dependency array - platform is stable from context, only run once
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.metadata.isTauri, platform.lifecycle]);
// Cycle through loading messages every 3 seconds
useEffect(() => {
+8 -6
View File
@@ -124,6 +124,13 @@ export function AudioTab() {
);
}
const handleChannelDelete = async (e, channelId) => {
e.stopPropagation();
if (await confirm('Delete this channel?')) {
deleteChannel.mutate(channelId);
}
}
const allChannels = channels || [];
const allDevices = devices || [];
const selectedChannel = selectedChannelId
@@ -241,12 +248,7 @@ export function AudioTab() {
variant="ghost"
size="sm"
className="h-8 w-8 p-0"
onClick={(e) => {
e.stopPropagation();
if (confirm('Delete this channel?')) {
deleteChannel.mutate(channel.id);
}
}}
onClick={(e) => handleChannelDelete(e, channel.id)}
>
<Trash2 className="h-4 w-4" />
</Button>
@@ -1,6 +1,6 @@
import { useMatchRoute } from '@tanstack/react-router';
import { AnimatePresence, motion } from 'framer-motion';
import { Loader2, MessageSquare, Sparkles } from 'lucide-react';
import { Loader2, SlidersHorizontal, Sparkles } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Form, FormControl, FormField, FormItem, FormMessage } from '@/components/ui/form';
@@ -187,7 +187,7 @@ export function FloatingGenerateBox({
}}
>
<motion.div
className="bg-background/30 backdrop-blur-2xl border border-accent/20 rounded-[2rem] shadow-2xl hover:bg-background/40 hover:border-accent/20 transition-all duration-300 overflow-hidden p-3"
className="bg-background/30 backdrop-blur-2xl border border-accent/20 rounded-[2rem] shadow-2xl hover:bg-background/40 hover:border-accent/20 transition-all duration-300 p-3"
transition={{ duration: 0.6, ease: 'easeInOut' }}
>
<Form {...form}>
@@ -274,7 +274,7 @@ export function FloatingGenerateBox({
field.ref(node);
}
}}
placeholder="Add delivery instructions..."
placeholder="e.g. very happy and excited"
className="resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm placeholder:text-muted-foreground/60 w-full"
style={{
minHeight: isExpanded ? '100px' : '32px',
@@ -294,18 +294,27 @@ export function FloatingGenerateBox({
</motion.div>
<div className="relative shrink-0">
<Button
type="submit"
disabled={isPending || !selectedProfileId}
className="h-10 w-10 rounded-full bg-accent hover:bg-accent/90 hover:scale-105 text-accent-foreground shadow-lg hover:shadow-accent/50 transition-all duration-200"
size="icon"
>
{isPending ? (
<Loader2 className="h-4 w-4 animate-spin" />
) : (
<Sparkles className="h-4 w-4" />
)}
</Button>
<div className="group relative">
<Button
type="submit"
disabled={isPending || !selectedProfileId}
className="h-10 w-10 rounded-full bg-accent hover:bg-accent/90 hover:scale-105 text-accent-foreground shadow-lg hover:shadow-accent/50 transition-all duration-200"
size="icon"
>
{isPending ? (
<Loader2 className="h-4 w-4 animate-spin" />
) : (
<Sparkles className="h-4 w-4" />
)}
</Button>
<span className="pointer-events-none absolute bottom-full left-1/2 -translate-x-1/2 mb-2 whitespace-nowrap rounded-md bg-popover px-3 py-1.5 text-xs text-popover-foreground border border-border opacity-0 transition-opacity group-hover:opacity-100 z-[9999]">
{isPending
? 'Generating...'
: !selectedProfileId
? 'Select a voice profile first'
: 'Generate speech'}
</span>
</div>
<AnimatePresence>
{isExpanded && (
<motion.div
@@ -315,20 +324,25 @@ export function FloatingGenerateBox({
transition={{ duration: 0.2 }}
className="absolute top-0 right-[calc(100%+0.5rem)]"
>
<Button
type="button"
variant="ghost"
size="icon"
onClick={() => setIsInstructMode(!isInstructMode)}
className={cn(
'h-10 w-10 rounded-full transition-all duration-200',
isInstructMode
? 'bg-accent text-accent-foreground border border-accent hover:bg-accent/90'
: 'bg-card border border-border hover:bg-background/50',
)}
>
<MessageSquare className="h-4 w-4" />
</Button>
<div className="group relative">
<Button
type="button"
variant="ghost"
size="icon"
onClick={() => setIsInstructMode(!isInstructMode)}
className={cn(
'h-10 w-10 rounded-full transition-all duration-200',
isInstructMode
? 'bg-accent text-accent-foreground border border-accent hover:bg-accent/90'
: 'bg-card border border-border hover:bg-background/50',
)}
>
<SlidersHorizontal className="h-4 w-4" />
</Button>
<span className="pointer-events-none absolute bottom-full left-1/2 -translate-x-1/2 mb-2 whitespace-nowrap rounded-md bg-popover px-3 py-1.5 text-xs text-popover-foreground border border-border opacity-0 transition-opacity group-hover:opacity-100 z-[9999]">
Fine tune instructions
</span>
</div>
</motion.div>
)}
</AnimatePresence>
+16 -3
View File
@@ -1,6 +1,13 @@
import { AudioWaveform, Download, FileArchive, Loader2, MoreHorizontal, Play, Trash2 } from 'lucide-react';
import {
AudioWaveform,
Download,
FileArchive,
Loader2,
MoreHorizontal,
Play,
Trash2,
} from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import type { HistoryResponse } from '@/lib/api/types';
import { Button } from '@/components/ui/button';
import {
Dialog,
@@ -19,6 +26,7 @@ import {
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import type { HistoryResponse } from '@/lib/api/types';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import {
useDeleteGeneration,
@@ -50,7 +58,11 @@ export function HistoryTable() {
const limit = 20;
const { toast } = useToast();
const { data: historyData, isLoading, isFetching } = useHistory({
const {
data: historyData,
isLoading,
isFetching,
} = useHistory({
limit,
offset: page * limit,
});
@@ -280,6 +292,7 @@ export function HistoryTable() {
<Textarea
value={gen.text}
className="flex-1 resize-none text-sm text-muted-foreground select-text"
readOnly
/>
</div>
@@ -1,6 +1,6 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { Download, Loader2, Trash2 } from 'lucide-react';
import { useState } from 'react';
import { useCallback, useState } from 'react';
import {
AlertDialog,
AlertDialogAction,
@@ -17,7 +17,6 @@ import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/com
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
import { ModelProgress } from './ModelProgress';
export function ModelManagement() {
const { toast } = useToast();
@@ -27,15 +26,36 @@ export function ModelManagement() {
const { data: modelStatus, isLoading } = useQuery({
queryKey: ['modelStatus'],
queryFn: () => apiClient.getModelStatus(),
queryFn: async () => {
console.log('[Query] Fetching model status');
const result = await apiClient.getModelStatus();
console.log('[Query] Model status fetched:', result);
return result;
},
refetchInterval: 5000, // Refresh every 5 seconds
});
// Callbacks for download completion
const handleDownloadComplete = useCallback(() => {
console.log('[ModelManagement] Download complete, clearing state');
setDownloadingModel(null);
setDownloadingDisplayName(null);
queryClient.invalidateQueries({ queryKey: ['modelStatus'] });
}, [queryClient]);
const handleDownloadError = useCallback(() => {
console.log('[ModelManagement] Download error, clearing state');
setDownloadingModel(null);
setDownloadingDisplayName(null);
}, []);
// Use progress toast hook for the downloading model
useModelDownloadToast({
modelName: downloadingModel || '',
displayName: downloadingDisplayName || '',
enabled: !!downloadingModel && !!downloadingDisplayName,
onComplete: handleDownloadComplete,
onError: handleDownloadError,
});
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
@@ -45,44 +65,69 @@ export function ModelManagement() {
sizeMb?: number;
} | null>(null);
const downloadMutation = useMutation({
mutationFn: (modelName: string) => {
const handleDownload = async (modelName: string) => {
console.log('[Download] Button clicked for:', modelName, 'at', new Date().toISOString());
// Find display name
const model = modelStatus?.models.find((m) => m.model_name === modelName);
const displayName = model?.display_name || modelName;
try {
// IMPORTANT: Call the API FIRST before setting state
// Setting state enables the SSE EventSource in useModelDownloadToast,
// which can block/delay the download fetch due to HTTP/1.1 connection limits
console.log('[Download] Calling download API for:', modelName);
const result = await apiClient.triggerModelDownload(modelName);
console.log('[Download] Download API responded:', result);
// NOW set state to enable SSE tracking (after download has started on backend)
setDownloadingModel(modelName);
// Find display name from model status
const model = modelStatus?.models.find((m) => m.model_name === modelName);
setDownloadingDisplayName(model?.display_name || modelName);
return apiClient.triggerModelDownload(modelName);
},
onSuccess: () => {
// Download completed - clear state and refetch status
setDownloadingModel(null);
setDownloadingDisplayName(null);
setDownloadingDisplayName(displayName);
// Download initiated successfully - state will be cleared when SSE reports completion
// or by the polling interval detecting the model is downloaded
queryClient.invalidateQueries({ queryKey: ['modelStatus'] });
},
onError: (error: Error) => {
} catch (error) {
console.error('[Download] Download failed:', error);
setDownloadingModel(null);
setDownloadingDisplayName(null);
toast({
title: 'Download failed',
description: error.message,
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
},
});
}
};
const deleteMutation = useMutation({
mutationFn: (modelName: string) => apiClient.deleteModel(modelName),
onSuccess: () => {
mutationFn: async (modelName: string) => {
console.log('[Delete] Deleting model:', modelName);
const result = await apiClient.deleteModel(modelName);
console.log('[Delete] Model deleted successfully:', modelName);
return result;
},
onSuccess: async (_data, _modelName) => {
console.log('[Delete] onSuccess - showing toast and invalidating queries');
toast({
title: 'Model deleted',
description: `${modelToDelete?.displayName || 'Model'} has been deleted successfully.`,
});
setDeleteDialogOpen(false);
setModelToDelete(null);
// Refetch status to update UI
queryClient.invalidateQueries({ queryKey: ['modelStatus'] });
// Invalidate AND explicitly refetch to ensure UI updates
// Using refetchType: 'all' ensures we refetch even if the query is stale
console.log('[Delete] Invalidating modelStatus query');
await queryClient.invalidateQueries({
queryKey: ['modelStatus'],
refetchType: 'all',
});
// Also explicitly refetch to guarantee fresh data
console.log('[Delete] Explicitly refetching modelStatus query');
await queryClient.refetchQueries({ queryKey: ['modelStatus'] });
console.log('[Delete] Query refetched');
},
onError: (error: Error) => {
console.log('[Delete] onError:', error);
toast({
title: 'Delete failed',
description: error.message,
@@ -124,7 +169,7 @@ export function ModelManagement() {
<ModelItem
key={model.model_name}
model={model}
onDownload={() => downloadMutation.mutate(model.model_name)}
onDownload={() => handleDownload(model.model_name)}
onDelete={() => {
setModelToDelete({
name: model.model_name,
@@ -152,7 +197,7 @@ export function ModelManagement() {
<ModelItem
key={model.model_name}
model={model}
onDownload={() => downloadMutation.mutate(model.model_name)}
onDownload={() => handleDownload(model.model_name)}
onDelete={() => {
setModelToDelete({
name: model.model_name,
@@ -168,21 +213,6 @@ export function ModelManagement() {
</div>
</div>
{/* Progress indicators */}
<div className="pt-4 border-t">
<h3 className="text-sm font-semibold mb-3 text-muted-foreground">
Download Progress
</h3>
<div className="space-y-2">
{modelStatus.models.map((model) => (
<ModelProgress
key={model.model_name}
modelName={model.model_name}
displayName={model.display_name}
/>
))}
</div>
</div>
</div>
) : null}
</CardContent>
@@ -235,16 +265,20 @@ interface ModelItemProps {
model_name: string;
display_name: string;
downloaded: boolean;
downloading?: boolean; // From server - true if download in progress
size_mb?: number;
loaded: boolean;
};
onDownload: () => void;
onDelete: () => void;
isDownloading: boolean;
isDownloading: boolean; // Local state - true if user just clicked download
formatSize: (sizeMb?: number) => string;
}
function ModelItem({ model, onDownload, onDelete, isDownloading, formatSize }: ModelItemProps) {
// Use server's downloading state OR local state (for immediate feedback before server updates)
const showDownloading = model.downloading || isDownloading;
return (
<div className="flex items-center justify-between p-3 border rounded-lg">
<div className="flex-1">
@@ -255,20 +289,21 @@ function ModelItem({ model, onDownload, onDelete, isDownloading, formatSize }: M
Loaded
</Badge>
)}
{model.downloaded && !model.loaded && (
{/* Only show Downloaded if actually downloaded AND not downloading */}
{model.downloaded && !model.loaded && !showDownloading && (
<Badge variant="secondary" className="text-xs">
Downloaded
</Badge>
)}
</div>
{model.downloaded && model.size_mb && (
{model.downloaded && model.size_mb && !showDownloading && (
<div className="text-xs text-muted-foreground mt-1">
Size: {formatSize(model.size_mb)}
</div>
)}
</div>
<div className="flex items-center gap-2">
{model.downloaded ? (
{model.downloaded && !showDownloading ? (
<div className="flex items-center gap-2">
<div className="flex items-center gap-1 text-sm text-muted-foreground">
<span>Ready</span>
@@ -283,19 +318,15 @@ function ModelItem({ model, onDownload, onDelete, isDownloading, formatSize }: M
<Trash2 className="h-4 w-4" />
</Button>
</div>
) : showDownloading ? (
<Button size="sm" variant="outline" disabled>
<Loader2 className="h-4 w-4 mr-2 animate-spin" />
Downloading...
</Button>
) : (
<Button size="sm" onClick={onDownload} disabled={isDownloading} variant="outline">
{isDownloading ? (
<>
<Loader2 className="h-4 w-4 mr-2 animate-spin" />
Downloading...
</>
) : (
<>
<Download className="h-4 w-4 mr-2" />
Download
</>
)}
<Button size="sm" onClick={onDownload} variant="outline">
<Download className="h-4 w-4 mr-2" />
Download
</Button>
)}
</div>
@@ -8,14 +8,23 @@ import { useServerStore } from '@/stores/serverStore';
interface ModelProgressProps {
modelName: string;
displayName: string;
/** Only connect to SSE when actively downloading - prevents connection exhaustion */
isDownloading?: boolean;
}
export function ModelProgress({ modelName, displayName }: ModelProgressProps) {
export function ModelProgress({ modelName, displayName, isDownloading = false }: ModelProgressProps) {
const [progress, setProgress] = useState<ModelProgressType | null>(null);
const serverUrl = useServerStore((state) => state.serverUrl);
useEffect(() => {
if (!serverUrl) return;
// IMPORTANT: Only connect to SSE when this specific model is downloading
// Opening SSE connections for all models exhausts HTTP/1.1 connection limits (6 per origin)
// which causes other fetches (like the download trigger) to be queued/blocked
if (!serverUrl || !isDownloading) {
return;
}
console.log(`[ModelProgress] Connecting SSE for ${modelName}`);
// Subscribe to progress updates via Server-Sent Events
const eventSource = new EventSource(`${serverUrl}/models/progress/${modelName}`);
@@ -27,6 +36,7 @@ export function ModelProgress({ modelName, displayName }: ModelProgressProps) {
// Close connection if complete or error
if (data.status === 'complete' || data.status === 'error') {
console.log(`[ModelProgress] Download ${data.status} for ${modelName}, closing SSE`);
eventSource.close();
}
} catch (error) {
@@ -35,14 +45,15 @@ export function ModelProgress({ modelName, displayName }: ModelProgressProps) {
};
eventSource.onerror = (error) => {
console.error('SSE error:', error);
console.error(`[ModelProgress] SSE error for ${modelName}:`, error);
eventSource.close();
};
return () => {
console.log(`[ModelProgress] Cleanup - closing SSE for ${modelName}`);
eventSource.close();
};
}, [serverUrl, modelName]);
}, [serverUrl, modelName, isDownloading]);
// Don't render if no progress or if complete/error and some time has passed
if (
@@ -13,9 +13,10 @@ export function UpdateStatus() {
const [currentVersion, setCurrentVersion] = useState<string>('');
useEffect(() => {
platform.metadata.getVersion()
platform.metadata
.getVersion()
.then(setCurrentVersion)
.catch(() => setCurrentVersion('0.1.0'));
.catch(() => setCurrentVersion('Unknown'));
}, [platform]);
return (
@@ -58,6 +58,7 @@ export function AudioSampleRecording({
// Request microphone access when component mounts
useEffect(() => {
if (!showWaveform) return;
if (!navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) return;
let stream: MediaStream | null = null;
@@ -43,7 +43,7 @@ import {
} from '@/lib/hooks/useProfiles';
import { useSystemAudioCapture } from '@/lib/hooks/useSystemAudioCapture';
import { useTranscription } from '@/lib/hooks/useTranscription';
import { formatAudioDuration, getAudioDuration } from '@/lib/utils/audio';
import { convertToWav, formatAudioDuration, getAudioDuration } from '@/lib/utils/audio';
import { usePlatform } from '@/platform/PlatformContext';
import { useServerStore } from '@/stores/serverStore';
import { type ProfileFormDraft, useUIStore } from '@/stores/uiStore';
@@ -505,10 +505,23 @@ export function ProfileForm() {
language: data.language,
});
// Convert non-WAV uploads to WAV so the backend can always use soundfile.
// Recorded audio is already WAV (from useAudioRecording's convertToWav call).
let fileToUpload: File = sampleFile;
if (!sampleFile.type.includes('wav') && !sampleFile.name.toLowerCase().endsWith('.wav')) {
try {
const wavBlob = await convertToWav(sampleFile);
const wavName = sampleFile.name.replace(/\.[^.]+$/, '.wav');
fileToUpload = new File([wavBlob], wavName, { type: 'audio/wav' });
} catch {
// If browser can't decode the format, send the original and let the backend try.
}
}
try {
await addSample.mutateAsync({
profileId: profile.id,
file: sampleFile,
file: fileToUpload,
referenceText: referenceText,
});
+3 -3
View File
@@ -79,8 +79,8 @@ export function VoicesTab() {
setDialogOpen(true);
};
const handleDelete = (profileId: string) => {
if (confirm('Are you sure you want to delete this profile?')) {
const handleProfileDelete = async (profileId: string) => {
if (await confirm('Are you sure you want to delete this profile?')) {
deleteProfile.mutate(profileId);
}
};
@@ -147,7 +147,7 @@ export function VoicesTab() {
channels={channels || []}
onChannelChange={(channelIds) => handleChannelChange(profile.id, channelIds)}
onEdit={() => handleEdit(profile.id)}
onDelete={() => handleDelete(profile.id)}
onDelete={() => handleProfileDelete(profile.id)}
/>
))}
</TableBody>
+16 -10
View File
@@ -1,4 +1,4 @@
import { useCallback, useEffect, useState } from 'react';
import { useCallback, useEffect, useRef, useState } from 'react';
import { usePlatform } from '@/platform/PlatformContext';
import type { UpdateStatus } from '@/platform/types';
@@ -7,9 +7,8 @@ export type { UpdateStatus };
export function useAutoUpdater(checkOnMount = false) {
const platform = usePlatform();
const [status, setStatus] = useState<UpdateStatus>(
platform.updater.getStatus(),
);
const [status, setStatus] = useState<UpdateStatus>(platform.updater.getStatus());
const hasCheckedRef = useRef(false);
// Subscribe to updater status changes
useEffect(() => {
@@ -17,25 +16,32 @@ export function useAutoUpdater(checkOnMount = false) {
setStatus(newStatus);
});
return unsubscribe;
}, [platform]);
// Empty dependency array - platform is stable from context
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.updater.subscribe]);
const checkForUpdates = useCallback(async () => {
await platform.updater.checkForUpdates();
}, [platform]);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.updater.checkForUpdates]);
const downloadAndInstall = useCallback(async () => {
await platform.updater.downloadAndInstall();
}, [platform]);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.updater.downloadAndInstall]);
const restartAndInstall = useCallback(async () => {
await platform.updater.restartAndInstall();
}, [platform]);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.updater.restartAndInstall]);
useEffect(() => {
if (checkOnMount && platform.metadata.isTauri) {
if (checkOnMount && platform.metadata.isTauri && !hasCheckedRef.current) {
hasCheckedRef.current = true;
checkForUpdates();
}
}, [checkOnMount, checkForUpdates, platform.metadata.isTauri]);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.metadata.isTauricheckOnMountcheckForUpdates]);
return {
status,
+209
View File
@@ -0,0 +1,209 @@
import { Download, RefreshCw } from 'lucide-react';
import { useCallback, useEffect, useRef, useState } from 'react';
import { Progress } from '@/components/ui/progress';
import { ToastAction } from '@/components/ui/toast';
import { useToast } from '@/components/ui/use-toast';
import { usePlatform } from '@/platform/PlatformContext';
import type { UpdateStatus } from '@/platform/types';
// Re-export UpdateStatus for backwards compatibility
export type { UpdateStatus };
interface UseAutoUpdaterOptions {
checkOnMount?: boolean;
showToast?: boolean;
}
export function useAutoUpdater(options: boolean | UseAutoUpdaterOptions = false) {
// Support both old boolean API and new options object
const { checkOnMount, showToast } =
typeof options === 'boolean'
? { checkOnMount: options, showToast: false }
: { checkOnMount: options.checkOnMount ?? false, showToast: options.showToast ?? false };
const platform = usePlatform();
const { toast } = useToast();
const [status, setStatus] = useState<UpdateStatus>(platform.updater.getStatus());
const hasCheckedRef = useRef(false);
const toastIdRef = useRef<string | null>(null);
const toastUpdateRef = useRef<
| ((props: {
title?: React.ReactNode;
description?: React.ReactNode;
duration?: number;
variant?: 'default' | 'destructive';
open?: boolean;
action?: React.ReactElement<typeof ToastAction>;
}) => void)
| null
>(null);
// Subscribe to updater status changes
useEffect(() => {
const unsubscribe = platform.updater.subscribe((newStatus) => {
setStatus(newStatus);
});
return unsubscribe;
// Empty dependency array - platform is stable from context
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.updater.subscribe]);
const checkForUpdates = useCallback(async () => {
await platform.updater.checkForUpdates();
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.updater.checkForUpdates]);
const downloadAndInstall = useCallback(async () => {
await platform.updater.downloadAndInstall();
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.updater.downloadAndInstall]);
const restartAndInstall = useCallback(async () => {
await platform.updater.restartAndInstall();
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.updater.restartAndInstall]);
// Check for updates on mount
useEffect(() => {
if (checkOnMount && platform.metadata.isTauri && !hasCheckedRef.current) {
hasCheckedRef.current = true;
checkForUpdates().catch((error) => {
console.error('Auto update check failed:', error);
});
}
// Empty dependency array - only run once on mount
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.metadata.isTauricheckOnMountcheckForUpdates]);
// Show toast when update is available
useEffect(() => {
if (
!showToast ||
!status.available ||
status.downloading ||
status.readyToInstall ||
toastIdRef.current
) {
return;
}
const handleUpdateNow = async () => {
await downloadAndInstall();
};
const toastResult = toast({
title: 'Update Available',
description: `Version ${status.version} is ready to download.`,
duration: Infinity,
action: (
<ToastAction altText="Update now" onClick={handleUpdateNow}>
Update Now
</ToastAction>
),
});
toastIdRef.current = toastResult.id;
// Type assertion needed because update function has broader type than our ref
toastUpdateRef.current = toastResult.update as typeof toastUpdateRef.current;
}, [
showToast,
status.available,
status.downloading,
status.readyToInstall,
status.version,
downloadAndInstall,
toast,
]);
// Update toast when downloading
useEffect(() => {
if (!showToast || !status.downloading || !toastIdRef.current || !toastUpdateRef.current) {
return;
}
const progressPercent = status.downloadProgress || 0;
const progressText =
status.downloadedBytes !== undefined &&
status.totalBytes !== undefined &&
status.totalBytes > 0
? `${(status.downloadedBytes / 1024 / 1024).toFixed(1)} MB / ${(status.totalBytes / 1024 / 1024).toFixed(1)} MB`
: '';
toastUpdateRef.current({
title: (
<div className="flex items-center gap-2">
<Download className="h-4 w-4 animate-pulse" />
<span>Downloading Update</span>
</div>
),
description: (
<div className="space-y-2">
<div className="text-sm">Version {status.version}</div>
{progressPercent > 0 && (
<>
<Progress value={progressPercent} className="h-2" />
{progressText && <div className="text-xs text-muted-foreground">{progressText}</div>}
</>
)}
</div>
),
duration: Infinity,
});
}, [
showToast,
status.downloading,
status.downloadProgress,
status.downloadedBytes,
status.totalBytes,
status.version,
]);
// Update toast when ready to install
useEffect(() => {
if (!showToast || !status.readyToInstall || !toastIdRef.current || !toastUpdateRef.current) {
return;
}
const handleRestartNow = async () => {
await restartAndInstall();
};
toastUpdateRef.current({
title: 'Update Ready',
description: `Version ${status.version} has been downloaded and is ready to install.`,
duration: Infinity,
action: (
<ToastAction altText="Restart now" onClick={handleRestartNow}>
<RefreshCw className="h-3 w-3 mr-1" />
Restart Now
</ToastAction>
),
});
}, [showToast, status.readyToInstall, status.version, restartAndInstall]);
// Handle errors in toast
useEffect(() => {
if (!showToast || !status.error || !toastIdRef.current || !toastUpdateRef.current) {
return;
}
toastUpdateRef.current({
title: 'Update Failed',
description: status.error,
variant: 'destructive',
duration: 5000,
});
setTimeout(() => {
toastIdRef.current = null;
toastUpdateRef.current = null;
}, 5000);
}, [showToast, status.error]);
return {
status,
checkForUpdates,
downloadAndInstall,
restartAndInstall,
};
}
+4 -1
View File
@@ -310,10 +310,13 @@ class ApiClient {
}
async triggerModelDownload(modelName: string): Promise<{ message: string }> {
return this.request<{ message: string }>('/models/download', {
console.log('[API] triggerModelDownload called for:', modelName, 'at', new Date().toISOString());
const result = await this.request<{ message: string }>('/models/download', {
method: 'POST',
body: JSON.stringify({ model_name: modelName } as ModelDownloadRequest),
});
console.log('[API] triggerModelDownload response:', result);
return result;
}
async deleteModel(modelName: string): Promise<{ message: string }> {
+1
View File
@@ -9,6 +9,7 @@ export type ModelStatus = {
model_name: string;
display_name: string;
downloaded: boolean;
downloading?: boolean; // True if download is in progress
size_mb?: number | null;
loaded?: boolean;
};
+1
View File
@@ -96,6 +96,7 @@ export interface ModelStatus {
model_name: string;
display_name: string;
downloaded: boolean;
downloading: boolean; // True if download is in progress
size_mb?: number;
loaded: boolean;
}
+26 -20
View File
@@ -20,11 +20,13 @@ export function useAudioRecording({
const streamRef = useRef<MediaStream | null>(null);
const timerRef = useRef<number | null>(null);
const startTimeRef = useRef<number | null>(null);
const cancelledRef = useRef<boolean>(false);
const startRecording = useCallback(async () => {
try {
setError(null);
chunksRef.current = [];
cancelledRef.current = false;
setDuration(0);
// Check if getUserMedia is available
@@ -87,31 +89,34 @@ export function useAudioRecording({
};
mediaRecorder.onstop = async () => {
// Snapshot the cancellation flag and recorded duration immediately —
// cancelRecording() clears chunks and sets cancelledRef synchronously
// before this async handler runs, so we must check it first.
const wasCancelled = cancelledRef.current;
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
const webmBlob = new Blob(chunksRef.current, { type: 'audio/webm' });
// Convert to WAV format to avoid needing ffmpeg on backend
try {
const wavBlob = await convertToWav(webmBlob);
// Pass the actual recorded duration
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
onRecordingComplete?.(wavBlob, recordedDuration);
} catch (err) {
console.error('Error converting audio to WAV:', err);
// Fallback to original blob if conversion fails
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
onRecordingComplete?.(webmBlob, recordedDuration);
}
// Stop all tracks
// Stop all tracks now that we have the data
streamRef.current?.getTracks().forEach((track) => {
track.stop();
});
streamRef.current = null;
// Don't fire completion callback if the recording was cancelled
if (wasCancelled) return;
// Convert to WAV format to avoid needing ffmpeg on backend
try {
const wavBlob = await convertToWav(webmBlob);
onRecordingComplete?.(wavBlob, recordedDuration);
} catch (err) {
console.error('Error converting audio to WAV:', err);
// Fallback to original blob if conversion fails
onRecordingComplete?.(webmBlob, recordedDuration);
}
};
mediaRecorder.onerror = (event) => {
@@ -167,9 +172,10 @@ export function useAudioRecording({
const cancelRecording = useCallback(() => {
if (mediaRecorderRef.current) {
cancelledRef.current = true; // Must be set before stop() triggers onstop
chunksRef.current = [];
mediaRecorderRef.current.stop();
setIsRecording(false);
chunksRef.current = [];
setDuration(0);
}
+74 -34
View File
@@ -1,14 +1,16 @@
import { useEffect, useRef } from 'react';
import { useToast } from '@/components/ui/use-toast';
import { useServerStore } from '@/stores/serverStore';
import { CheckCircle2, Loader2, XCircle } from 'lucide-react';
import { useCallback, useEffect, useRef } from 'react';
import { Progress } from '@/components/ui/progress';
import { Loader2, CheckCircle2, XCircle } from 'lucide-react';
import { useToast } from '@/components/ui/use-toast';
import type { ModelProgress } from '@/lib/api/types';
import { useServerStore } from '@/stores/serverStore';
interface UseModelDownloadToastOptions {
modelName: string;
displayName: string;
enabled?: boolean;
onComplete?: () => void;
onError?: () => void;
}
/**
@@ -19,47 +21,64 @@ export function useModelDownloadToast({
modelName,
displayName,
enabled = false,
onComplete,
onError,
}: UseModelDownloadToastOptions) {
const { toast } = useToast();
const serverUrl = useServerStore((state) => state.serverUrl);
const toastIdRef = useRef<string | null>(null);
const toastUpdateRef = useRef<
((props: {
title?: React.ReactNode;
description?: React.ReactNode;
duration?: number;
variant?: 'default' | 'destructive';
open?: boolean;
}) => void) | null
>(null);
// biome-ignore lint: Using any for toast update ref to handle complex toast types
const toastUpdateRef = useRef<any>(null);
const eventSourceRef = useRef<EventSource | null>(null);
const formatBytes = (bytes: number): string => {
const formatBytes = useCallback((bytes: number): string => {
if (bytes === 0) return '0 B';
const k = 1024;
const sizes = ['B', 'KB', 'MB', 'GB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return `${(bytes / Math.pow(k, i)).toFixed(1)} ${sizes[i]}`;
};
return `${(bytes / k ** i).toFixed(1)} ${sizes[i]}`;
}, []);
useEffect(() => {
console.log('[useModelDownloadToast] useEffect triggered', {
enabled,
serverUrl,
modelName,
displayName,
});
if (!enabled || !serverUrl || !modelName) {
console.log('[useModelDownloadToast] Not enabled, skipping');
return;
}
console.log('[useModelDownloadToast] Creating toast and EventSource for:', modelName);
// Create initial toast
const toastResult = toast({
title: displayName,
description: 'Starting download...',
description: (
<div className="flex items-center gap-2">
<Loader2 className="h-4 w-4 animate-spin" />
<span>Connecting to download...</span>
</div>
),
duration: Infinity, // Don't auto-dismiss, we'll handle it manually
});
toastIdRef.current = toastResult.id;
toastUpdateRef.current = toastResult.update;
// Subscribe to progress updates via Server-Sent Events
const eventSource = new EventSource(`${serverUrl}/models/progress/${modelName}`);
const eventSourceUrl = `${serverUrl}/models/progress/${modelName}`;
console.log('[useModelDownloadToast] Creating EventSource to:', eventSourceUrl);
const eventSource = new EventSource(eventSourceUrl);
eventSource.onopen = () => {
console.log('[useModelDownloadToast] EventSource connection opened for:', modelName);
};
eventSource.onmessage = (event) => {
console.log('[useModelDownloadToast] Received SSE message:', event.data);
try {
const progress = JSON.parse(event.data) as ModelProgress;
@@ -86,7 +105,7 @@ export function useModelDownloadToast({
break;
case 'downloading':
statusIcon = <Loader2 className="h-4 w-4 animate-spin" />;
statusText = progress.filename ? `Downloading ${progress.filename}...` : 'Downloading...';
statusText = progress.filename || 'Downloading...';
break;
case 'extracting':
statusIcon = <Loader2 className="h-4 w-4 animate-spin" />;
@@ -117,21 +136,40 @@ export function useModelDownloadToast({
});
// Close connection and dismiss toast on completion or error
if (progress.status === 'complete' || progress.status === 'error') {
// Also treat progress >= 100% as complete
const isComplete = progress.status === 'complete' || progress.progress >= 100;
const isError = progress.status === 'error';
if (isComplete || isError) {
console.log('[useModelDownloadToast] Download finished:', {
isComplete,
isError,
progress: progress.progress,
});
eventSource.close();
eventSourceRef.current = null;
// Auto-dismiss on completion after delay
if (progress.status === 'complete') {
setTimeout(() => {
if (toastIdRef.current && toastUpdateRef.current) {
toastUpdateRef.current({
open: false,
});
toastIdRef.current = null;
toastUpdateRef.current = null;
}
}, 5000);
// Update toast to show completion state before callbacks
if (isComplete && toastUpdateRef.current) {
toastUpdateRef.current({
title: (
<div className="flex items-center gap-2">
<CheckCircle2 className="h-4 w-4 text-green-500" />
<span>{displayName}</span>
</div>
),
description: 'Download complete',
duration: 3000,
});
}
// Call callbacks
if (isComplete && onComplete) {
console.log('[useModelDownloadToast] Download complete, calling onComplete callback');
onComplete();
} else if (isError && onError) {
console.log('[useModelDownloadToast] Download error, calling onError callback');
onError();
}
}
}
@@ -141,7 +179,8 @@ export function useModelDownloadToast({
};
eventSource.onerror = (error) => {
console.error('SSE error:', error);
console.error('[useModelDownloadToast] SSE error for:', modelName, error);
console.log('[useModelDownloadToast] EventSource readyState:', eventSource.readyState);
eventSource.close();
eventSourceRef.current = null;
@@ -162,15 +201,16 @@ export function useModelDownloadToast({
// Cleanup on unmount or when disabled
return () => {
console.log('[useModelDownloadToast] Cleanup - closing EventSource for:', modelName);
if (eventSourceRef.current) {
eventSourceRef.current.close();
eventSourceRef.current = null;
}
// Note: We don't dismiss the toast here as it might still be showing completion state
};
}, [enabled, serverUrl, modelName, displayName, toast]);
}, [enabled, serverUrl, modelName, displayName, toast, formatBytes, onComplete, onError]);
return {
isTracking: enabled && eventSourceRef.current !== null,
};
}
}
+36 -18
View File
@@ -22,6 +22,11 @@ export function formatAudioDuration(seconds: number): string {
* If the file has a recordedDuration property (from recording hooks),
* use that instead of trying to read metadata. This fixes issues on Windows
* where WebM files from MediaRecorder don't have proper duration metadata.
*
* For uploaded files we use AudioContext.decodeAudioData which fully decodes
* the audio and returns the exact duration. This is more reliable than
* HTMLMediaElement.duration which can return incorrect large values for VBR
* MP3 files that lack a proper XING/VBRI header.
*/
export async function getAudioDuration(
file: File & { recordedDuration?: number },
@@ -30,26 +35,39 @@ export async function getAudioDuration(
return file.recordedDuration;
}
return new Promise((resolve, reject) => {
const audio = new Audio();
const url = URL.createObjectURL(file);
// Use Web Audio API for accurate duration — avoids VBR MP3 metadata issues.
try {
const audioContext = new AudioContext();
try {
const arrayBuffer = await file.arrayBuffer();
const audioBuffer = await audioContext.decodeAudioData(arrayBuffer);
return audioBuffer.duration;
} finally {
await audioContext.close();
}
} catch {
// Fallback: read duration from the media element (less accurate but works for WAV).
return new Promise((resolve, reject) => {
const audio = new Audio();
const url = URL.createObjectURL(file);
audio.addEventListener('loadedmetadata', () => {
URL.revokeObjectURL(url);
if (Number.isFinite(audio.duration) && audio.duration > 0) {
resolve(audio.duration);
} else {
reject(new Error('Audio file has invalid duration metadata'));
}
audio.addEventListener('loadedmetadata', () => {
URL.revokeObjectURL(url);
if (Number.isFinite(audio.duration) && audio.duration > 0) {
resolve(audio.duration);
} else {
reject(new Error('Audio file has invalid duration metadata'));
}
});
audio.addEventListener('error', () => {
URL.revokeObjectURL(url);
reject(new Error('Failed to load audio file'));
});
audio.src = url;
});
audio.addEventListener('error', () => {
URL.revokeObjectURL(url);
reject(new Error('Failed to load audio file'));
});
audio.src = url;
});
}
}
/**
+1 -1
View File
@@ -1,3 +1,3 @@
# Backend package
__version__ = "0.1.11"
__version__ = "0.1.13"
+150 -48
View File
@@ -52,6 +52,47 @@ class MLXTTSBackend:
return hf_model_id
def _is_model_cached(self, model_size: str) -> bool:
"""
Check if the model is already cached locally AND fully downloaded.
Args:
model_size: Model size to check
Returns:
True if model is fully cached, False if missing or incomplete
"""
try:
from huggingface_hub import constants as hf_constants
model_path = self._get_model_path(model_size)
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_path.replace("/", "--"))
if not repo_cache.exists():
return False
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
print(f"[_is_model_cached] Found .incomplete files for {model_size}, treating as not cached")
return False
# Check that actual model weight files exist in snapshots
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = (
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.bin")) or
any(snapshots_dir.rglob("*.npz"))
)
if not has_weights:
print(f"[_is_model_cached] No model weights found for {model_size}, treating as not cached")
return False
return True
except Exception as e:
print(f"[_is_model_cached] Error checking cache for {model_size}: {e}")
return False
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the MLX TTS model.
@@ -79,46 +120,63 @@ class MLXTTSBackend:
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
try:
from mlx_audio.tts import load
# Get model path
# Get model path BEFORE importing mlx_audio
model_path = self._get_model_path(model_size)
# Set up progress tracking
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
# Start tracking download task
task_manager = get_task_manager()
task_manager.start_download(model_name)
# Check if model is already cached
is_cached = self._is_model_cached(model_size)
# Set up progress callback
# If cached: filter out non-download progress
# If not cached: report all progress (we're actually downloading)
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
print(f"Loading MLX TTS model {model_size}...")
# Initialize progress state
progress_manager.update_progress(
model_name=model_name,
current=0,
total=1,
filename="",
status="downloading",
)
# Only track download progress if model is NOT cached
if not is_cached:
# Start tracking download task
task_manager.start_download(model_name)
# Initialize progress state so SSE endpoint has initial data to send
# This provides immediate feedback while HuggingFace fetches metadata
progress_manager.update_progress(
model_name=model_name,
current=0,
total=0, # Will be updated once actual total is known
filename="Connecting to HuggingFace...",
status="downloading",
)
# Set up progress callback
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback)
# IMPORTANT: Patch tqdm BEFORE importing mlx_audio
# Otherwise mlx_audio caches reference to original tqdm
tracker_context = tracker.patch_download()
tracker_context.__enter__()
# Use progress tracker during download
with tracker.patch_download():
# Load MLX model (downloads automatically)
# Import mlx_audio AFTER patching tqdm
from mlx_audio.tts import load
# Load MLX model (downloads automatically)
try:
self.model = load(model_path)
finally:
# Exit the patch context
tracker_context.__exit__(None, None, None)
# Only mark download as complete if we were tracking it
if not is_cached:
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
self._current_model_size = model_size
self.model_size = model_size
# Mark as complete
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
print(f"MLX TTS model {model_size} loaded successfully")
except ImportError as e:
@@ -332,6 +390,47 @@ class MLXSTTBackend:
"""Check if model is loaded."""
return self.model is not None
def _is_model_cached(self, model_size: str) -> bool:
"""
Check if the Whisper model is already cached locally AND fully downloaded.
Args:
model_size: Model size to check
Returns:
True if model is fully cached, False if missing or incomplete
"""
try:
from huggingface_hub import constants as hf_constants
model_name = f"openai/whisper-{model_size}"
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_name.replace("/", "--"))
if not repo_cache.exists():
return False
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
print(f"[_is_model_cached] Found .incomplete files for whisper-{model_size}, treating as not cached")
return False
# Check that actual model weight files exist in snapshots
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = (
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.bin")) or
any(snapshots_dir.rglob("*.npz"))
)
if not has_weights:
print(f"[_is_model_cached] No model weights found for whisper-{model_size}, treating as not cached")
return False
return True
except Exception as e:
print(f"[_is_model_cached] Error checking cache for whisper-{model_size}: {e}")
return False
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the MLX Whisper model.
@@ -354,55 +453,58 @@ class MLXSTTBackend:
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
try:
# IMPORTANT: Set up progress tracking BEFORE importing mlx_audio
# This ensures tqdm is patched before any HuggingFace Hub imports
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
# Check if model is already cached
is_cached = self._is_model_cached(model_size)
# Set up progress callback and tracker
# If cached: filter out non-download progress
# If not cached: report all progress (we're actually downloading)
progress_callback = create_hf_progress_callback(progress_model_name, progress_manager)
tracker = HFProgressTracker(progress_callback)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
# Patch tqdm BEFORE importing mlx_audio
# This is critical because mlx_audio imports huggingface_hub which imports tqdm
print("[DEBUG] Starting tqdm patch BEFORE mlx_audio import")
tracker_context = tracker.patch_download()
tracker_context.__enter__()
print("[DEBUG] tqdm patched, now importing mlx_audio")
# NOW import mlx_audio - it will use our patched tqdm
# Import mlx_audio
from mlx_audio.stt import load
# MLX Whisper uses the standard OpenAI models
model_name = f"openai/whisper-{model_size}"
# Start tracking download task
task_manager = get_task_manager()
task_manager.start_download(progress_model_name)
print(f"Loading MLX Whisper model {model_size}...")
# Initialize progress state
progress_manager.update_progress(
model_name=progress_model_name,
current=0,
total=1,
filename="",
status="downloading",
)
# Only track download progress if model is NOT cached
if not is_cached:
# Start tracking download task
task_manager.start_download(progress_model_name)
# Initialize progress state so SSE endpoint has initial data to send
progress_manager.update_progress(
model_name=progress_model_name,
current=0,
total=0,
filename="Connecting to HuggingFace...",
status="downloading",
)
# Load the model (tqdm is already patched from above)
# Load the model (tqdm is patched, but filters out non-download progress)
try:
self.model = load(model_name)
finally:
# Exit the patch context
tracker_context.__exit__(None, None, None)
self.model_size = model_size
# Only mark download as complete if we were tracking it
if not is_cached:
progress_manager.mark_complete(progress_model_name)
task_manager.complete_download(progress_model_name)
# Mark as complete
progress_manager.mark_complete(progress_model_name)
task_manager.complete_download(progress_model_name)
self.model_size = model_size
print(f"MLX Whisper model {model_size} loaded successfully")
+181 -55
View File
@@ -29,9 +29,23 @@ class PyTorchTTSBackend:
"""Get the best available device."""
if torch.cuda.is_available():
return "cuda"
elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
# MPS can have issues, use CPU for stability
return "cpu"
# Intel Arc / Intel Xe GPU via intel-extension-for-pytorch (IPEX)
try:
import intel_extension_for_pytorch # noqa: F401
if hasattr(torch, 'xpu') and torch.xpu.is_available():
return "xpu"
except ImportError:
pass
# Any GPU on Windows via DirectML (torch-directml)
try:
import torch_directml
if torch_directml.device_count() > 0:
return torch_directml.device(0)
except ImportError:
pass
# MPS (Apple Silicon) — kept for completeness but MLX backend is preferred
if hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
return "cpu" # MPS disabled for stability; MLX backend handles Apple Silicon
return "cpu"
def is_loaded(self) -> bool:
@@ -58,6 +72,46 @@ class PyTorchTTSBackend:
return hf_model_map[model_size]
def _is_model_cached(self, model_size: str) -> bool:
"""
Check if the model is already cached locally AND fully downloaded.
Args:
model_size: Model size to check
Returns:
True if model is fully cached, False if missing or incomplete
"""
try:
from huggingface_hub import constants as hf_constants
model_path = self._get_model_path(model_size)
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_path.replace("/", "--"))
if not repo_cache.exists():
return False
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
print(f"[_is_model_cached] Found .incomplete files for {model_size}, treating as not cached")
return False
# Check that actual model weight files exist in snapshots
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = (
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.bin"))
)
if not has_weights:
print(f"[_is_model_cached] No model weights found for {model_size}, treating as not cached")
return False
return True
except Exception as e:
print(f"[_is_model_cached] Error checking cache for {model_size}: {e}")
return False
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the TTS model with automatic downloading from HuggingFace Hub.
@@ -85,20 +139,24 @@ class PyTorchTTSBackend:
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
try:
# IMPORTANT: Set up progress tracking BEFORE importing qwen_tts
# This ensures tqdm is patched before any HuggingFace Hub imports
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
# Check if model is already cached
is_cached = self._is_model_cached(model_size)
# Set up progress callback and tracker
# If cached: filter out non-download progress (like "Segment 1/1" during generation)
# If not cached: report all progress (we're actually downloading)
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
# Patch tqdm BEFORE importing qwen_tts
tracker_context = tracker.patch_download()
tracker_context.__enter__()
# NOW import qwen_tts - it will use our patched tqdm
# Import qwen_tts
from qwen_tts import Qwen3TTSModel
# Get model path (local or HuggingFace Hub ID)
@@ -106,33 +164,45 @@ class PyTorchTTSBackend:
print(f"Loading TTS model {model_size} on {self.device}...")
# Start tracking download task
task_manager = get_task_manager()
task_manager.start_download(model_name)
# Only track download progress if model is NOT cached
if not is_cached:
# Start tracking download task
task_manager.start_download(model_name)
# Initialize progress state to show download has started
progress_manager.update_progress(
model_name=model_name,
current=0,
total=1, # Set to 1 initially, will be updated by callback
filename="",
status="downloading",
)
# Load the model (tqdm is already patched from above)
try:
self.model = Qwen3TTSModel.from_pretrained(
model_path,
device_map=self.device,
torch_dtype=torch.float32 if self.device == "cpu" else torch.bfloat16,
# Initialize progress state so SSE endpoint has initial data to send
progress_manager.update_progress(
model_name=model_name,
current=0,
total=0, # Will be updated once actual total is known
filename="Connecting to HuggingFace...",
status="downloading",
)
# Load the model (tqdm is patched, but filters out non-download progress)
try:
# Don't pass device_map on CPU: accelerate's meta-tensor mechanism
# causes "Cannot copy out of meta tensor" when moving to CPU.
# Instead load directly then call .to(device) if needed.
if self.device == "cpu":
self.model = Qwen3TTSModel.from_pretrained(
model_path,
torch_dtype=torch.float32,
low_cpu_mem_usage=False,
)
else:
self.model = Qwen3TTSModel.from_pretrained(
model_path,
device_map=self.device,
torch_dtype=torch.bfloat16,
)
finally:
# Exit the patch context
tracker_context.__exit__(None, None, None)
# Mark as complete
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
# Only mark download as complete if we were tracking it
if not is_cached:
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
self._current_model_size = model_size
self.model_size = model_size
@@ -312,15 +382,68 @@ class PyTorchSTTBackend:
"""Get the best available device."""
if torch.cuda.is_available():
return "cuda"
elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
# MPS support for Whisper
return "cpu" # Use CPU for stability
# Intel Arc / Intel Xe GPU via intel-extension-for-pytorch (IPEX)
try:
import intel_extension_for_pytorch # noqa: F401
if hasattr(torch, 'xpu') and torch.xpu.is_available():
return "xpu"
except ImportError:
pass
# Any GPU on Windows via DirectML (torch-directml)
try:
import torch_directml
if torch_directml.device_count() > 0:
return torch_directml.device(0)
except ImportError:
pass
if hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
return "cpu" # MPS disabled for stability
return "cpu"
def is_loaded(self) -> bool:
"""Check if model is loaded."""
return self.model is not None
def _is_model_cached(self, model_size: str) -> bool:
"""
Check if the Whisper model is already cached locally AND fully downloaded.
Args:
model_size: Model size to check
Returns:
True if model is fully cached, False if missing or incomplete
"""
try:
from huggingface_hub import constants as hf_constants
model_name = f"openai/whisper-{model_size}"
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_name.replace("/", "--"))
if not repo_cache.exists():
return False
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
print(f"[_is_model_cached] Found .incomplete files for whisper-{model_size}, treating as not cached")
return False
# Check that actual model weight files exist in snapshots
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = (
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.bin"))
)
if not has_weights:
print(f"[_is_model_cached] No model weights found for whisper-{model_size}, treating as not cached")
return False
return True
except Exception as e:
print(f"[_is_model_cached] Error checking cache for whisper-{model_size}: {e}")
return False
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the Whisper model.
@@ -349,14 +472,18 @@ class PyTorchSTTBackend:
"""Synchronous model loading."""
print(f"[DEBUG] _load_model_sync called for Whisper {model_size}")
try:
# IMPORTANT: Set up progress tracking BEFORE importing transformers
# This ensures tqdm is patched before any HuggingFace Hub imports
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
# Check if model is already cached
is_cached = self._is_model_cached(model_size)
# Set up progress callback and tracker
# If cached: filter out non-download progress
# If not cached: report all progress (we're actually downloading)
progress_callback = create_hf_progress_callback(progress_model_name, progress_manager)
tracker = HFProgressTracker(progress_callback)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
# Patch tqdm BEFORE importing transformers
print("[DEBUG] Starting tqdm patch BEFORE transformers import")
@@ -364,31 +491,29 @@ class PyTorchSTTBackend:
tracker_context.__enter__()
print("[DEBUG] tqdm patched, now importing transformers")
# NOW import transformers - it will use our patched tqdm
# Import transformers
from transformers import WhisperProcessor, WhisperForConditionalGeneration
model_name = f"openai/whisper-{model_size}"
print(f"[DEBUG] Model name: {model_name}")
# Start tracking download task
task_manager = get_task_manager()
task_manager.start_download(progress_model_name)
print(f"[DEBUG] Task manager started download")
print(f"Loading Whisper model {model_size} on {self.device}...")
# Initialize progress state to show download has started
print(f"[DEBUG] Calling update_progress...")
progress_manager.update_progress(
model_name=progress_model_name,
current=0,
total=1, # Set to 1 initially, will be updated by callback
filename="",
status="downloading",
)
print(f"[DEBUG] update_progress called, listeners: {len(progress_manager._listeners.get(progress_model_name, []))}")
# Only track download progress if model is NOT cached
if not is_cached:
# Start tracking download task
task_manager.start_download(progress_model_name)
# Load models (tqdm is already patched from above)
# Initialize progress state so SSE endpoint has initial data to send
progress_manager.update_progress(
model_name=progress_model_name,
current=0,
total=0, # Will be updated once actual total is known
filename="Connecting to HuggingFace...",
status="downloading",
)
# Load models (tqdm is patched, but filters out non-download progress)
try:
self.processor = WhisperProcessor.from_pretrained(model_name)
self.model = WhisperForConditionalGeneration.from_pretrained(model_name)
@@ -396,13 +521,14 @@ class PyTorchSTTBackend:
# Exit the patch context
tracker_context.__exit__(None, None, None)
# Only mark download as complete if we were tracking it
if not is_cached:
progress_manager.mark_complete(progress_model_name)
task_manager.complete_download(progress_model_name)
self.model.to(self.device)
self.model_size = model_size
# Mark as complete
progress_manager.mark_complete(progress_model_name)
task_manager.complete_download(progress_model_name)
print(f"Whisper model {model_size} loaded successfully")
except Exception as e:
+7 -3
View File
@@ -83,9 +83,13 @@ def build_server():
'--hidden-import', 'mlx_audio.stt',
'--collect-submodules', 'mlx',
'--collect-submodules', 'mlx_audio',
# Collect MLX data files including Metal shader libraries (.metallib)
'--collect-data', 'mlx',
'--collect-data', 'mlx_audio',
# Use --collect-all so PyInstaller bundles both data files AND
# native shared libraries (.dylib, .metallib) for MLX.
# Previously only --collect-data was used, which caused MLX to
# raise OSError at runtime inside the bundled binary because
# the Metal shader libraries were missing.
'--collect-all', 'mlx',
'--collect-all', 'mlx_audio',
])
else:
print("Building for non-Apple Silicon platform - PyTorch only")
+9
View File
@@ -4,8 +4,17 @@ Configuration module for voicebox backend.
Handles data directory configuration for production bundling.
"""
import os
from pathlib import Path
# Allow users to override the HuggingFace model download directory.
# Set VOICEBOX_MODELS_DIR to an absolute path before starting the server.
# This sets HF_HUB_CACHE so all huggingface_hub downloads go to that path.
_custom_models_dir = os.environ.get("VOICEBOX_MODELS_DIR")
if _custom_models_dir:
os.environ["HF_HUB_CACHE"] = _custom_models_dir
print(f"[config] Model download path set to: {_custom_models_dir}")
# Default data directory (used in development)
_data_dir = Path("data")
+254 -81
View File
@@ -22,6 +22,24 @@ import uuid
import asyncio
import signal
import os
from urllib.parse import quote
def _safe_content_disposition(disposition_type: str, filename: str) -> str:
"""Build a Content-Disposition header that is safe for non-ASCII filenames.
Uses RFC 5987 ``filename*`` parameter so that browsers can decode
UTF-8 filenames while the ``filename`` fallback stays ASCII-only.
"""
ascii_name = "".join(
c for c in filename if c.isascii() and (c.isalnum() or c in " -_.")
).strip() or "download"
utf8_name = quote(filename, safe="")
return (
f'{disposition_type}; filename="{ascii_name}"; '
f"filename*=UTF-8''{utf8_name}"
)
from . import database, models, profiles, history, tts, transcribe, config, export_import, channels, stories, __version__
from .database import get_db, Generation as DBGeneration, VoiceProfile as DBVoiceProfile
@@ -77,10 +95,39 @@ async def health():
tts_model = tts.get_tts_model()
backend_type = get_backend_type()
# Check for GPU availability (CUDA or MPS)
# Check for GPU availability (CUDA, MPS, Intel Arc XPU, or DirectML)
has_cuda = torch.cuda.is_available()
has_mps = hasattr(torch.backends, 'mps') and torch.backends.mps.is_available()
gpu_available = has_cuda or has_mps
# Intel Arc / Intel Xe via intel-extension-for-pytorch (IPEX)
has_xpu = False
xpu_name = None
try:
import intel_extension_for_pytorch as ipex # noqa: F401
if hasattr(torch, 'xpu') and torch.xpu.is_available():
has_xpu = True
try:
xpu_name = torch.xpu.get_device_name(0)
except Exception:
xpu_name = "Intel GPU"
except ImportError:
pass
# DirectML backend (torch-directml) for any Windows GPU
has_directml = False
directml_name = None
try:
import torch_directml
if torch_directml.device_count() > 0:
has_directml = True
try:
directml_name = torch_directml.device_name(0)
except Exception:
directml_name = "DirectML GPU"
except ImportError:
pass
gpu_available = has_cuda or has_mps or has_xpu or has_directml or backend_type == "mlx"
gpu_type = None
if has_cuda:
@@ -89,6 +136,10 @@ async def health():
gpu_type = "MPS (Apple Silicon)"
elif backend_type == "mlx":
gpu_type = "Metal (Apple Silicon via MLX)"
elif has_xpu:
gpu_type = f"XPU ({xpu_name})"
elif has_directml:
gpu_type = f"DirectML ({directml_name})"
vram_used = None
if has_cuda:
@@ -252,12 +303,17 @@ async def add_profile_sample(
db: Session = Depends(get_db),
):
"""Add a sample to a voice profile."""
# Save uploaded file to temporary location
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
# Preserve the uploaded file's extension so librosa can detect format correctly.
# Defaulting to .wav was causing soundfile to reject MP3/WebM content as invalid WAV.
_allowed_audio_exts = {'.wav', '.mp3', '.m4a', '.ogg', '.flac', '.aac', '.webm', '.opus'}
_uploaded_ext = Path(file.filename or '').suffix.lower()
file_suffix = _uploaded_ext if _uploaded_ext in _allowed_audio_exts else '.wav'
with tempfile.NamedTemporaryFile(suffix=file_suffix, delete=False) as tmp:
content = await file.read()
tmp.write(content)
tmp_path = tmp.name
try:
sample = await profiles.add_profile_sample(
profile_id,
@@ -268,6 +324,8 @@ async def add_profile_sample(
return sample
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
except Exception as e:
raise HTTPException(status_code=500, detail=f"Failed to process audio file: {str(e)}")
finally:
# Clean up temp file
Path(tmp_path).unlink(missing_ok=True)
@@ -388,7 +446,7 @@ async def export_profile(
io.BytesIO(zip_bytes),
media_type="application/zip",
headers={
"Content-Disposition": f'attachment; filename="{filename}"'
"Content-Disposition": _safe_content_disposition("attachment", filename)
}
)
except ValueError as e:
@@ -542,47 +600,50 @@ async def generate_speech(
if not profile:
raise HTTPException(status_code=404, detail="Profile not found")
# Create voice prompt from profile
voice_prompt = await profiles.create_voice_prompt_for_profile(
data.profile_id,
db,
)
# Generate audio
# Resolve model size and load the correct model FIRST.
# This must happen before create_voice_prompt_for_profile because that
# function calls load_model_async(None), which falls back to self.model_size.
# If the model is already loaded with the right size at that point, it
# returns immediately and the voice prompt is created by the correct model.
tts_model = tts.get_tts_model()
# Load the requested model size if different from current (async to not block)
model_size = data.model_size or "1.7B"
# Check if model needs to be downloaded first
model_path = tts_model._get_model_path(model_size)
if model_path.startswith("Qwen/"):
# Model not cached - check if it exists remotely or needs download
from huggingface_hub import constants as hf_constants
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_path.replace("/", "--"))
if not repo_cache.exists():
# Start download in background
model_name = f"qwen-tts-{model_size}"
if not tts_model._is_model_cached(model_size):
# Model is not fully cached — kick off a background download and tell
# the client to retry once it's ready.
model_name = f"qwen-tts-{model_size}"
async def download_model_background():
try:
await tts_model.load_model_async(model_size)
except Exception as e:
task_manager.error_download(model_name, str(e))
async def download_model_background():
try:
await tts_model.load_model_async(model_size)
except Exception as e:
task_manager.error_download(model_name, str(e))
task_manager.start_download(model_name)
asyncio.create_task(download_model_background())
task_manager.start_download(model_name)
asyncio.create_task(download_model_background())
# Return 202 Accepted with download info
raise HTTPException(
status_code=202,
detail={
"message": f"Model {model_size} is being downloaded. Please wait and try again.",
"model_name": model_name,
"downloading": True
}
)
raise HTTPException(
status_code=202,
detail={
"message": f"Model {model_size} is being downloaded. Please wait and try again.",
"model_name": model_name,
"downloading": True,
},
)
# Load (or switch to) the requested model before building the voice prompt
await tts_model.load_model_async(model_size)
# Create voice prompt from profile (model is already loaded with correct size)
voice_prompt = await profiles.create_voice_prompt_for_profile(
data.profile_id,
db,
)
audio, sample_rate = await tts_model.generate(
data.text,
voice_prompt,
@@ -625,6 +686,59 @@ async def generate_speech(
raise HTTPException(status_code=500, detail=str(e))
@app.post("/generate/stream")
async def stream_speech(
data: models.GenerationRequest,
db: Session = Depends(get_db),
):
"""
Generate speech and stream the WAV audio directly without saving to disk.
Returns raw WAV bytes via a StreamingResponse so the client can start
playing audio before the entire file has been received. This endpoint
does NOT create a history entry — use /generate for that.
"""
profile = await profiles.get_profile(data.profile_id, db)
if not profile:
raise HTTPException(status_code=404, detail="Profile not found")
tts_model = tts.get_tts_model()
model_size = data.model_size or "1.7B"
if not tts_model._is_model_cached(model_size):
raise HTTPException(
status_code=400,
detail=f"Model {model_size} is not downloaded yet. Use /generate to trigger a download.",
)
# Load the correct model before building the voice prompt (fixes issue #96)
await tts_model.load_model_async(model_size)
voice_prompt = await profiles.create_voice_prompt_for_profile(data.profile_id, db)
audio, sample_rate = await tts_model.generate(
data.text,
voice_prompt,
data.language,
data.seed,
data.instruct,
)
wav_bytes = tts.audio_to_wav_bytes(audio, sample_rate)
async def _wav_stream():
# Yield in chunks so large responses don't block the event loop
chunk_size = 64 * 1024 # 64 KB
for i in range(0, len(wav_bytes), chunk_size):
yield wav_bytes[i : i + chunk_size]
return StreamingResponse(
_wav_stream(),
media_type="audio/wav",
headers={"Content-Disposition": 'attachment; filename="speech.wav"'},
)
# ============================================
# HISTORY ENDPOINTS
# ============================================
@@ -753,7 +867,7 @@ async def export_generation(
io.BytesIO(zip_bytes),
media_type="application/zip",
headers={
"Content-Disposition": f'attachment; filename="{filename}"'
"Content-Disposition": _safe_content_disposition("attachment", filename)
}
)
except ValueError as e:
@@ -786,7 +900,7 @@ async def export_generation_audio(
audio_path,
media_type="audio/wav",
headers={
"Content-Disposition": f'attachment; filename="{filename}"'
"Content-Disposition": _safe_content_disposition("attachment", filename)
}
)
@@ -1054,7 +1168,7 @@ async def export_story_audio(
io.BytesIO(audio_bytes),
media_type="audio/wav",
headers={
"Content-Disposition": f'attachment; filename="{filename}"'
"Content-Disposition": _safe_content_disposition("attachment", filename)
}
)
except HTTPException:
@@ -1156,11 +1270,14 @@ async def get_model_progress(model_name: str):
@app.get("/models/status", response_model=models.ModelStatusListResponse)
async def get_model_status():
"""Get status of all available models."""
from huggingface_hub import hf_hub_download, constants as hf_constants
from huggingface_hub import constants as hf_constants
from pathlib import Path
import os
backend_type = get_backend_type()
task_manager = get_task_manager()
# Get set of currently downloading model names
active_download_names = {task.model_name for task in task_manager.get_active_downloads()}
# Try to import scan_cache_dir (might not be available in older versions)
try:
@@ -1189,10 +1306,11 @@ async def get_model_status():
if backend_type == "mlx":
tts_1_7b_id = "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16"
tts_0_6b_id = "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16" # Fallback to 1.7B
whisper_base_id = "mlx-community/whisper-base"
whisper_small_id = "mlx-community/whisper-small"
whisper_medium_id = "mlx-community/whisper-medium"
whisper_large_id = "mlx-community/whisper-large"
# MLX backend uses openai/whisper-* models, not mlx-community
whisper_base_id = "openai/whisper-base"
whisper_small_id = "openai/whisper-small"
whisper_medium_id = "openai/whisper-medium"
whisper_large_id = "openai/whisper-large"
else:
tts_1_7b_id = "Qwen/Qwen3-TTS-12Hz-1.7B-Base"
tts_0_6b_id = "Qwen/Qwen3-TTS-12Hz-0.6B-Base"
@@ -1246,6 +1364,13 @@ async def get_model_status():
},
]
# Build a mapping of model_name -> hf_repo_id so we can check if shared repos are downloading
model_to_repo = {cfg["model_name"]: cfg["hf_repo_id"] for cfg in model_configs}
# Get the set of hf_repo_ids that are currently being downloaded
# This handles the case where multiple models share the same repo (e.g., 0.6B and 1.7B on MLX)
active_download_repos = {model_to_repo.get(name) for name in active_download_names if name in model_to_repo}
# Get HuggingFace cache info (if available)
cache_info = None
if use_scan_cache:
@@ -1268,13 +1393,37 @@ async def get_model_status():
repo_id = config["hf_repo_id"]
for repo in cache_info.repos:
if repo.repo_id == repo_id:
downloaded = True
# Calculate size from cache info
# Check if actual model weight files exist (not just config files)
# scan_cache_dir only shows completed files, so check if any are model weights
has_model_weights = False
for rev in repo.revisions:
for f in rev.files:
fname = f.file_name.lower()
if fname.endswith(('.safetensors', '.bin', '.pt', '.pth', '.npz')):
has_model_weights = True
break
if has_model_weights:
break
# Also check for .incomplete files in blobs directory (downloads in progress)
has_incomplete = False
try:
total_size = sum(revision.size_on_disk for revision in repo.revisions)
size_mb = total_size / (1024 * 1024)
cache_dir = hf_constants.HF_HUB_CACHE
blobs_dir = Path(cache_dir) / ("models--" + repo_id.replace("/", "--")) / "blobs"
if blobs_dir.exists():
has_incomplete = any(blobs_dir.glob("*.incomplete"))
except Exception:
pass
# Only mark as downloaded if we have model weights AND no incomplete files
if has_model_weights and not has_incomplete:
downloaded = True
# Calculate size from cache info
try:
total_size = sum(revision.size_on_disk for revision in repo.revisions)
size_mb = total_size / (1024 * 1024)
except Exception:
pass
break
# Method 2: Fallback to checking cache directory directly (using HuggingFace's OS-specific cache location)
@@ -1284,42 +1433,40 @@ async def get_model_status():
repo_cache = Path(cache_dir) / ("models--" + config["hf_repo_id"].replace("/", "--"))
if repo_cache.exists():
# Check for model files (bin, safetensors, or other common model files)
# MLX models may use .npz or .safetensors
has_model_files = (
any(repo_cache.rglob("*.bin")) or
any(repo_cache.rglob("*.safetensors")) or
any(repo_cache.rglob("*.pt")) or
any(repo_cache.rglob("*.pth")) or
any(repo_cache.rglob("*.npz")) or
any(repo_cache.rglob("model.safetensors.index.json")) or
any(repo_cache.rglob("pytorch_model.bin.index.json"))
)
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
has_incomplete = blobs_dir.exists() and any(blobs_dir.glob("*.incomplete"))
if has_model_files:
downloaded = True
# Calculate size
try:
total_size = sum(f.stat().st_size for f in repo_cache.rglob("*") if f.is_file())
size_mb = total_size / (1024 * 1024)
except Exception:
pass
if not has_incomplete:
# Check for actual model weight files (not just index files)
# in the snapshots directory (symlinks to completed blobs)
snapshots_dir = repo_cache / "snapshots"
has_model_files = False
if snapshots_dir.exists():
has_model_files = (
any(snapshots_dir.rglob("*.bin")) or
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.pt")) or
any(snapshots_dir.rglob("*.pth")) or
any(snapshots_dir.rglob("*.npz"))
)
if has_model_files:
downloaded = True
# Calculate size (exclude .incomplete files)
try:
total_size = sum(
f.stat().st_size for f in repo_cache.rglob("*")
if f.is_file() and not f.name.endswith('.incomplete')
)
size_mb = total_size / (1024 * 1024)
except Exception:
pass
except Exception:
pass
# Method 3: Try to check if model can be loaded locally (last resort)
if not downloaded:
try:
# Try to download with local_files_only=True to check if cached
hf_hub_download(
repo_id=config["hf_repo_id"],
filename="config.json", # Try a common file
local_files_only=True,
)
downloaded = True
except Exception:
# File not found locally, model not downloaded
pass
# Method 3 removed - checking for config.json is too lenient
# Methods 1 and 2 properly verify that model weight files exist
# Check if loaded in memory
try:
@@ -1327,10 +1474,19 @@ async def get_model_status():
except Exception:
loaded = False
# Check if this model (or its shared repo) is currently being downloaded
is_downloading = config["hf_repo_id"] in active_download_repos
# If downloading, don't report as downloaded (partial files exist)
if is_downloading:
downloaded = False
size_mb = None # Don't show partial size during download
statuses.append(models.ModelStatus(
model_name=config["model_name"],
display_name=config["display_name"],
downloaded=downloaded,
downloading=is_downloading,
size_mb=size_mb,
loaded=loaded,
))
@@ -1341,10 +1497,14 @@ async def get_model_status():
except Exception:
loaded = False
# Check if this model (or its shared repo) is currently being downloaded
is_downloading = config["hf_repo_id"] in active_download_repos
statuses.append(models.ModelStatus(
model_name=config["model_name"],
display_name=config["display_name"],
downloaded=False, # Assume not downloaded if check failed
downloading=is_downloading,
size_mb=None,
loaded=loaded,
))
@@ -1358,6 +1518,7 @@ async def trigger_model_download(request: models.ModelDownloadRequest):
import asyncio
task_manager = get_task_manager()
progress_manager = get_progress_manager()
model_configs = {
"qwen-tts-1.7B": {
@@ -1405,6 +1566,18 @@ async def trigger_model_download(request: models.ModelDownloadRequest):
# Start tracking download
task_manager.start_download(request.model_name)
# Initialize progress state so SSE endpoint has initial data to send.
# This fixes a race condition where the frontend connects to SSE before
# any progress callbacks have fired (especially for large models like Qwen
# where huggingface_hub takes time to fetch metadata for all files).
progress_manager.update_progress(
model_name=request.model_name,
current=0,
total=0, # Will be updated once actual total is known
filename="Connecting to HuggingFace...",
status="downloading",
)
# Start download in background task (don't await)
asyncio.create_task(download_in_background())
+1
View File
@@ -134,6 +134,7 @@ class ModelStatus(BaseModel):
model_name: str
display_name: str
downloaded: bool
downloading: bool = False # True if download is in progress
size_mb: Optional[float] = None
loaded: bool = False
+7 -5
View File
@@ -19,15 +19,17 @@ def is_apple_silicon() -> bool:
def get_backend_type() -> Literal["mlx", "pytorch"]:
"""
Detect the best backend for the current platform.
Returns:
"mlx" on Apple Silicon (if MLX is available), "pytorch" otherwise
"mlx" on Apple Silicon (if MLX is available and functional), "pytorch" otherwise
"""
if is_apple_silicon():
try:
import mlx
import mlx.core # noqa: F401 — triggers native lib loading
return "mlx"
except ImportError:
# MLX not installed, fallback to PyTorch
except (ImportError, OSError, RuntimeError):
# MLX not installed, or native libraries failed to load inside a
# PyInstaller bundle (OSError on missing .dylib / .metallib).
# Fall through to PyTorch.
return "pytorch"
return "pytorch"
+1
View File
@@ -18,6 +18,7 @@ qwen-tts>=0.0.5
librosa>=0.10.0
soundfile>=0.12.0
numpy>=1.24.0
numba>=0.60.0,<0.61.0
# Utilities
python-multipart>=0.0.6
+58
View File
@@ -0,0 +1,58 @@
# Backend Tests
Manual test scripts for debugging and validating backend functionality.
## Test Files
### `test_generation_progress.py`
Tests TTS generation with SSE progress monitoring to identify UX issues where users see download progress even when the model is already cached.
**Usage:**
```bash
cd backend
python tests/test_generation_progress.py
```
**Prerequisites:**
- Server must be running (`python main.py`)
- At least one voice profile must exist
### `test_real_download.py`
Tests real model download with SSE progress monitoring.
**Usage:**
```bash
cd backend
# Delete cache first to force fresh download
rm -rf ~/.cache/huggingface/hub/models--openai--whisper-base
python tests/test_real_download.py
```
**Prerequisites:**
- Server must be running (`python main.py`)
### `test_progress.py`
Unit tests for ProgressManager and HFProgressTracker functionality.
**Usage:**
```bash
cd backend
python tests/test_progress.py
```
### `test_check_progress_state.py`
Debugging script to inspect the internal state of ProgressManager and TaskManager.
**Usage:**
```bash
cd backend
python tests/test_check_progress_state.py
```
## Notes
These are manual test scripts, not automated unit tests. They're designed for:
- Debugging progress tracking issues
- Validating SSE event streams
- Monitoring real-time download behavior
- Inspecting internal state during development
+6
View File
@@ -0,0 +1,6 @@
"""
Test suite for Voicebox backend.
This directory contains manual test scripts for debugging and validating
progress tracking, model downloads, and generation functionality.
"""
+321
View File
@@ -0,0 +1,321 @@
"""
Test TTS generation with SSE progress monitoring.
This test captures the exact SSE events triggered during generation
to identify UX issues where users see download progress even when
the model is already cached.
"""
import asyncio
import json
import httpx
from typing import List, Dict, Optional
from datetime import datetime
async def monitor_sse_stream(model_name: str, timeout: int = 120):
"""Monitor SSE stream for a model during generation."""
events: List[Dict] = []
url = f"http://localhost:8000/models/progress/{model_name}"
print(f"[{_timestamp()}] Connecting to SSE endpoint: {url}")
try:
async with httpx.AsyncClient(timeout=timeout) as client:
async with client.stream("GET", url) as response:
print(f"[{_timestamp()}] SSE connected, status: {response.status_code}")
if response.status_code != 200:
print(f"[{_timestamp()}] Error: SSE endpoint returned {response.status_code}")
return events
async for line in response.aiter_lines():
if not line:
continue
timestamp = _timestamp()
if line.startswith("data: "):
try:
data = json.loads(line[6:])
print(f"[{timestamp}] → SSE Event: {data['status']:12} {data.get('progress', 0):6.1f}% {data.get('filename', '')}")
events.append({
**data,
"_timestamp": timestamp
})
# Stop if complete or error
if data.get("status") in ("complete", "error"):
print(f"[{timestamp}] → Model {data['status']}!")
break
except json.JSONDecodeError as e:
print(f"[{timestamp}] Error parsing JSON: {e}")
print(f" Line was: {line}")
elif line.startswith(": heartbeat"):
print(f"[{timestamp}] ♥ heartbeat")
except asyncio.TimeoutError:
print(f"[{_timestamp()}] SSE monitoring timed out")
except Exception as e:
print(f"[{_timestamp()}] SSE error: {e}")
return events
async def trigger_generation(profile_id: str, text: str, model_size: str = "1.7B"):
"""Trigger TTS generation via the API."""
url = "http://localhost:8000/generate"
print(f"\n[{_timestamp()}] Triggering generation...")
print(f" Profile: {profile_id}")
print(f" Text: {text[:50]}...")
print(f" Model: {model_size}")
try:
async with httpx.AsyncClient(timeout=120) as client:
response = await client.post(url, json={
"profile_id": profile_id,
"text": text,
"language": "en",
"model_size": model_size,
})
print(f"[{_timestamp()}] Response: {response.status_code}")
if response.status_code == 200:
result = response.json()
print(f"[{_timestamp()}] ✓ Generation successful!")
print(f" Generation ID: {result.get('id')}")
print(f" Duration: {result.get('duration', 0):.2f}s")
return True, result
elif response.status_code == 202:
# Model is being downloaded
result = response.json()
print(f"[{_timestamp()}] → Model download in progress")
print(f" Detail: {result}")
return False, result
else:
print(f"[{_timestamp()}] ✗ Error: {response.text}")
return False, None
except Exception as e:
print(f"[{_timestamp()}] ✗ Exception: {e}")
return False, None
async def get_first_profile():
"""Get the first available voice profile."""
url = "http://localhost:8000/profiles"
try:
async with httpx.AsyncClient(timeout=10) as client:
response = await client.get(url)
if response.status_code == 200:
profiles = response.json()
if profiles:
return profiles[0]["id"]
except Exception as e:
print(f"Error getting profiles: {e}")
return None
async def check_server():
"""Check if the server is running."""
try:
async with httpx.AsyncClient(timeout=5) as client:
response = await client.get("http://localhost:8000/health")
return response.status_code == 200
except Exception as e:
print(f"Server not running: {e}")
return False
def _timestamp():
"""Get current timestamp for logging."""
return datetime.now().strftime("%H:%M:%S.%f")[:-3]
async def test_generation_with_cached_model():
"""
Test Case 1: Generation when model is already cached.
This should NOT show any download progress events.
If it does, that's the UX bug we're trying to fix.
"""
print("\n" + "=" * 80)
print("TEST CASE 1: Generation with Cached Model")
print("=" * 80)
print("Expected: No download progress events (or minimal/instant completion)")
print("Actual UX Issue: Users see 'started' and 'finished' events even for cached models")
print("=" * 80)
model_size = "1.7B"
model_name = f"qwen-tts-{model_size}"
# Get a profile
profile_id = await get_first_profile()
if not profile_id:
print("✗ No voice profiles found. Please create a profile first.")
return False
print(f"\nUsing profile: {profile_id}")
# Start SSE monitor BEFORE triggering generation
monitor_task = asyncio.create_task(monitor_sse_stream(model_name, timeout=30))
# Wait for SSE to connect
await asyncio.sleep(1)
# Trigger generation
test_text = "Hello, this is a test of the voice generation system."
success, result = await trigger_generation(profile_id, test_text, model_size)
if not success and result and result.get("downloading"):
print("\n⚠ Model is being downloaded. Waiting for download to complete...")
# Wait for SSE monitor to capture download events
events = await monitor_task
return events
# Wait a bit more to catch any progress events
await asyncio.sleep(3)
# Cancel SSE monitor
monitor_task.cancel()
try:
events = await monitor_task
except asyncio.CancelledError:
events = []
return events
async def test_generation_with_fresh_download():
"""
Test Case 2: Generation when model needs to be downloaded.
This SHOULD show download progress events.
"""
print("\n" + "=" * 80)
print("TEST CASE 2: Generation with Model Download")
print("=" * 80)
print("Expected: Download progress events from 0% to 100%")
print("=" * 80)
# Use a different model size to force download
model_size = "0.6B" # Smaller model for faster testing
model_name = f"qwen-tts-{model_size}"
# Get a profile
profile_id = await get_first_profile()
if not profile_id:
print("✗ No voice profiles found. Please create a profile first.")
return False
print(f"\nUsing profile: {profile_id}")
print("Note: This will download the model if not cached")
# Start SSE monitor BEFORE triggering generation
monitor_task = asyncio.create_task(monitor_sse_stream(model_name, timeout=300))
# Wait for SSE to connect
await asyncio.sleep(1)
# Trigger generation
test_text = "This should trigger a model download if the model is not cached."
success, result = await trigger_generation(profile_id, test_text, model_size)
if not success and result and result.get("downloading"):
print("\n→ Model download initiated. Monitoring progress...")
# Wait for download to complete
events = await monitor_task
# Try generation again
print(f"\n[{_timestamp()}] Retrying generation after download...")
await asyncio.sleep(2)
success, result = await trigger_generation(profile_id, test_text, model_size)
if success:
print("✓ Generation successful after download")
return events
# If model was already cached
await asyncio.sleep(3)
monitor_task.cancel()
try:
events = await monitor_task
except asyncio.CancelledError:
events = []
return events
async def main():
print("=" * 80)
print("TTS Generation Progress Test")
print("=" * 80)
print("Purpose: Capture exact SSE events during generation to identify UX issues")
print("=" * 80)
# Check if server is running
print(f"\n[{_timestamp()}] Checking if server is running...")
if not await check_server():
print("✗ Server is not running on http://localhost:8000")
print("\nPlease start the server first:")
print(" cd backend && python main.py")
return False
print("✓ Server is running")
# Test Case 1: Cached model
print("\n" + "🧪 " * 20)
events_cached = await test_generation_with_cached_model()
# Results for Test Case 1
print("\n" + "=" * 80)
print("TEST CASE 1 RESULTS: Generation with Cached Model")
print("=" * 80)
if not events_cached:
print("✓ GOOD: No SSE progress events received")
print(" This is the expected behavior for a cached model.")
else:
print(f"⚠ ISSUE FOUND: Received {len(events_cached)} SSE events:")
print("\nEvent Timeline:")
for i, event in enumerate(events_cached, 1):
timestamp = event.pop("_timestamp", "??:??:??.???")
print(f" {i}. [{timestamp}] {event}")
print("\n⚠ This explains the UX issue!")
print(" Users see progress events even when the model is already cached,")
print(" making them think the model is downloading again.")
# Test Case 2: Fresh download (optional, commented out by default)
# Uncomment if you want to test download progress
# print("\n" + "🧪 " * 20)
# events_download = await test_generation_with_fresh_download()
#
# print("\n" + "=" * 80)
# print("TEST CASE 2 RESULTS: Generation with Model Download")
# print("=" * 80)
#
# if not events_download:
# print("ℹ Model was already cached, no download occurred")
# else:
# print(f"✓ Received {len(events_download)} download progress events")
# print("\nDownload Timeline:")
# for i, event in enumerate(events_download, 1):
# timestamp = event.pop("_timestamp", "??:??:??.???")
# print(f" {i}. [{timestamp}] {event}")
print("\n" + "=" * 80)
print("Test Complete!")
print("=" * 80)
return True
if __name__ == "__main__":
asyncio.run(main())
+313
View File
@@ -0,0 +1,313 @@
"""
Test script to debug model download progress tracking.
"""
import asyncio
import json
import time
from typing import List, Dict
import logging
# Set up logging to see what's happening
logging.basicConfig(
level=logging.DEBUG,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
from utils.progress import ProgressManager, get_progress_manager
from utils.hf_progress import HFProgressTracker, create_hf_progress_callback
def test_progress_manager_basic():
"""Test 1: Basic ProgressManager functionality."""
print("\n" + "=" * 60)
print("Test 1: ProgressManager Basic Operations")
print("=" * 60)
pm = ProgressManager()
# Test update_progress
pm.update_progress(
model_name="test-model",
current=50,
total=100,
filename="test.bin",
status="downloading"
)
# Test get_progress
progress = pm.get_progress("test-model")
print(f"✓ Progress stored: {progress}")
assert progress is not None
assert progress["progress"] == 50.0
assert progress["filename"] == "test.bin"
assert progress["status"] == "downloading"
# Test mark_complete
pm.mark_complete("test-model")
progress = pm.get_progress("test-model")
print(f"✓ Marked complete: {progress}")
assert progress["status"] == "complete"
assert progress["progress"] == 100.0
print("✓ Test 1 PASSED\n")
return True
async def test_progress_manager_sse():
"""Test 2: ProgressManager SSE streaming."""
print("\n" + "=" * 60)
print("Test 2: ProgressManager SSE Streaming")
print("=" * 60)
pm = ProgressManager()
collected_events: List[Dict] = []
# Simulate SSE client
async def sse_client():
"""Simulates a frontend SSE connection."""
print(" SSE client: Subscribing to test-model-sse...")
async for event in pm.subscribe("test-model-sse"):
# Parse SSE event
if event.startswith("data: "):
data = json.loads(event[6:])
print(f" SSE client: Received event: {data['status']} - {data.get('progress', 0):.1f}%")
collected_events.append(data)
# Stop when complete
if data.get("status") in ("complete", "error"):
break
elif event.startswith(": heartbeat"):
print(" SSE client: Received heartbeat")
# Simulate download progress updates (from backend thread)
async def simulate_download():
"""Simulates backend sending progress updates."""
print(" Backend: Starting simulated download...")
await asyncio.sleep(0.2) # Let SSE client subscribe first
# Send progress updates
for i in range(0, 101, 20):
print(f" Backend: Updating progress to {i}%")
pm.update_progress(
model_name="test-model-sse",
current=i,
total=100,
filename=f"file_{i}.bin",
status="downloading" if i < 100 else "downloading"
)
await asyncio.sleep(0.1)
# Mark complete
print(" Backend: Marking download complete")
pm.mark_complete("test-model-sse")
# Run SSE client and download simulation concurrently
await asyncio.gather(
sse_client(),
simulate_download()
)
# Verify we got events
print(f"\n Collected {len(collected_events)} events")
assert len(collected_events) > 0, "Should have received at least one event"
assert collected_events[-1]["status"] == "complete", "Last event should be 'complete'"
print("✓ Test 2 PASSED\n")
return True
def test_hf_progress_tracker():
"""Test 3: HFProgressTracker tqdm patching."""
print("\n" + "=" * 60)
print("Test 3: HFProgressTracker tqdm Patching")
print("=" * 60)
captured_progress: List[tuple] = []
def progress_callback(downloaded: int, total: int, filename: str):
"""Capture progress updates."""
captured_progress.append((downloaded, total, filename))
print(f" Progress callback: {downloaded}/{total} bytes ({filename})")
tracker = HFProgressTracker(progress_callback)
# Simulate a download with tqdm
with tracker.patch_download():
try:
from tqdm import tqdm
# Simulate downloading a file
print(" Simulating download with tqdm...")
total_size = 1000
with tqdm(total=total_size, desc="model.bin", unit="B", unit_scale=True) as pbar:
for chunk in range(0, total_size, 100):
pbar.update(100)
time.sleep(0.01)
print(f" Captured {len(captured_progress)} progress updates")
assert len(captured_progress) > 0, "Should have captured progress updates"
# Verify progress increases
last_downloaded = 0
for downloaded, total, filename in captured_progress:
assert downloaded >= last_downloaded, "Downloaded bytes should increase"
assert total == total_size, "Total should be consistent"
last_downloaded = downloaded
print("✓ Test 3 PASSED\n")
return True
except ImportError:
print("✗ tqdm not available, skipping test\n")
return None
async def test_full_integration():
"""Test 4: Full integration test."""
print("\n" + "=" * 60)
print("Test 4: Full Integration (ProgressManager + HFProgressTracker)")
print("=" * 60)
pm = get_progress_manager()
collected_events: List[Dict] = []
# SSE client
async def sse_client():
print(" SSE client: Subscribing...")
async for event in pm.subscribe("integration-test"):
if event.startswith("data: "):
data = json.loads(event[6:])
print(f" SSE client: {data['status']} - {data.get('progress', 0):.1f}% - {data.get('filename', '')}")
collected_events.append(data)
if data.get("status") in ("complete", "error"):
break
# Simulate backend download with HFProgressTracker
async def simulate_real_download():
await asyncio.sleep(0.2) # Let SSE subscribe
print(" Backend: Starting download with HFProgressTracker...")
# Set up tracking (like the real backend does)
progress_callback = create_hf_progress_callback("integration-test", pm)
tracker = HFProgressTracker(progress_callback)
# Initialize progress
pm.update_progress(
model_name="integration-test",
current=0,
total=1,
filename="",
status="downloading"
)
# Simulate download with tqdm patching
with tracker.patch_download():
try:
from tqdm import tqdm
# Simulate multi-file download (like HuggingFace does)
files = [
("model.safetensors", 5000),
("config.json", 1000),
("tokenizer.json", 500),
]
for filename, size in files:
print(f" Backend: Downloading {filename}...")
with tqdm(total=size, desc=filename, unit="B") as pbar:
for chunk in range(0, size, 500):
chunk_size = min(500, size - chunk)
pbar.update(chunk_size)
await asyncio.sleep(0.05)
# Mark complete
print(" Backend: Download complete")
pm.mark_complete("integration-test")
except ImportError:
print(" ✗ tqdm not available")
pm.mark_error("integration-test", "tqdm not available")
# Run both
await asyncio.gather(
sse_client(),
simulate_real_download()
)
# Verify
print(f"\n Collected {len(collected_events)} events")
if len(collected_events) > 0:
print(f" First event: {collected_events[0]}")
print(f" Last event: {collected_events[-1]}")
assert collected_events[-1]["status"] == "complete", "Should end with 'complete'"
print("✓ Test 4 PASSED\n")
return True
else:
print("✗ Test 4 FAILED - No events received\n")
return False
async def main():
"""Run all tests."""
print("\n" + "=" * 60)
print("Voicebox Progress Tracking Test Suite")
print("=" * 60)
results = []
# Test 1: Basic operations
try:
results.append(("Basic Operations", test_progress_manager_basic()))
except Exception as e:
print(f"✗ Test 1 FAILED: {e}\n")
results.append(("Basic Operations", False))
# Test 2: SSE streaming
try:
results.append(("SSE Streaming", await test_progress_manager_sse()))
except Exception as e:
print(f"✗ Test 2 FAILED: {e}\n")
results.append(("SSE Streaming", False))
# Test 3: tqdm patching
try:
results.append(("tqdm Patching", test_hf_progress_tracker()))
except Exception as e:
print(f"✗ Test 3 FAILED: {e}\n")
results.append(("tqdm Patching", False))
# Test 4: Full integration
try:
results.append(("Full Integration", await test_full_integration()))
except Exception as e:
print(f"✗ Test 4 FAILED: {e}\n")
results.append(("Full Integration", False))
# Summary
print("\n" + "=" * 60)
print("Test Results Summary")
print("=" * 60)
for name, result in results:
status = "✓ PASS" if result else ("⊘ SKIP" if result is None else "✗ FAIL")
print(f" {status:8} {name}")
passed = sum(1 for _, r in results if r is True)
failed = sum(1 for _, r in results if r is False)
skipped = sum(1 for _, r in results if r is None)
print()
print(f" Total: {len(results)} tests")
print(f" Passed: {passed}")
print(f" Failed: {failed}")
print(f" Skipped: {skipped}")
print("=" * 60 + "\n")
return failed == 0
if __name__ == "__main__":
success = asyncio.run(main())
exit(0 if success else 1)
+317
View File
@@ -0,0 +1,317 @@
"""
Test Qwen TTS model download with SSE progress monitoring.
This specifically tests the MLX TTS backend download progress tracking,
which requires tqdm to be patched BEFORE mlx_audio is imported.
Usage:
cd backend && python -m tests.test_qwen_download
Prerequisites:
- Server must be running: cd backend && python main.py
- Delete model first for fresh download test:
curl -X DELETE http://localhost:8000/models/qwen-tts-0.6B
"""
import asyncio
import json
import httpx
import time
from typing import List, Dict, Optional
async def monitor_sse_stream(model_name: str, timeout: int = 600) -> List[Dict]:
"""
Monitor SSE stream for a model download.
Args:
model_name: Name of the model to monitor
timeout: Maximum time to wait for download (seconds)
Returns:
List of SSE events received
"""
events: List[Dict] = []
url = f"http://localhost:8000/models/progress/{model_name}"
last_progress = -1
print(f"\n📡 Connecting to SSE endpoint: {url}")
try:
async with httpx.AsyncClient(timeout=timeout) as client:
async with client.stream("GET", url) as response:
print(f" SSE connected, status: {response.status_code}")
if response.status_code != 200:
print(f" ❌ Error: SSE endpoint returned {response.status_code}")
return events
async for line in response.aiter_lines():
if not line:
continue
if line.startswith("data: "):
try:
data = json.loads(line[6:])
events.append(data)
# Print progress (only when it changes significantly)
progress = data.get('progress', 0)
status = data.get('status', 'unknown')
filename = data.get('filename', '')
current = data.get('current', 0)
total = data.get('total', 0)
# Print every 5% change or status change
if abs(progress - last_progress) >= 5 or status in ('complete', 'error'):
current_mb = current / (1024 * 1024)
total_mb = total / (1024 * 1024)
print(f" 📊 {status:12} {progress:6.1f}% ({current_mb:.1f}MB / {total_mb:.1f}MB) {filename[:50]}")
last_progress = progress
# Stop if complete or error
if status in ("complete", "error"):
if status == "complete":
print(f" ✅ Download complete!")
else:
print(f" ❌ Download error: {data.get('error', 'unknown')}")
break
except json.JSONDecodeError as e:
print(f" ⚠️ Error parsing JSON: {e}")
elif line.startswith(": heartbeat"):
# Heartbeat every 1 second, don't spam
pass
except asyncio.CancelledError:
print(" ⏹️ SSE monitor cancelled")
except Exception as e:
print(f" ❌ SSE error: {e}")
return events
async def trigger_download(model_name: str) -> bool:
"""Trigger a model download via the API."""
url = "http://localhost:8000/models/download"
print(f"\n🚀 Triggering download for: {model_name}")
try:
async with httpx.AsyncClient(timeout=30) as client:
response = await client.post(url, json={"model_name": model_name})
result = response.json()
print(f" Response: {response.status_code} - {result}")
return response.status_code == 200
except Exception as e:
print(f" ❌ Error triggering download: {e}")
return False
async def delete_model(model_name: str) -> bool:
"""Delete a model from cache."""
url = f"http://localhost:8000/models/{model_name}"
print(f"\n🗑️ Deleting model: {model_name}")
try:
async with httpx.AsyncClient(timeout=30) as client:
response = await client.delete(url)
if response.status_code == 200:
print(f" ✅ Model deleted")
return True
elif response.status_code == 404:
print(f" ℹ️ Model not found (already deleted)")
return True
else:
print(f" ⚠️ Delete response: {response.status_code} - {response.text}")
return False
except Exception as e:
print(f" ❌ Error deleting model: {e}")
return False
async def check_model_status(model_name: str) -> Optional[Dict]:
"""Check the status of a model."""
try:
async with httpx.AsyncClient(timeout=10) as client:
response = await client.get("http://localhost:8000/models/status")
if response.status_code == 200:
data = response.json()
for model in data.get("models", []):
if model["model_name"] == model_name:
return model
except Exception as e:
print(f" ⚠️ Error checking model status: {e}")
return None
async def check_server() -> bool:
"""Check if the server is running."""
try:
async with httpx.AsyncClient(timeout=5) as client:
response = await client.get("http://localhost:8000/health")
return response.status_code == 200
except Exception:
return False
async def main():
print("=" * 70)
print("🧪 Qwen TTS Model Download Progress Test")
print("=" * 70)
print("\nThis test verifies that MLX TTS download progress tracking works.")
print("It specifically tests the tqdm patching for mlx_audio.tts imports.")
# Check if server is running
print("\n📡 Checking if server is running...")
if not await check_server():
print(" ❌ Server is not running on http://localhost:8000")
print("\n Please start the server first:")
print(" cd backend && python main.py")
return False
print(" ✅ Server is running")
# Test model
model_name = "qwen-tts-0.6B" # Note: 0.6B currently maps to 1.7B on MLX
# Check current status
print(f"\n📊 Checking status of {model_name}...")
status = await check_model_status(model_name)
if status:
print(f" Downloaded: {status.get('downloaded', False)}")
print(f" Downloading: {status.get('downloading', False)}")
print(f" Loaded: {status.get('loaded', False)}")
if status.get('size_mb'):
print(f" Size: {status['size_mb']:.1f} MB")
else:
print(" ⚠️ Could not get model status")
# Ask if user wants to delete first
print("\n" + "-" * 70)
if status and status.get('downloaded'):
print("⚠️ Model is already downloaded. Delete it for a fresh download test?")
print(" [y] Yes, delete and download fresh")
print(" [n] No, just test SSE connection")
print(" [q] Quit")
choice = input("\nChoice [y/n/q]: ").strip().lower()
if choice == 'q':
print("Exiting...")
return True
if choice == 'y':
if not await delete_model(model_name):
print("Failed to delete model. Continue anyway? [y/n]")
if input().strip().lower() != 'y':
return False
else:
print("Model not downloaded. Will perform fresh download test.")
input("Press Enter to continue...")
# Run the test
print("\n" + "=" * 70)
print("🏃 Starting Download Test")
print("=" * 70)
async def run_test():
# Start SSE monitor in background FIRST
monitor_task = asyncio.create_task(monitor_sse_stream(model_name, timeout=600))
# Wait for SSE to connect
await asyncio.sleep(1)
# Trigger download
success = await trigger_download(model_name)
if not success:
print(" ❌ Failed to trigger download")
monitor_task.cancel()
try:
await monitor_task
except asyncio.CancelledError:
pass
return []
# Wait for SSE monitor to complete
print("\n⏳ Waiting for download to complete (this may take several minutes)...")
events = await monitor_task
return events
start_time = time.time()
events = await run_test()
elapsed = time.time() - start_time
# Results
print("\n" + "=" * 70)
print("📋 Test Results")
print("=" * 70)
print(f"\n⏱️ Elapsed time: {elapsed:.1f} seconds")
print(f"📨 Total SSE events received: {len(events)}")
if not events:
print("\n❌ FAILED - No SSE events received!")
print("\nPossible causes:")
print(" 1. SSE endpoint not working")
print(" 2. tqdm not patched before mlx_audio import")
print(" 3. Progress callbacks not firing")
print(" 4. Model already fully downloaded")
print("\nDebug steps:")
print(" 1. Check server logs for [DEBUG] messages")
print(" 2. Look for 'tqdm patched' before 'mlx_audio.tts import'")
print(f" 3. Delete model: curl -X DELETE http://localhost:8000/models/{model_name}")
return False
# Analyze events
first_event = events[0]
last_event = events[-1]
print(f"\n📊 First event:")
print(f" Status: {first_event.get('status')}")
print(f" Progress: {first_event.get('progress', 0):.1f}%")
print(f"\n📊 Last event:")
print(f" Status: {last_event.get('status')}")
print(f" Progress: {last_event.get('progress', 0):.1f}%")
# Check for expected behaviors
has_progress_updates = len(events) > 2
has_increasing_progress = False
has_complete = any(e.get('status') == 'complete' for e in events)
has_100_percent = any(e.get('progress', 0) >= 100 for e in events)
# Check if progress increased over time
if len(events) >= 2:
progress_values = [e.get('progress', 0) for e in events]
has_increasing_progress = progress_values[-1] > progress_values[0]
print("\n📋 Checks:")
print(f" {'✅' if has_progress_updates else '❌'} Multiple progress updates received ({len(events)} events)")
print(f" {'✅' if has_increasing_progress else '❌'} Progress increased over time")
print(f" {'✅' if has_100_percent else '❌'} Reached 100% progress")
print(f" {'✅' if has_complete else '❌'} Received 'complete' status")
# Overall result
success = has_progress_updates and has_complete
if success:
print("\n" + "=" * 70)
print("✅ TEST PASSED - Qwen TTS download progress tracking works!")
print("=" * 70)
else:
print("\n" + "=" * 70)
print("❌ TEST FAILED - Progress tracking has issues")
print("=" * 70)
print("\nCheck the server logs for debug output.")
return success
if __name__ == "__main__":
result = asyncio.run(main())
exit(0 if result else 1)
+178
View File
@@ -0,0 +1,178 @@
"""
Test real model download with SSE progress monitoring.
"""
import asyncio
import json
import httpx
import time
from typing import List, Dict
async def monitor_sse_stream(model_name: str, timeout: int = 300):
"""Monitor SSE stream for a model download."""
events: List[Dict] = []
url = f"http://localhost:8000/models/progress/{model_name}"
print(f"Connecting to SSE endpoint: {url}")
async with httpx.AsyncClient(timeout=timeout) as client:
async with client.stream("GET", url) as response:
print(f"SSE connected, status: {response.status_code}")
if response.status_code != 200:
print(f"Error: SSE endpoint returned {response.status_code}")
return events
async for line in response.aiter_lines():
if not line:
continue
print(f" Raw SSE: {line[:100]}...") # Print first 100 chars
if line.startswith("data: "):
try:
data = json.loads(line[6:])
print(f" → {data['status']:12} {data.get('progress', 0):6.1f}% {data.get('filename', '')}")
events.append(data)
# Stop if complete or error
if data.get("status") in ("complete", "error"):
print(f" Download {data['status']}!")
break
except json.JSONDecodeError as e:
print(f" Error parsing JSON: {e}")
print(f" Line was: {line}")
elif line.startswith(": heartbeat"):
print(" ♥ heartbeat")
return events
async def trigger_download(model_name: str):
"""Trigger a model download via the API."""
url = "http://localhost:8000/models/download"
print(f"\nTriggering download for: {model_name}")
async with httpx.AsyncClient(timeout=300) as client:
response = await client.post(url, json={"model_name": model_name})
print(f"Response: {response.status_code} - {response.json()}")
return response.status_code == 200
async def check_server():
"""Check if the server is running."""
try:
async with httpx.AsyncClient(timeout=5) as client:
response = await client.get("http://localhost:8000/health")
return response.status_code == 200
except Exception as e:
print(f"Server not running: {e}")
return False
async def main():
print("=" * 60)
print("Real Model Download Progress Test")
print("=" * 60)
# Check if server is running
print("\nChecking if server is running...")
if not await check_server():
print("✗ Server is not running on http://localhost:8000")
print("\nPlease start the server first:")
print(" cd backend && python main.py")
return False
print("✓ Server is running")
# Choose a small model for testing
model_name = "whisper-base" # ~150MB, faster to download
print(f"\nUsing model: {model_name}")
# Option to delete model first if it exists
print("\nDo you want to delete the model first to force a fresh download? (y/n)")
# For automated testing, skip deletion prompt
# delete_first = input().strip().lower() == 'y'
delete_first = False
if delete_first:
print(f"Deleting {model_name}...")
async with httpx.AsyncClient(timeout=30) as client:
response = await client.delete(f"http://localhost:8000/models/{model_name}")
print(f"Delete response: {response.status_code}")
print("\n" + "=" * 60)
print("Starting Test")
print("=" * 60)
# Start monitoring SSE stream BEFORE triggering download
async def run_test():
# Start SSE monitor in background
monitor_task = asyncio.create_task(monitor_sse_stream(model_name))
# Wait a bit to ensure SSE is connected
await asyncio.sleep(1)
# Trigger download
success = await trigger_download(model_name)
if not success:
print("✗ Failed to trigger download")
monitor_task.cancel()
return False
# Wait for SSE monitor to complete
events = await monitor_task
return events
events = await run_test()
# Results
print("\n" + "=" * 60)
print("Test Results")
print("=" * 60)
if not events:
print("✗ FAILED - No SSE events received!")
print("\nPossible causes:")
print(" 1. SSE endpoint not working")
print(" 2. Progress updates not being sent")
print(" 3. Model already downloaded (no progress to report)")
print("\nTry deleting the model first to force a fresh download:")
print(f" curl -X DELETE http://localhost:8000/models/{model_name}")
return False
print(f"✓ Received {len(events)} SSE events")
print(f"\nFirst event: {events[0]}")
print(f"Last event: {events[-1]}")
# Check if we got meaningful progress
has_progress = any(e.get('progress', 0) > 0 for e in events)
has_complete = any(e.get('status') == 'complete' for e in events)
if has_progress:
print("✓ Progress updates received")
else:
print("✗ No progress updates (might be already downloaded)")
if has_complete:
print("✓ Download completed successfully")
else:
print("✗ Download did not complete")
success = has_progress and has_complete
if success:
print("\n✓ TEST PASSED - Progress tracking works!")
else:
print("\n⊘ TEST INCONCLUSIVE - Try with a fresh download")
return success
if __name__ == "__main__":
asyncio.run(main())
-8
View File
@@ -32,11 +32,3 @@ def audio_to_wav_bytes(audio: np.ndarray, sample_rate: int) -> bytes:
sf.write(buffer, audio, sample_rate, format="WAV")
buffer.seek(0)
return buffer.read()
def audio_to_wav_bytes(audio: np.ndarray, sample_rate: int) -> bytes:
"""Convert audio array to WAV bytes."""
buffer = io.BytesIO()
sf.write(buffer, audio, sample_rate, format="WAV")
buffer.seek(0)
return buffer.read()
+153 -31
View File
@@ -11,8 +11,9 @@ import sys
class HFProgressTracker:
"""Tracks HuggingFace Hub download progress by intercepting tqdm."""
def __init__(self, progress_callback: Optional[Callable] = None):
def __init__(self, progress_callback: Optional[Callable] = None, filter_non_downloads: bool = False):
self.progress_callback = progress_callback
self.filter_non_downloads = filter_non_downloads # Only filter if True
self._original_tqdm_class = None
self._lock = threading.Lock()
self._total_downloaded = 0
@@ -21,6 +22,7 @@ class HFProgressTracker:
self._file_downloaded = {} # Track downloaded bytes per file
self._current_filename = ""
self._active_tqdms = {} # Track active tqdm instances
self._hf_tqdm_original_update = None # For monkey-patching hf's tqdm
def _create_tracked_tqdm_class(self):
"""Create a tqdm subclass that tracks progress."""
@@ -31,7 +33,6 @@ class HFProgressTracker:
"""A tqdm subclass that reports progress to our tracker."""
def __init__(self, *args, **kwargs):
print(f"[DEBUG TrackedTqdm] __init__ called with desc: {kwargs.get('desc', '')}")
# Extract filename from desc before passing to parent
desc = kwargs.get("desc", "")
if not desc and args:
@@ -80,7 +81,6 @@ class HFProgressTracker:
}
def update(self, n=1):
print(f"[DEBUG TrackedTqdm] update called with n={n}")
result = super().update(n)
# Report progress
@@ -91,6 +91,16 @@ class HFProgressTracker:
total = getattr(self, "total", 0)
if total and total > 0:
# Always filter out non-byte progress bars (e.g., "Fetching 12 files")
# These cause crazy percentages because they're counting files, not bytes
if self._is_non_byte_progress(filename):
return result
# When model is cached, also filter out generation-related progress
if tracker.filter_non_downloads:
if not self._is_download_progress(filename):
return result
# Update per-file tracking
tracker._file_sizes[filename] = total
tracker._file_downloaded[filename] = current
@@ -99,6 +109,13 @@ class HFProgressTracker:
tracker._total_size = sum(tracker._file_sizes.values())
tracker._total_downloaded = sum(tracker._file_downloaded.values())
# Only report progress once we have a meaningful total (at least 1MB)
# This avoids the "100% at 0MB" issue when small config
# files are counted before the real model files
MIN_TOTAL_BYTES = 1_000_000 # 1MB
if tracker._total_size < MIN_TOTAL_BYTES:
return result
# Call progress callback
if tracker.progress_callback:
tracker.progress_callback(
@@ -109,6 +126,50 @@ class HFProgressTracker:
return result
def _is_non_byte_progress(self, filename: str) -> bool:
"""Check if this progress bar should be SKIPPED (returns True to skip).
We want to track byte-based progress bars. This method identifies
progress bars that count files/items instead of bytes, which would
cause crazy percentages if mixed with our byte counting.
Returns:
True = SKIP this bar (it's not byte-based)
False = TRACK this bar (it counts bytes)
"""
if not filename:
return False
filename_lower = filename.lower()
# Skip "Fetching X files" - it counts files (total=12), not bytes
# Don't skip "Downloading (incomplete total...)" - that IS byte-based
skip_patterns = [
'fetching', # "Fetching 12 files" has total=12 files, not bytes
]
return any(pattern in filename_lower for pattern in skip_patterns)
def _is_download_progress(self, filename: str) -> bool:
"""Check if this is a real file download progress bar vs internal processing."""
if not filename or filename == "unknown":
return False
# Real downloads have file extensions
download_extensions = [
'.safetensors', '.bin', '.pt', '.pth', # Model weights
'.json', '.txt', '.py', # Config files
'.msgpack', '.h5', # Other formats
]
filename_lower = filename.lower()
has_extension = any(filename_lower.endswith(ext) for ext in download_extensions)
# Skip generation-related progress indicators
skip_patterns = ['segment', 'processing', 'generating', 'loading']
has_skip_pattern = any(pattern in filename_lower for pattern in skip_patterns)
return has_extension and not has_skip_pattern
def close(self):
with tracker._lock:
if id(self) in tracker._active_tqdms:
@@ -120,13 +181,11 @@ class HFProgressTracker:
@contextmanager
def patch_download(self):
"""Context manager to patch tqdm for progress tracking."""
print("[DEBUG HFProgressTracker] patch_download called")
try:
import tqdm as tqdm_module
# Store original tqdm class
self._original_tqdm_class = tqdm_module.tqdm
print(f"[DEBUG HFProgressTracker] Original tqdm class: {self._original_tqdm_class}")
# Reset totals
with self._lock:
@@ -139,39 +198,89 @@ class HFProgressTracker:
# Create our tracked tqdm class
tracked_tqdm = self._create_tracked_tqdm_class()
print(f"[DEBUG HFProgressTracker] Created TrackedTqdm class: {tracked_tqdm}")
# Patch tqdm.tqdm
tqdm_module.tqdm = tracked_tqdm
print(f"[DEBUG HFProgressTracker] Patched tqdm.tqdm")
# Also patch tqdm.auto.tqdm if it exists (used by huggingface_hub)
self._original_tqdm_auto = None
if hasattr(tqdm_module, "auto") and hasattr(tqdm_module.auto, "tqdm"):
self._original_tqdm_auto = tqdm_module.auto.tqdm
tqdm_module.auto.tqdm = tracked_tqdm
print(f"[DEBUG HFProgressTracker] Patched tqdm.auto.tqdm")
# Patch in sys.modules to catch already-imported references
# huggingface_hub uses: from tqdm.auto import tqdm as base_tqdm
# So we need to patch both 'tqdm' and 'base_tqdm' attributes
self._patched_modules = {}
tqdm_attr_names = ['tqdm', 'base_tqdm', 'old_tqdm'] # Various names used
patched_count = 0
for module_name in list(sys.modules.keys()):
if "huggingface" in module_name or module_name.startswith("tqdm"):
try:
module = sys.modules[module_name]
if hasattr(module, "tqdm"):
attr = getattr(module, "tqdm")
# Only patch if it's the original tqdm class (not already patched)
if attr is self._original_tqdm_class or (
hasattr(attr, "__name__") and attr.__name__ == "tqdm"
):
self._patched_modules[module_name] = attr
setattr(module, "tqdm", tracked_tqdm)
patched_count += 1
print(f"[DEBUG HFProgressTracker] Patched {module_name}.tqdm")
for attr_name in tqdm_attr_names:
if hasattr(module, attr_name):
attr = getattr(module, attr_name)
# Only patch if it's a tqdm class (not already patched)
is_tqdm_class = (
attr is self._original_tqdm_class or
(self._original_tqdm_auto and attr is self._original_tqdm_auto) or
(hasattr(attr, "__name__") and attr.__name__ == "tqdm" and
hasattr(attr, "update")) # tqdm classes have update method
)
if is_tqdm_class:
key = f"{module_name}.{attr_name}"
self._patched_modules[key] = (module, attr_name, attr)
setattr(module, attr_name, tracked_tqdm)
patched_count += 1
except (AttributeError, TypeError):
pass
print(f"[DEBUG HFProgressTracker] Patched {patched_count} modules in sys.modules")
# ALSO monkey-patch the update method on huggingface_hub's tqdm class
# This is needed because the class was already defined at import time
self._hf_tqdm_original_update = None
try:
from huggingface_hub.utils import tqdm as hf_tqdm_module
if hasattr(hf_tqdm_module, 'tqdm'):
hf_tqdm_class = hf_tqdm_module.tqdm
self._hf_tqdm_original_update = hf_tqdm_class.update
# Create a wrapper that calls our tracking
tracker = self # Reference to HFProgressTracker instance
def patched_update(tqdm_self, n=1):
result = tracker._hf_tqdm_original_update(tqdm_self, n)
# Track this progress
with tracker._lock:
desc = getattr(tqdm_self, 'desc', '') or ''
current = getattr(tqdm_self, 'n', 0)
total = getattr(tqdm_self, 'total', 0) or 0
# Skip non-byte progress bars
if 'fetching' in desc.lower():
return result
# Skip until we have a meaningful total (at least 1MB)
# This avoids the "100% at 0MB" issue when small config
# files are counted before the real model files
MIN_TOTAL_BYTES = 1_000_000 # 1MB
if total >= MIN_TOTAL_BYTES:
tracker._total_downloaded = current
tracker._total_size = total
if tracker.progress_callback:
tracker.progress_callback(current, total, desc)
return result
hf_tqdm_class.update = patched_update
patched_count += 1
print(f"[HFProgressTracker] Monkey-patched huggingface_hub.utils.tqdm.tqdm.update")
except (ImportError, AttributeError) as e:
print(f"[HFProgressTracker] Could not monkey-patch hf_tqdm: {e}")
print(f"[HFProgressTracker] Patched {patched_count} tqdm references")
yield
@@ -189,15 +298,24 @@ class HFProgressTracker:
tqdm_module.auto.tqdm = self._original_tqdm_auto
# Restore patched modules
for module_name, original in self._patched_modules.items():
for key, (module, attr_name, original) in self._patched_modules.items():
try:
module = sys.modules.get(module_name)
if module and original:
setattr(module, "tqdm", original)
setattr(module, attr_name, original)
except (AttributeError, TypeError):
pass
self._patched_modules = {}
# Restore hf_tqdm's original update method
if self._hf_tqdm_original_update:
try:
from huggingface_hub.utils import tqdm as hf_tqdm_module
if hasattr(hf_tqdm_module, 'tqdm'):
hf_tqdm_module.tqdm.update = self._hf_tqdm_original_update
except (ImportError, AttributeError):
pass
self._hf_tqdm_original_update = None
except (ImportError, AttributeError):
pass
@@ -205,13 +323,17 @@ class HFProgressTracker:
def create_hf_progress_callback(model_name: str, progress_manager):
"""Create a progress callback for HuggingFace downloads."""
def callback(downloaded: int, total: int, filename: str = ""):
"""Progress callback."""
if total > 0:
progress_manager.update_progress(
model_name=model_name,
current=downloaded,
total=total,
filename=filename or "",
status="downloading",
)
"""Progress callback.
Note: We send updates even when total=0 (unknown) to provide feedback
during the "incomplete total" phase of huggingface_hub downloads.
The frontend handles total=0 gracefully.
"""
progress_manager.update_progress(
model_name=model_name,
current=downloaded,
total=total,
filename=filename or "",
status="downloading",
)
return callback
+42 -11
View File
@@ -16,11 +16,17 @@ class ProgressManager:
Thread-safe: can be called from background threads (e.g., via asyncio.to_thread).
"""
# Throttle settings to prevent overwhelming SSE clients
THROTTLE_INTERVAL_SECONDS = 0.5 # Minimum time between updates
THROTTLE_PROGRESS_DELTA = 1.0 # Minimum progress change (%) to force update
def __init__(self):
self._progress: Dict[str, Dict] = {}
self._listeners: Dict[str, list] = {}
self._lock = threading.Lock() # Thread-safe lock for progress dict
self._main_loop: Optional[asyncio.AbstractEventLoop] = None
self._last_notify_time: Dict[str, float] = {} # Last notification time per model
self._last_notify_progress: Dict[str, float] = {} # Last notified progress per model
def _set_main_loop(self, loop: asyncio.AbstractEventLoop):
"""Set the main event loop for thread-safe operations."""
@@ -67,6 +73,10 @@ class ProgressManager:
Update progress for a model download.
Thread-safe: can be called from background threads.
Progress updates are throttled to prevent overwhelming SSE clients.
Updates are sent at most every THROTTLE_INTERVAL_SECONDS, or when
progress changes by at least THROTTLE_PROGRESS_DELTA percent.
Args:
model_name: Name of the model (e.g., "qwen-tts-1.7B", "whisper-base")
@@ -76,9 +86,17 @@ class ProgressManager:
status: Status string (downloading, extracting, complete, error)
"""
import logging
import time
logger = logging.getLogger(__name__)
progress_pct = (current / total * 100) if total > 0 else 0
# Calculate progress percentage, clamped to 0-100 range
# This prevents crazy percentages from edge cases like:
# - current > total temporarily during aggregation
# - mixing file-count progress with byte-count progress
if total > 0:
progress_pct = min(100.0, max(0.0, (current / total * 100)))
else:
progress_pct = 0
progress_data = {
"model_name": model_name,
@@ -90,25 +108,38 @@ class ProgressManager:
"timestamp": datetime.now().isoformat(),
}
print(f"[DEBUG] update_progress called: {model_name}, {progress_pct:.1f}%")
# Thread-safe update of progress dict
# Thread-safe update of progress dict (always update internal state)
with self._lock:
self._progress[model_name] = progress_data
# Check if we should notify listeners (throttling)
current_time = time.time()
last_time = self._last_notify_time.get(model_name, 0)
last_progress = self._last_notify_progress.get(model_name, -100)
time_delta = current_time - last_time
progress_delta = abs(progress_pct - last_progress)
# Always notify for complete/error status, or if throttle conditions are met
should_notify = (
status in ("complete", "error") or
time_delta >= self.THROTTLE_INTERVAL_SECONDS or
progress_delta >= self.THROTTLE_PROGRESS_DELTA
)
if not should_notify:
return # Skip this update (throttled)
# Update throttle tracking
self._last_notify_time[model_name] = current_time
self._last_notify_progress[model_name] = progress_pct
# Notify all listeners (thread-safe)
listener_count = len(self._listeners.get(model_name, []))
print(f"[DEBUG] Listener count for {model_name}: {listener_count}")
print(f"[DEBUG] All listeners: {list(self._listeners.keys())}")
print(f"[DEBUG] Main loop set: {self._main_loop is not None}")
if self._main_loop:
print(f"[DEBUG] Main loop running: {self._main_loop.is_running()}")
if listener_count > 0:
logger.debug(f"Notifying {listener_count} listeners for {model_name}: {progress_pct:.1f}% ({filename})")
print(f"[DEBUG] About to notify listeners...")
self._notify_listeners_threadsafe(model_name, progress_data)
print(f"[DEBUG] Notified listeners")
else:
logger.debug(f"No listeners for {model_name}, progress update stored: {progress_pct:.1f}%")
+8 -3
View File
@@ -6,8 +6,13 @@ from PyInstaller.utils.hooks import copy_metadata
datas = []
hiddenimports = ['backend', 'backend.main', 'backend.config', 'backend.database', 'backend.models', 'backend.profiles', 'backend.history', 'backend.tts', 'backend.transcribe', 'backend.platform_detect', 'backend.backends', 'backend.backends.pytorch_backend', 'backend.utils.audio', 'backend.utils.cache', 'backend.utils.progress', 'backend.utils.hf_progress', 'backend.utils.validation', 'torch', 'transformers', 'fastapi', 'uvicorn', 'sqlalchemy', 'librosa', 'soundfile', 'qwen_tts', 'qwen_tts.inference', 'qwen_tts.inference.qwen3_tts_model', 'qwen_tts.inference.qwen3_tts_tokenizer', 'qwen_tts.core', 'qwen_tts.cli', 'pkg_resources.extern', 'backend.backends.mlx_backend', 'mlx', 'mlx.core', 'mlx.nn', 'mlx_audio', 'mlx_audio.tts', 'mlx_audio.stt']
datas += collect_data_files('qwen_tts')
datas += collect_data_files('mlx')
datas += collect_data_files('mlx_audio')
# Use collect_all (not collect_data_files) so native .dylib and .metallib
# files are bundled as binaries, not data. Without this, MLX raises OSError
# when loading Metal shaders inside the PyInstaller bundle.
from PyInstaller.utils.hooks import collect_all as _collect_all
_mlx_datas, _mlx_bins, _mlx_hidden = _collect_all('mlx')
_mlxa_datas, _mlxa_bins, _mlxa_hidden = _collect_all('mlx_audio')
datas += _mlx_datas + _mlxa_datas
datas += copy_metadata('qwen-tts')
hiddenimports += collect_submodules('qwen_tts')
hiddenimports += collect_submodules('jaraco')
@@ -18,7 +23,7 @@ hiddenimports += collect_submodules('mlx_audio')
a = Analysis(
['server.py'],
pathex=[],
binaries=[],
binaries=_mlx_bins + _mlxa_bins,
datas=datas,
hiddenimports=hiddenimports,
hookspath=[],
+8 -4
View File
@@ -13,7 +13,7 @@
},
"app": {
"name": "@voicebox/app",
"version": "0.1.9",
"version": "0.1.11",
"dependencies": {
"@dnd-kit/core": "^6.3.1",
"@dnd-kit/sortable": "^10.0.0",
@@ -68,7 +68,7 @@
},
"landing": {
"name": "@voicebox/landing",
"version": "0.1.9",
"version": "0.1.11",
"dependencies": {
"@radix-ui/react-separator": "^1.1.8",
"@radix-ui/react-slot": "^1.2.4",
@@ -93,10 +93,14 @@
},
"tauri": {
"name": "@voicebox/tauri",
"version": "0.1.9",
"version": "0.1.11",
"dependencies": {
"@tauri-apps/api": "^2.0.0",
"@tauri-apps/plugin-dialog": "^2.0.0",
"@tauri-apps/plugin-fs": "^2.0.0",
"@tauri-apps/plugin-process": "^2.0.0",
"@tauri-apps/plugin-shell": "^2.0.0",
"@tauri-apps/plugin-updater": "^2.0.0",
},
"devDependencies": {
"@tailwindcss/vite": "^4.1.18",
@@ -112,7 +116,7 @@
},
"web": {
"name": "@voicebox/web",
"version": "0.1.9",
"version": "0.1.11",
"dependencies": {
"@tanstack/react-query": "^5.0.0",
"react": "^18.3.0",
+1 -1
View File
@@ -5,7 +5,7 @@ description: "Welcome to Voicebox - the open-source voice synthesis studio"
## What is Voicebox?
Voicebox is a **local-first voice cloning studio** with DAW-like features for professional voice synthesis. Think of it as the **Ollama for voice** — download models, clone voices, and generate speech entirely on your machine.
Voicebox is a **local-first voice cloning studio** with DAW-like features for professional voice synthesis. Think of it as a **local, free and open-source alternative to ElevenLabs** — download models, clone voices, and generate speech entirely on your machine.
<Frame>
<img src="/images/app-screenshot-1.webp" alt="Voicebox App Screenshot" />
+964
View File
@@ -0,0 +1,964 @@
# TTS Provider Architecture
**Status:** Planned for v0.1.13
**Created:** 2025-01-31
**Problem:** GitHub 2GB release limit + poor UX for frequent updates requiring 2.4GB re-downloads
---
## Overview
Split the monolithic backend into modular components:
1. **Main App** (~150-200MB): Tauri + FastAPI backend + Whisper + UI/profiles/history
2. **TTS Providers** (downloadable plugins): Separate executables for model inference
This architecture solves:
- ✅ GitHub 2GB release artifact limit
- ✅ Frequent app updates without re-downloading large python binaries
- ✅ User choice of compute backend (CPU/GPU/Cloud)
- ✅ External provider support (OpenAI, custom servers)
- ✅ Future extensibility
---
## Architecture Diagram
```
┌─────────────────────────────────────────────────────────┐
│ Voicebox App (Tauri + Backend) ~150MB │
│ ├─ UI Layer (React) │
│ ├─ Backend (FastAPI) │
│ │ ├─ Voice Profiles │
│ │ ├─ Generation History │
│ │ ├─ Audio Editing / Stories │
│ │ └─ Provider Manager ◄──────────────┐ │
│ └─ Whisper (bundled, tiny ~50MB) │ │
└─────────────────────────────────────────┼────────────────┘
│
HTTP/IPC │
│
┌────────────────────────────────┼─────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌──────────────────┐
│ TTS Provider: │ │ TTS Provider: │ │ TTS Provider: │
│ PyTorch CPU │ │ PyTorch CUDA │ │ MLX (Apple) │
│ │ │ │ │ │
│ ~300MB │ │ ~2.4GB │ │ ~800MB │
│ │ │ │ │ │
│ Local inference │ │ GPU inference │ │ Metal inference │
└─────────────────┘ └─────────────────┘ └──────────────────┘
│ │ │
└────────────────────────┴─────────────────────┘
│
┌─────────────▼──────────────┐
│ Future Providers: │
│ • Remote Server │
│ • OpenAI API │
│ • ElevenLabs │
│ • Custom Docker Container │
└────────────────────────────┘
```
---
## Problem Statement
### Current Architecture Issues
**Monolithic Binary:**
- CPU version: ~295MB
- CUDA version: ~2.37GB
- GitHub releases: 2GB file size limit (BLOCKED)
- Updates require re-downloading entire binary
- Poor UX: update app → restart → download CUDA update → restart again
**User Pain Points:**
1. Cannot release CUDA version on GitHub (over 2GB)
2. Every app update forces 2.4GB re-download for GPU users
3. No flexibility (can't use OpenAI, remote servers, etc.)
4. Wastes bandwidth for small bug fixes
---
## Solution: Pluggable TTS Providers
### Component Breakdown
#### 1. Main App (voicebox.exe / .app / .AppImage)
**Size:** ~100-150MB
**Includes:**
- Tauri runtime + React UI
- FastAPI backend (pure Python, no PyTorch)
- Whisper model (tiny, ~50MB)
- SQLite database
- Profile/history/audio editing logic
- Provider management system
**Does NOT include:**
- PyTorch (CPU or CUDA)
- TTS models (Qwen3-TTS)
- Heavy ML dependencies
**Updates frequently:** UI fixes, feature additions, non-ML changes
---
#### 2. TTS Provider: PyTorch CPU
**Binary:** `tts-provider-pytorch-cpu.exe`
**Size:** ~200MB
**Includes:**
- PyTorch CPU build
- Qwen3-TTS package
- Transformers
- No CUDA libraries
**Download source:** Cloudflare R2
**Updates rarely:** Only when model code changes
---
#### 3. TTS Provider: PyTorch CUDA
**Binary:** `tts-provider-pytorch-cuda.exe`
**Size:** ~2.4GB
**Includes:**
- PyTorch CUDA build (cu121)
- Qwen3-TTS package
- CUDA runtime, cuDNN, cuBLAS
- Transformers
**Download source:** Cloudflare R2
**Platform:** Windows + Linux (NVIDIA GPU)
**Updates rarely:** Only when model code or CUDA version changes
---
#### 4. TTS Provider: MLX
**Binary:** `tts-provider-mlx`
**Size:** ~150MB
**Includes:**
- MLX framework
- MLX-optimized Qwen3-TTS
- Metal acceleration
**Platform:** macOS only (Apple Silicon)
**Download source:** Cloudflare R2
---
#### 5. TTS Provider: Remote
**Binary:** None (built-in config)
**Size:** 0MB
**How it works:**
- User provides URL to their own TTS server
- Backend proxies requests to that server
- Implements API spec from `EXTERNAL_PROVIDERS.md`
**Use cases:**
- AMD GPU users running their own server
- Team deployments with shared GPU server
- Cloud hosting (Modal, RunPod, Replicate)
---
#### 6. TTS Provider: OpenAI
**Binary:** None (API wrapper)
**Size:** 0MB
**How it works:**
- User provides OpenAI API key
- Backend wraps OpenAI Audio API
- Voice profiles map to OpenAI voices
**Benefits:**
- Zero local compute
- Pay-per-use
- Instant setup
---
## Communication Protocol
### Provider API Specification
All TTS providers must implement these endpoints:
#### POST /tts/generate
Generate speech from text.
**Request:**
```json
{
"text": "Hello world!",
"voice_prompt": {
/* voice prompt object */
},
"language": "en",
"seed": 12345,
"model_size": "1.7B"
}
```
**Response:**
```json
{
"audio": "base64-encoded-audio",
"sample_rate": 24000,
"duration": 2.5
}
```
#### POST /tts/create_voice_prompt
Create voice prompt from reference audio.
**Request:** (multipart/form-data)
- `audio`: Audio file
- `reference_text`: Transcript
**Response:**
```json
{
"voice_prompt": {
/* serialized prompt */
}
}
```
#### GET /tts/health
Health check.
**Response:**
```json
{
"status": "healthy",
"provider": "pytorch-cuda",
"version": "1.0.0",
"model": "Qwen3-TTS-12Hz-1.7B-Base",
"device": "cuda:0"
}
```
#### GET /tts/status
Model status.
**Response:**
```json
{
"model_loaded": true,
"model_size": "1.7B",
"available_sizes": ["0.6B", "1.7B"],
"gpu_available": true,
"vram_used_mb": 1234
}
```
---
## Backend Implementation
### Provider Manager
**File:** `backend/providers/__init__.py`
```python
class ProviderManager:
"""Manages TTS provider lifecycle."""
def __init__(self):
self.active_provider: Optional[Provider] = None
self.config = load_provider_config()
async def start_provider(self, provider_type: str) -> str:
"""Start a TTS provider process."""
if provider_type == "pytorch-cpu":
return await self._start_local_provider("tts-provider-pytorch-cpu.exe")
elif provider_type == "pytorch-cuda":
return await self._start_local_provider("tts-provider-pytorch-cuda.exe")
elif provider_type == "mlx":
return await self._start_local_provider("tts-provider-mlx")
elif provider_type == "remote":
return self.config["remote_url"]
elif provider_type == "openai":
return None # No subprocess, API wrapper
async def _start_local_provider(self, binary_name: str) -> str:
"""Start local provider subprocess."""
provider_path = get_provider_binary_path(binary_name)
if not provider_path.exists():
raise ProviderNotInstalledException(binary_name)
# Start subprocess on random port
port = get_free_port()
process = subprocess.Popen([
str(provider_path),
"--port", str(port),
"--data-dir", str(config.get_data_dir())
])
# Wait for provider to be ready
await wait_for_provider_health(f"http://localhost:{port}")
self.active_provider = Provider(process, port)
return f"http://localhost:{port}"
async def stop_provider(self):
"""Stop active provider."""
if self.active_provider:
self.active_provider.process.terminate()
self.active_provider = None
```
---
### Provider Abstraction
**File:** `backend/providers/base.py`
```python
class TTSProvider(ABC):
"""Abstract base for TTS providers."""
@abstractmethod
async def generate(
self,
text: str,
voice_prompt: dict,
language: str,
seed: Optional[int]
) -> tuple[np.ndarray, int]:
"""Generate speech audio."""
pass
@abstractmethod
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str
) -> dict:
"""Create voice prompt from reference audio."""
pass
```
**File:** `backend/providers/local.py`
```python
class LocalProvider(TTSProvider):
"""Provider that communicates with local subprocess via HTTP."""
def __init__(self, base_url: str):
self.base_url = base_url
self.client = httpx.AsyncClient()
async def generate(self, text, voice_prompt, language, seed):
response = await self.client.post(
f"{self.base_url}/tts/generate",
json={
"text": text,
"voice_prompt": voice_prompt,
"language": language,
"seed": seed
}
)
data = response.json()
audio = np.frombuffer(base64.b64decode(data["audio"]), dtype=np.float32)
return audio, data["sample_rate"]
```
**File:** `backend/providers/openai.py`
```python
class OpenAIProvider(TTSProvider):
"""Provider that wraps OpenAI Audio API."""
def __init__(self, api_key: str):
self.client = OpenAI(api_key=api_key)
async def generate(self, text, voice_prompt, language, seed):
# Map voice_prompt to OpenAI voice name
voice = map_profile_to_openai_voice(voice_prompt)
response = await self.client.audio.speech.create(
model="tts-1",
voice=voice,
input=text
)
# Convert to numpy array
audio_data = response.content
audio, sr = load_audio_from_bytes(audio_data)
return audio, sr
```
---
## Provider Installation
### Download Manager
**File:** `backend/providers/installer.py`
```python
class ProviderInstaller:
"""Handles provider download and installation."""
async def download_provider(self, provider_type: str):
"""Download provider binary from R2."""
binary_name = {
"pytorch-cpu": "tts-provider-pytorch-cpu.exe",
"pytorch-cuda": "tts-provider-pytorch-cuda.exe",
"mlx": "tts-provider-mlx"
}[provider_type]
download_url = f"https://downloads.voicebox.sh/providers/v{PROVIDER_VERSION}/{binary_name}"
# Download with progress tracking (reuse existing SSE system)
await download_with_progress(
url=download_url,
destination=get_provider_install_path(binary_name),
progress_key=f"provider-{provider_type}"
)
```
**Provider Storage Location:**
- Windows: `%APPDATA%/voicebox/providers/`
- macOS: `~/Library/Application Support/voicebox/providers/`
- Linux: `~/.local/share/voicebox/providers/`
---
## Frontend Implementation
### Provider Settings UI
**Component:** `app/src/components/ServerSettings/ProviderSettings.tsx`
```tsx
export function ProviderSettings() {
const [selectedProvider, setSelectedProvider] =
useState<ProviderType>("auto");
const {data: installedProviders} = useQuery({
queryKey: ["providers", "installed"],
queryFn: () => apiClient.getInstalledProviders(),
});
return (
<Card>
<CardHeader>
<CardTitle>TTS Provider</CardTitle>
<CardDescription>Choose how Voicebox generates speech</CardDescription>
</CardHeader>
<CardContent>
<RadioGroup
value={selectedProvider}
onValueChange={setSelectedProvider}
>
{/* Auto-detect */}
<div className="flex items-center space-x-2">
<RadioGroupItem value="auto" id="auto" />
<Label htmlFor="auto">
<div className="font-medium">Auto-detect (Recommended)</div>
<div className="text-sm text-muted-foreground">
Automatically choose the best available provider
</div>
</Label>
</div>
{/* PyTorch CUDA */}
<div className="flex items-center justify-between">
<div className="flex items-center space-x-2">
<RadioGroupItem
value="pytorch-cuda"
id="cuda"
disabled={!gpuAvailable}
/>
<Label htmlFor="cuda">
<div className="font-medium">PyTorch CUDA (NVIDIA GPU)</div>
<div className="text-sm text-muted-foreground">
4-5x faster inference on NVIDIA GPUs
</div>
</Label>
</div>
{!installedProviders?.includes("pytorch-cuda") && gpuAvailable && (
<Button
onClick={() => downloadProvider("pytorch-cuda")}
size="sm"
>
Download (2.4GB)
</Button>
)}
</div>
{/* PyTorch CPU */}
<div className="flex items-center justify-between">
<div className="flex items-center space-x-2">
<RadioGroupItem value="pytorch-cpu" id="cpu" />
<Label htmlFor="cpu">
<div className="font-medium">PyTorch CPU</div>
<div className="text-sm text-muted-foreground">
Works on any system, slower inference
</div>
</Label>
</div>
{!installedProviders?.includes("pytorch-cpu") && (
<Button onClick={() => downloadProvider("pytorch-cpu")} size="sm">
Download (300MB)
</Button>
)}
</div>
{/* MLX (macOS only) */}
{isMacOS && (
<div className="flex items-center justify-between">
<div className="flex items-center space-x-2">
<RadioGroupItem value="mlx" id="mlx" />
<Label htmlFor="mlx">
<div className="font-medium">MLX (Apple Silicon)</div>
<div className="text-sm text-muted-foreground">
Optimized for M1/M2/M3 chips
</div>
</Label>
</div>
{!installedProviders?.includes("mlx") && (
<Button onClick={() => downloadProvider("mlx")} size="sm">
Download (800MB)
</Button>
)}
</div>
)}
{/* Remote */}
<div className="space-y-2">
<div className="flex items-center space-x-2">
<RadioGroupItem value="remote" id="remote" />
<Label htmlFor="remote">
<div className="font-medium">Remote Server</div>
<div className="text-sm text-muted-foreground">
Connect to your own TTS server
</div>
</Label>
</div>
{selectedProvider === "remote" && (
<Input placeholder="http://your-server:8000" className="ml-6" />
)}
</div>
{/* OpenAI */}
<div className="space-y-2">
<div className="flex items-center space-x-2">
<RadioGroupItem value="openai" id="openai" />
<Label htmlFor="openai">
<div className="font-medium">OpenAI API</div>
<div className="text-sm text-muted-foreground">
Use OpenAI's TTS API (requires API key)
</div>
</Label>
</div>
{selectedProvider === "openai" && (
<Input type="password" placeholder="sk-..." className="ml-6" />
)}
</div>
</RadioGroup>
</CardContent>
</Card>
);
}
```
---
## File Structure
```
voicebox/
├── backend/
│ ├── main.py # Main FastAPI app (no TTS code)
│ ├── providers/
│ │ ├── __init__.py # ProviderManager
│ │ ├── base.py # TTSProvider ABC
│ │ ├── local.py # LocalProvider (subprocess)
│ │ ├── remote.py # RemoteProvider (HTTP)
│ │ ├── openai.py # OpenAIProvider (API wrapper)
│ │ └── installer.py # Provider download logic
│ ├── profiles.py # Voice profile management
│ ├── history.py # Generation history
│ ├── transcribe.py # Whisper (still bundled)
│ └── ... (other backend modules)
│
├── providers/
│ ├── pytorch-cpu/
│ │ ├── main.py # FastAPI server for TTS
│ │ ├── tts_backend.py # PyTorch TTS logic
│ │ ├── requirements.txt # torch (CPU), qwen-tts, transformers
│ │ └── build.spec # PyInstaller spec
│ │
│ ├── pytorch-cuda/
│ │ ├── main.py # FastAPI server for TTS
│ │ ├── tts_backend.py # PyTorch TTS logic
│ │ ├── requirements.txt # torch+cu121, qwen-tts, transformers
│ │ └── build.spec # PyInstaller spec
│ │
│ └── mlx/
│ ├── main.py # FastAPI server for TTS
│ ├── mlx_backend.py # MLX TTS logic
│ ├── requirements.txt # mlx, qwen-tts-mlx
│ └── build.spec # PyInstaller spec
│
├── app/ # Frontend (Tauri + React)
│ └── src/
│ └── components/
│ └── ServerSettings/
│ └── ProviderSettings.tsx
│
└── tauri/
└── src-tauri/
└── tauri.conf.json # No externalBin for providers
```
---
## Migration Path
### Phase 1: Refactor Backend (No User Changes)
**Goal:** Abstract TTS behind provider interface
1. Create `backend/providers/` module structure
2. Implement `TTSProvider` abstract base class
3. Create `LocalProvider` wrapper for current PyTorch code
4. Modify `backend/tts.py` to use provider abstraction
5. Keep PyTorch bundled in main app
**Result:** Code is prepared, but user experience unchanged
---
### Phase 2: Build Provider Binaries
**Goal:** Create standalone TTS provider executables
1. Create separate PyInstaller specs for each provider
2. Build provider executables:
- `tts-provider-pytorch-cpu.exe` (~300MB)
- `tts-provider-pytorch-cuda.exe` (~2.4GB)
- `tts-provider-mlx` (~800MB, macOS)
3. Test subprocess communication
4. Upload providers to Cloudflare R2
**Result:** Provider binaries exist but aren't used yet
---
### Phase 3: Remove PyTorch from Main App
**Goal:** Split main app from providers
1. Exclude PyTorch/Qwen3-TTS from main app PyInstaller spec
2. Main app now requires provider download
3. Update GitHub CI to build multiple artifacts:
- `voicebox-{version}-{platform}.exe` (~150MB)
- `tts-provider-pytorch-cpu-{version}.exe`
- `tts-provider-pytorch-cuda-{version}.exe`
- `tts-provider-mlx-{version}` (macOS)
**Result:** Main app is small, providers downloaded separately
---
### Phase 4: Add Provider UI
**Goal:** User-facing provider management
1. Create Provider Settings page
2. Implement provider download UI
3. Add provider status indicators
4. Show active provider in UI
**Result:** Users can choose and download providers
---
### Phase 5: External Providers
**Goal:** Enable remote and cloud providers
1. Implement `RemoteProvider` (HTTP client)
2. Implement `OpenAIProvider` (API wrapper)
3. Add provider configuration UI (URLs, API keys)
4. Document external provider API spec
**Result:** Full provider ecosystem
---
## Provider Versioning
### Independent Versioning
Providers have their own version numbers, independent of the main app:
- **App version:** `v0.2.0` (frequent updates)
- **Provider version:** `v1.0.0` (rare updates)
### Compatibility Matrix
**Example:**
| App Version | Min Provider Version | Max Provider Version |
| ----------- | -------------------- | -------------------- |
| v0.2.0 | v1.0.0 | v1.x.x |
| v0.3.0 | v1.0.0 | v1.x.x |
| v0.4.0 | v1.2.0 | v1.x.x |
| v1.0.0 | v2.0.0 | v2.x.x |
**Backend checks compatibility:**
```python
async def check_provider_compatibility(provider_version: str) -> bool:
"""Check if provider version is compatible with current app."""
min_version = "1.0.0"
max_version = "1.999.999"
return min_version <= provider_version < max_version
```
**UI shows warning if incompatible:**
```
⚠️ Provider version 0.9.0 is outdated. Update to v1.0.0+
```
---
## User Flows
### First-Time Setup
1. User downloads and installs Voicebox (~150MB)
2. App launches → detects no TTS provider installed
3. Shows setup wizard:
```
Choose your TTS provider:
[ ] PyTorch CUDA (2.4GB) [Download]
✓ Fastest on NVIDIA GPUs
✗ Requires NVIDIA GPU
[●] PyTorch CPU (300MB) [Download]
✓ Works on any system
✗ Slower inference
[ ] MLX (800MB) [Download]
✓ Fast on Apple Silicon
✗ macOS only (M1/M2/M3)
[ ] Remote Server
URL: ___________________
[ ] OpenAI API
API Key: ________________
```
4. User selects provider → downloads with progress bar
5. Provider installs to AppData/Application Support
6. App starts provider → ready to use
---
### App Update Flow (No Provider Change)
**Scenario:** Bug fix in UI, no backend changes
1. User gets update notification: "Voicebox v0.2.1 available"
2. Downloads update (~150MB, not 2.4GB!)
3. Installs and restarts
4. **Provider stays the same** (no re-download needed)
5. App starts using existing provider
**User experience:** Fast updates, no multi-GB downloads
---
### Provider Update Flow
**Scenario:** New Qwen3-TTS model version released
1. User opens Settings → Provider tab
2. Sees notification: "Provider update available (v1.1.0)"
3. Clicks "Update Provider"
4. Downloads new provider binary
5. Old provider binary is replaced
6. Restart app to use new provider
**Frequency:** Rare (only when TTS model/backend changes)
---
### Switching Providers
**Scenario:** User upgrades to NVIDIA GPU
1. User goes to Settings → Provider
2. Selects "PyTorch CUDA"
3. Clicks "Download" → downloads 2.4GB
4. Download completes → restarts app
5. App now uses CUDA provider
---
## Benefits
| Benefit | Details |
| ----------------------------- | --------------------------------------------------------- |
| **GitHub Releases Work** | Main app ~150MB << 2GB limit |
| **Fast Updates** | UI/feature updates don't require re-downloading providers |
| **User Choice** | CPU, CUDA, MLX, OpenAI, remote server |
| **External Provider Support** | Users can run their own TTS servers |
| **Bandwidth Savings** | Only download provider once, app updates are small |
| **Future-Proof** | Easy to add new providers (ElevenLabs, custom models) |
| **Team Deployments** | Multiple users share one remote provider |
| **Cloud-Ready** | Works with Modal, Replicate, RunPod, etc. |
---
## Open Questions
### 1. Provider Versioning
**Question:** Should providers have independent versions or match app version?
**Options:**
- A. Independent (providers: v1.x, app: v0.2.x)
- B. Matched (both use v0.2.x)
**Recommendation:** Independent versioning with compatibility matrix
---
### 2. Auto-Update Providers
**Question:** Should providers auto-update separately from app?
**Options:**
- A. Manual updates only (user clicks "Update Provider")
- B. Optional auto-update (user can enable)
- C. Always auto-update
**Recommendation:** Optional auto-update (default off)
---
### 3. Provider Discovery
**Question:** How does app find installed providers?
**Options:**
- A. Check standard paths in AppData/Application Support
- B. Registry (Windows) / plist (macOS)
- C. Config file with provider locations
**Recommendation:** Standard paths + config fallback
---
### 4. Fallback Behavior
**Question:** What if no provider is installed?
**Options:**
- A. Show setup wizard on first launch
- B. Block app until provider installed
- C. Allow app to run in "demo mode" (transcription only)
**Recommendation:** Setup wizard on first launch
---
### 5. Provider Auto-Start
**Question:** Should provider start automatically with app?
**Options:**
- A. Always start selected provider on app launch
- B. Start on-demand (when user generates speech)
- C. User preference
**Recommendation:** Auto-start (configurable in settings)
---
## Future Enhancements
- [ ] **Provider Marketplace:** Built-in directory of community providers
- [ ] **Multi-Provider Support:** Use different providers per voice/language
- [ ] **Provider Health Monitoring:** Automatic failover if provider crashes
- [ ] **Cost Tracking:** Monitor API usage for OpenAI/cloud providers
- [ ] **Performance Metrics:** Latency, throughput, VRAM usage dashboards
- [ ] **Docker Providers:** Run providers in Docker containers
- [ ] **Provider Plugins:** Load custom providers from user scripts
---
## Related Documents
- [EXTERNAL_PROVIDERS.md](./EXTERNAL_PROVIDERS.md) - External provider support plan
- [OPENAI_SUPPORT.md](./OPENAI_SUPPORT.md) - OpenAI API compatibility
- [github-2gb-limit-issue.md](../github-2gb-limit-issue.md) - Original problem
- [r2-setup.md](../r2-setup.md) - Cloudflare R2 configuration
---
## Contributing
If you want to build a custom TTS provider:
1. Implement the provider API spec (see above)
2. Test with Voicebox locally
3. Package as executable (PyInstaller, Docker, etc.)
4. Share in GitHub Discussions
**Questions?**
- GitHub Issues: [voicebox/issues](https://github.com/jamiepine/voicebox/issues)
- Discord: Coming soon
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@voicebox/landing",
"version": "0.1.11",
"version": "0.1.13",
"description": "Landing page for voicebox.sh",
"scripts": {
"dev": "bun --bun next dev --turbo",
+2
View File
@@ -1,6 +1,7 @@
import type { Metadata } from 'next';
import { Inter } from 'next/font/google';
import './globals.css';
import { Banner } from '@/components/Banner';
import { Footer } from '@/components/Footer';
import { Header } from '@/components/Header';
@@ -31,6 +32,7 @@ export default function RootLayout({ children }: { children: React.ReactNode })
<html lang="en" suppressHydrationWarning className="dark">
<body className={inter.variable}>
<div className="relative min-h-screen bg-background font-sans flex flex-col">
<Banner />
<Header />
<main className="container mx-auto px-4 sm:px-6 md:px-4 flex-1 py-4 sm:py-6 md:py-0">
{children}
+3 -2
View File
@@ -239,8 +239,9 @@ export default function Home() {
<div className="space-y-6 text-lg text-foreground/80 text-center">
<p>
Voicebox is a <strong>local-first voice cloning studio</strong> with DAW-like features
for professional voice synthesis. Think of it as the <strong>Ollama for voice</strong>{' '}
— download models, clone voices, and generate speech entirely on your machine.
for professional voice synthesis. Think of it as a{' '}
<strong>local, free and open-source alternative to ElevenLabs</strong> — download
models, clone voices, and generate speech entirely on your machine.
</p>
<p>
Unlike cloud services that lock your voice data behind subscriptions, Voicebox gives
+25
View File
@@ -0,0 +1,25 @@
import { ArrowRight } from 'lucide-react';
export function Banner() {
return (
<div className="bg-primary/[0.06] border-b border-border backdrop-blur-sm">
<div className="container mx-auto px-4">
<div className="flex items-center justify-center h-10 text-sm">
<a
href="https://spacebot.sh"
target="_blank"
rel="noopener noreferrer"
className="flex items-center gap-2 text-muted-foreground hover:text-foreground transition-colors group"
>
<span>
Also by the creator of Voicebox:{' '}
<strong className="text-foreground/90">Spacebot</strong>, an AI agent OS for teams.
Connect Discord, Slack, or Telegram in one click.
</span>
<ArrowRight className="h-3.5 w-3.5 transition-transform group-hover:translate-x-0.5" />
</a>
</div>
</div>
</div>
);
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "voicebox",
"version": "0.1.11",
"version": "0.1.13",
"private": true,
"workspaces": [
"app",
+9
View File
@@ -0,0 +1,9 @@
uvicorn
fastapi
sqlalchemy
torch
torchvision
soundfile
librosa
python-multipart
huggingface_hub
+6 -2
View File
@@ -1,7 +1,7 @@
{
"name": "@voicebox/tauri",
"private": true,
"version": "0.1.11",
"version": "0.1.13",
"type": "module",
"scripts": {
"dev": "vite",
@@ -10,7 +10,11 @@
},
"dependencies": {
"@tauri-apps/api": "^2.0.0",
"@tauri-apps/plugin-shell": "^2.0.0"
"@tauri-apps/plugin-dialog": "^2.0.0",
"@tauri-apps/plugin-fs": "^2.0.0",
"@tauri-apps/plugin-process": "^2.0.0",
"@tauri-apps/plugin-shell": "^2.0.0",
"@tauri-apps/plugin-updater": "^2.0.0"
},
"devDependencies": {
"@tailwindcss/vite": "^4.1.18",
+1 -1
View File
@@ -5041,7 +5041,7 @@ checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "voicebox"
version = "0.1.11"
version = "0.1.12"
dependencies = [
"base64 0.22.1",
"core-foundation-sys",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "voicebox"
version = "0.1.11"
version = "0.1.13"
description = "A production-quality desktop app for Qwen3-TTS voice cloning and generation"
authors = ["you"]
license = ""
Binary file not shown.
@@ -0,0 +1,16 @@
use crate::audio_capture::AudioCaptureState;
pub async fn start_capture(
state: &AudioCaptureState,
max_duration_secs: u32,
) -> Result<(), String> {
todo!("implement Linux audio capture")
}
pub async fn stop_capture(state: &AudioCaptureState) -> Result<String, String> {
todo!("implement Linux audio capture stop")
}
pub fn is_supported() -> bool {
false
}
+4
View File
@@ -2,11 +2,15 @@
mod macos;
#[cfg(target_os = "windows")]
mod windows;
#[cfg(target_os = "linux")]
mod linux;
#[cfg(target_os = "macos")]
pub use macos::*;
#[cfg(target_os = "windows")]
pub use windows::*;
#[cfg(target_os = "linux")]
pub use linux::*;
use std::sync::{Arc, Mutex};
+2 -2
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "Voicebox",
"version": "0.1.11",
"version": "0.1.13",
"identifier": "sh.voicebox.app",
"build": {
"beforeDevCommand": "bun run dev",
@@ -12,7 +12,7 @@
"bundle": {
"active": true,
"targets": "all",
"createUpdaterArtifacts": true,
"createUpdaterArtifacts": false,
"externalBin": ["binaries/voicebox-server"],
"icon": [
"icons/32x32.png",
+18 -22
View File
@@ -2,29 +2,25 @@ import type { PlatformFilesystem, FileFilter } from '@/platform/types';
export const tauriFilesystem: PlatformFilesystem = {
async saveFile(filename: string, blob: Blob, filters?: FileFilter[]) {
try {
const { save } = await import('@tauri-apps/plugin-dialog');
const filePath = await save({
defaultPath: filename,
filters: filters || [],
});
const { save } = await import('@tauri-apps/plugin-dialog');
const { writeFile } = await import('@tauri-apps/plugin-fs');
if (filePath) {
const { writeBinaryFile } = await import('@tauri-apps/plugin-fs');
const arrayBuffer = await blob.arrayBuffer();
await writeBinaryFile(filePath, new Uint8Array(arrayBuffer));
}
} catch (error) {
console.error('Failed to use Tauri dialog, falling back to browser download:', error);
// Fall back to browser download if Tauri dialog fails
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
const filePath = await save({
defaultPath: filename,
filters: filters || [],
});
if (!filePath) return; // User cancelled the dialog
const resolvedPath = typeof filePath === 'string'
? filePath
: (filePath as { path: string }).path;
if (!resolvedPath) {
throw new Error('Failed to resolve save path from dialog');
}
const arrayBuffer = await blob.arrayBuffer();
await writeFile(resolvedPath, new Uint8Array(arrayBuffer));
},
};
+2 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@voicebox/web",
"private": true,
"version": "0.1.11",
"version": "0.1.13",
"type": "module",
"scripts": {
"dev": "vite",
@@ -21,6 +21,7 @@
"@types/react-dom": "^18.3.0",
"@typescript-eslint/eslint-plugin": "^7.0.0",
"@typescript-eslint/parser": "^7.0.0",
"@tailwindcss/vite": "^4.0.0",
"@vitejs/plugin-react": "^4.3.0",
"eslint": "^8.57.0",
"eslint-plugin-react-hooks": "^4.6.0",
+2 -1
View File
@@ -1,9 +1,10 @@
import path from 'node:path';
import react from '@vitejs/plugin-react';
import tailwindcss from '@tailwindcss/vite';
import { defineConfig } from 'vite';
export default defineConfig({
plugins: [react()],
plugins: [react(), tailwindcss()],
resolve: {
alias: {
'@': path.resolve(__dirname, '../app/src'),