mirror of
https://github.com/jamiepine/voicebox.git
synced 2026-09-27 14:15:16 -07:00
Compare commits
18
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
81864e831a | ||
|
|
7bd72ea9f7 | ||
|
|
f96eae2567 | ||
|
|
7d53699c96 | ||
|
|
28e91ce2c1 | ||
|
|
88be097b62 | ||
|
|
564d787927 | ||
|
|
2c1ee94891 | ||
|
|
e789c937ad | ||
|
|
273483ffcf | ||
|
|
5774a168a9 | ||
|
|
6bf40bd2d0 | ||
|
|
12cda2e090 | ||
|
|
7a90290a76 | ||
|
|
b02ce8e2f3 | ||
|
|
4e7772a21d | ||
|
|
51fb320b8c | ||
|
|
8ac202aa58 |
@@ -0,0 +1,120 @@
|
||||
---
|
||||
name: add-tts-engine
|
||||
description: Use this skill to add a new TTS engine to Voicebox. It walks through dependency research, backend implementation, frontend wiring, PyInstaller bundling, and frozen-build testing. Always start with Phase 0 (dependency audit) before writing any code.
|
||||
---
|
||||
|
||||
# Add TTS Engine
|
||||
|
||||
## Goal
|
||||
|
||||
Integrate a new text-to-speech engine into Voicebox end-to-end: dependency research, backend protocol implementation, frontend UI wiring, PyInstaller bundling, and frozen-build verification. The user should only need to test the final build locally.
|
||||
|
||||
## Reference Doc
|
||||
|
||||
The full phased guide lives at `docs/content/docs/developer/tts-engines.mdx`. **Read this file in its entirety before starting.** It contains:
|
||||
|
||||
- Phase 0: Dependency research (mandatory before writing code)
|
||||
- Phase 1: Backend implementation (`TTSBackend` protocol)
|
||||
- Phase 2: Route and service integration (usually zero changes)
|
||||
- Phase 3: Frontend integration (5 files)
|
||||
- Phase 4: Dependencies (`requirements.txt`, justfile, CI, Docker)
|
||||
- Phase 5: PyInstaller bundling (`build_binary.py` + `server.py`)
|
||||
- Phase 6: Common upstream workarounds
|
||||
- Implementation checklist (gate between phases)
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Read the guide
|
||||
|
||||
```bash
|
||||
# Read the full TTS engines doc
|
||||
cat docs/content/docs/developer/tts-engines.mdx
|
||||
```
|
||||
|
||||
Internalize all phases, especially Phase 0 and Phase 5. The v0.2.3 release was three patch releases because Phase 0 was skipped.
|
||||
|
||||
### 2. Dependency research (Phase 0)
|
||||
|
||||
Clone the model library into a temporary directory and audit it. Do NOT skip this.
|
||||
|
||||
```bash
|
||||
mkdir /tmp/engine-research && cd /tmp/engine-research
|
||||
git clone <model-library-url>
|
||||
```
|
||||
|
||||
Run the grep searches from Phase 0.2 in the guide against the cloned source and its transitive dependencies. Produce a written dependency audit covering:
|
||||
|
||||
1. PyPI vs non-PyPI packages
|
||||
2. PyInstaller directives needed (`--collect-all`, `--copy-metadata`, `--hidden-import`)
|
||||
3. Runtime data files that must be bundled
|
||||
4. Native library paths that need env var overrides in frozen builds
|
||||
5. Monkey-patches needed (`torch.load`, float64, MPS, HF token)
|
||||
6. Sample rate
|
||||
7. Model download method (`from_pretrained` vs `snapshot_download` + `from_local`)
|
||||
|
||||
Test model loading and generation on CPU in the throwaway venv before proceeding.
|
||||
|
||||
### 3. Implement (Phases 1–4)
|
||||
|
||||
Follow the guide's phases in order. Key files to modify:
|
||||
|
||||
**Backend (Phase 1):**
|
||||
- Create `backend/backends/<engine>_backend.py`
|
||||
- Register in `backend/backends/__init__.py` (ModelConfig + TTS_ENGINES + factory)
|
||||
- Update regex in `backend/models.py`
|
||||
|
||||
**Frontend (Phase 3):**
|
||||
- `app/src/lib/api/types.ts` — engine union type
|
||||
- `app/src/lib/constants/languages.ts` — ENGINE_LANGUAGES
|
||||
- `app/src/components/Generation/EngineModelSelector.tsx` — ENGINE_OPTIONS, ENGINE_DESCRIPTIONS
|
||||
- `app/src/lib/hooks/useGenerationForm.ts` — Zod schema, model-name mapping
|
||||
- `app/src/components/ServerSettings/ModelManagement.tsx` — MODEL_DESCRIPTIONS
|
||||
|
||||
**Dependencies (Phase 4):**
|
||||
- `backend/requirements.txt`
|
||||
- `justfile` (setup-python, setup-python-release targets)
|
||||
- `.github/workflows/release.yml`
|
||||
- `Dockerfile` (if applicable)
|
||||
|
||||
### 4. PyInstaller bundling (Phase 5)
|
||||
|
||||
Register the engine in `backend/build_binary.py`:
|
||||
- `--hidden-import` for the backend module and model package
|
||||
- `--collect-all` for packages using `inspect.getsource`, shipping data files, or native libraries
|
||||
- `--copy-metadata` for packages using `importlib.metadata`
|
||||
|
||||
If the engine has native data paths, add `os.environ.setdefault()` in `backend/server.py` inside the `if getattr(sys, 'frozen', False):` block.
|
||||
|
||||
### 5. Verify in dev mode
|
||||
|
||||
```bash
|
||||
just dev
|
||||
```
|
||||
|
||||
Test the full chain: model download → load → generate → voice cloning.
|
||||
|
||||
### 6. Use the checklist
|
||||
|
||||
Walk through the Implementation Checklist at the bottom of `tts-engines.mdx`. Every item must be checked before handing the build to the user.
|
||||
|
||||
## Key Lessons (from v0.2.3)
|
||||
|
||||
These are the most common failure modes. Phase 0 research catches all of them:
|
||||
|
||||
| Pattern | Symptom in Frozen Build | Fix |
|
||||
|---------|------------------------|-----|
|
||||
| `@typechecked` / `inspect.getsource()` | "could not get source code" | `--collect-all <package>` |
|
||||
| Package ships pretrained model files | `FileNotFoundError` for `.pth.tar`, `.yaml` | `--collect-all <package>` |
|
||||
| C library with hardcoded system paths | `FileNotFoundError` for `/usr/share/...` | `--collect-all` + env var in `server.py` |
|
||||
| `importlib.metadata.version()` | "No package metadata found" | `--copy-metadata <package>` |
|
||||
| `torch.load` without `map_location` | CUDA device not available on CPU build | Monkey-patch `torch.load` |
|
||||
| `torch.from_numpy` on float64 data | dtype mismatch RuntimeError | Cast to `.float()` |
|
||||
| `token=True` in HF download calls | Auth failure without stored HF token | Use `snapshot_download(token=None)` + `from_local()` |
|
||||
|
||||
## Notes
|
||||
|
||||
- The route and service layers have zero per-engine dispatch points. `main.py` requires zero changes.
|
||||
- The model config registry in `backends/__init__.py` handles all dispatch automatically.
|
||||
- Use `get_torch_device()` and `model_load_progress()` from `backends/base.py` — don't reimplement device detection or progress tracking.
|
||||
- Always test with a **clean HuggingFace cache** (no pre-downloaded models from dev).
|
||||
- Do NOT push or create a release. Hand the build to the user for local testing.
|
||||
+1
-1
@@ -1,5 +1,5 @@
|
||||
[bumpversion]
|
||||
current_version = 0.3.0
|
||||
current_version = 0.3.1
|
||||
commit = True
|
||||
tag = True
|
||||
tag_name = v{new_version}
|
||||
|
||||
@@ -62,6 +62,7 @@ jobs:
|
||||
pip install pyinstaller
|
||||
pip install -r backend/requirements.txt
|
||||
pip install --no-deps chatterbox-tts
|
||||
pip install --no-deps hume-tada
|
||||
|
||||
- name: Install MLX dependencies (Apple Silicon only)
|
||||
if: matrix.backend == 'mlx'
|
||||
@@ -188,6 +189,7 @@ jobs:
|
||||
pip install pyinstaller
|
||||
pip install -r backend/requirements.txt
|
||||
pip install --no-deps chatterbox-tts
|
||||
pip install --no-deps hume-tada
|
||||
|
||||
- name: Install PyTorch with CUDA 12.6
|
||||
run: |
|
||||
@@ -198,33 +200,37 @@ jobs:
|
||||
run: |
|
||||
python -c "import torch; print(f'CUDA available in build: {torch.cuda.is_available()}'); print(f'CUDA version: {torch.version.cuda}')"
|
||||
|
||||
- name: Build CUDA server binary
|
||||
- name: Build CUDA server binary (onedir)
|
||||
shell: bash
|
||||
working-directory: backend
|
||||
run: python build_binary.py --cuda
|
||||
|
||||
- name: Split binary for GitHub Releases
|
||||
- name: Package into server core + CUDA libs archives
|
||||
shell: bash
|
||||
run: |
|
||||
python scripts/split_binary.py \
|
||||
backend/dist/voicebox-server-cuda.exe \
|
||||
--output release-assets/
|
||||
python scripts/package_cuda.py \
|
||||
backend/dist/voicebox-server-cuda/ \
|
||||
--output release-assets/ \
|
||||
--cuda-libs-version cu126-v1 \
|
||||
--torch-compat ">=2.6.0,<2.11.0"
|
||||
|
||||
- name: Upload split parts to GitHub Release
|
||||
- name: Upload archives to GitHub Release
|
||||
if: startsWith(github.ref, 'refs/tags/')
|
||||
uses: softprops/action-gh-release@v1
|
||||
uses: softprops/action-gh-release@v2
|
||||
with:
|
||||
files: |
|
||||
release-assets/voicebox-server-cuda.part*.exe
|
||||
release-assets/voicebox-server-cuda.sha256
|
||||
release-assets/voicebox-server-cuda.manifest
|
||||
release-assets/voicebox-server-cuda.tar.gz
|
||||
release-assets/voicebox-server-cuda.tar.gz.sha256
|
||||
release-assets/cuda-libs-cu126-v1.tar.gz
|
||||
release-assets/cuda-libs-cu126-v1.tar.gz.sha256
|
||||
release-assets/cuda-libs.json
|
||||
draft: true
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Upload binary as workflow artifact
|
||||
- name: Upload onedir as workflow artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: voicebox-server-cuda-windows
|
||||
path: backend/dist/voicebox-server-cuda.exe
|
||||
path: backend/dist/voicebox-server-cuda/
|
||||
retention-days: 7
|
||||
|
||||
@@ -53,6 +53,12 @@ tauri/src-tauri/gen/Assets.car
|
||||
tauri/src-tauri/gen/voicebox.icns
|
||||
tauri/src-tauri/gen/partial.plist
|
||||
|
||||
# PyInstaller
|
||||
*.spec
|
||||
|
||||
# Windows artifacts
|
||||
nul
|
||||
|
||||
# Temporary
|
||||
tmp/
|
||||
temp/
|
||||
|
||||
@@ -35,6 +35,8 @@ RUN pip install --no-cache-dir --upgrade pip
|
||||
|
||||
COPY backend/requirements.txt .
|
||||
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt
|
||||
RUN pip install --no-cache-dir --prefix=/install --no-deps chatterbox-tts
|
||||
RUN pip install --no-cache-dir --prefix=/install --no-deps hume-tada
|
||||
RUN pip install --no-cache-dir --prefix=/install \
|
||||
git+https://github.com/QwenLM/Qwen3-TTS.git
|
||||
|
||||
|
||||
@@ -59,10 +59,10 @@
|
||||
|
||||
## What is Voicebox?
|
||||
|
||||
Voicebox is a **local-first voice cloning studio** — a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
|
||||
Voicebox is a **local-first voice cloning studio** — a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 5 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
|
||||
|
||||
- **Complete privacy** — models and voice data stay on your machine
|
||||
- **4 TTS engines** — Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo
|
||||
- **5 TTS engines** — Qwen3-TTS, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo, and HumeAI TADA
|
||||
- **23 languages** — from English to Arabic, Japanese, Hindi, Swahili, and more
|
||||
- **Post-processing effects** — pitch shift, reverb, delay, chorus, compression, and filters
|
||||
- **Expressive speech** — paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo
|
||||
@@ -93,7 +93,7 @@ Voicebox is a **local-first voice cloning studio** — a free and open-source al
|
||||
|
||||
### Multi-Engine Voice Cloning
|
||||
|
||||
Four TTS engines with different strengths, switchable per-generation:
|
||||
Five TTS engines with different strengths, switchable per-generation:
|
||||
|
||||
| Engine | Languages | Strengths |
|
||||
| --------------------------- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
@@ -101,6 +101,7 @@ Four TTS engines with different strengths, switchable per-generation:
|
||||
| **LuxTTS** | English | Lightweight (~1GB VRAM), 48kHz output, 150x realtime on CPU |
|
||||
| **Chatterbox Multilingual** | 23 | Broadest language coverage — Arabic, Danish, Finnish, Greek, Hebrew, Hindi, Malay, Norwegian, Polish, Swahili, Swedish, Turkish and more |
|
||||
| **Chatterbox Turbo** | English | Fast 350M model with paralinguistic emotion/sound tags |
|
||||
| **TADA** (1B / 3B) | 10 | HumeAI speech-language model — 700s+ coherent audio, text-acoustic dual alignment |
|
||||
|
||||
### Emotions & Paralinguistic Tags
|
||||
|
||||
@@ -230,7 +231,7 @@ Full API documentation available at `http://localhost:17493/docs`.
|
||||
| Frontend | React, TypeScript, Tailwind CSS |
|
||||
| State | Zustand, React Query |
|
||||
| Backend | FastAPI (Python) |
|
||||
| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo |
|
||||
| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo, TADA |
|
||||
| Effects | Pedalboard (Spotify) |
|
||||
| Transcription | Whisper / Whisper Turbo (PyTorch or MLX) |
|
||||
| Inference | MLX (Apple Silicon) / PyTorch (CUDA/ROCm/XPU/CPU) |
|
||||
@@ -245,7 +246,7 @@ Full API documentation available at `http://localhost:17493/docs`.
|
||||
| ----------------------- | ---------------------------------------------- |
|
||||
| **Real-time Streaming** | Stream audio as it generates, word by word |
|
||||
| **Voice Design** | Create new voices from text descriptions |
|
||||
| **More Models** | XTTS, Bark, and other open-source voice models |
|
||||
| **More Models** | XTTS, Bark, and other open-source voice models |
|
||||
| **Plugin Architecture** | Extend with custom models and effects |
|
||||
| **Mobile Companion** | Control Voicebox from your phone |
|
||||
|
||||
@@ -276,6 +277,12 @@ just build # Build CPU server binary + Tauri app
|
||||
just build-local # (Windows) Build CPU + CUDA server binaries + Tauri app
|
||||
```
|
||||
|
||||
### Adding New Voice Models
|
||||
|
||||
The multi-engine architecture makes adding new TTS engines straightforward. A [step-by-step guide](docs/content/docs/developer/tts-engines.mdx) covers the full process: dependency research, backend protocol implementation, frontend wiring, and PyInstaller bundling.
|
||||
|
||||
The guide is optimized for AI coding agents. An [agent skill](.agents/skills/add-tts-engine/SKILL.md) can pick up a model name and handle the entire integration autonomously — you just test the build locally.
|
||||
|
||||
### Project Structure
|
||||
|
||||
```
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@voicebox/app",
|
||||
"version": "0.3.0",
|
||||
"version": "0.3.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
|
||||
@@ -20,6 +20,8 @@ const ENGINE_OPTIONS = [
|
||||
{ value: 'luxtts', label: 'LuxTTS' },
|
||||
{ value: 'chatterbox', label: 'Chatterbox' },
|
||||
{ value: 'chatterbox_turbo', label: 'Chatterbox Turbo' },
|
||||
{ value: 'tada:1B', label: 'TADA 1B' },
|
||||
{ value: 'tada:3B', label: 'TADA 3B Multilingual' },
|
||||
] as const;
|
||||
|
||||
const ENGINE_DESCRIPTIONS: Record<string, string> = {
|
||||
@@ -27,6 +29,7 @@ const ENGINE_DESCRIPTIONS: Record<string, string> = {
|
||||
luxtts: 'Fast, English-focused',
|
||||
chatterbox: '23 languages, incl. Hebrew',
|
||||
chatterbox_turbo: 'English, [laugh] [cough] tags',
|
||||
tada: 'HumeAI, 700s+ coherent audio',
|
||||
};
|
||||
|
||||
/** Engines that only support English and should force language to 'en' on select. */
|
||||
@@ -34,6 +37,7 @@ const ENGLISH_ONLY_ENGINES = new Set(['luxtts', 'chatterbox_turbo']);
|
||||
|
||||
function getSelectValue(engine: string, modelSize?: string): string {
|
||||
if (engine === 'qwen') return `qwen:${modelSize || '1.7B'}`;
|
||||
if (engine === 'tada') return `tada:${modelSize || '1B'}`;
|
||||
return engine;
|
||||
}
|
||||
|
||||
@@ -48,6 +52,20 @@ function handleEngineChange(form: UseFormReturn<GenerationFormValues>, value: st
|
||||
if (!available.some((l) => l.value === currentLang)) {
|
||||
form.setValue('language', available[0]?.value ?? 'en');
|
||||
}
|
||||
} else if (value.startsWith('tada:')) {
|
||||
const [, modelSize] = value.split(':');
|
||||
form.setValue('engine', 'tada');
|
||||
form.setValue('modelSize', modelSize as '1B' | '3B');
|
||||
// TADA 1B is English-only; 3B is multilingual
|
||||
if (modelSize === '1B') {
|
||||
form.setValue('language', 'en');
|
||||
} else {
|
||||
const currentLang = form.getValues('language');
|
||||
const available = getLanguageOptionsForEngine('tada');
|
||||
if (!available.some((l) => l.value === currentLang)) {
|
||||
form.setValue('language', available[0]?.value ?? 'en');
|
||||
}
|
||||
}
|
||||
} else {
|
||||
form.setValue('engine', value as GenerationFormValues['engine']);
|
||||
form.setValue('modelSize', undefined as unknown as '1.7B' | '0.6B');
|
||||
|
||||
@@ -62,6 +62,10 @@ const MODEL_DESCRIPTIONS: Record<string, string> = {
|
||||
'Production-grade open source TTS by Resemble AI. Supports 23 languages with voice cloning and emotion exaggeration control.',
|
||||
'chatterbox-turbo':
|
||||
'Streamlined 350M parameter TTS by Resemble AI. High-quality English speech with less compute and VRAM than larger models.',
|
||||
'tada-1b':
|
||||
'HumeAI TADA 1B — English speech-language model built on Llama 3.2 1B. Generates 700s+ of coherent audio with synchronized text-acoustic alignment.',
|
||||
'tada-3b-ml':
|
||||
'HumeAI TADA 3B Multilingual — built on Llama 3.2 3B. Supports 10 languages with high-fidelity voice cloning via text-acoustic dual alignment.',
|
||||
'whisper-base':
|
||||
'Smallest Whisper model (74M parameters). Fast transcription with moderate accuracy.',
|
||||
'whisper-small':
|
||||
@@ -391,7 +395,8 @@ export function ModelManagement() {
|
||||
(m) =>
|
||||
m.model_name.startsWith('qwen-tts') ||
|
||||
m.model_name.startsWith('luxtts') ||
|
||||
m.model_name.startsWith('chatterbox'),
|
||||
m.model_name.startsWith('chatterbox') ||
|
||||
m.model_name.startsWith('tada'),
|
||||
) ?? [];
|
||||
const whisperModels = modelStatus?.models.filter((m) => m.model_name.startsWith('whisper')) ?? [];
|
||||
|
||||
|
||||
@@ -42,8 +42,8 @@ export interface GenerationRequest {
|
||||
text: string;
|
||||
language: LanguageCode;
|
||||
seed?: number;
|
||||
model_size?: '1.7B' | '0.6B';
|
||||
engine?: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo';
|
||||
model_size?: '1.7B' | '0.6B' | '1B' | '3B';
|
||||
engine?: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo' | 'tada';
|
||||
instruct?: string;
|
||||
max_chunk_chars?: number;
|
||||
crossfade_ms?: number;
|
||||
|
||||
@@ -66,6 +66,7 @@ export const ENGINE_LANGUAGES: Record<string, readonly LanguageCode[]> = {
|
||||
'zh',
|
||||
],
|
||||
chatterbox_turbo: ['en'],
|
||||
tada: ['en', 'ar', 'zh', 'de', 'es', 'fr', 'it', 'ja', 'pl', 'pt'],
|
||||
} as const;
|
||||
|
||||
/** Helper: get language options for a given engine. */
|
||||
|
||||
@@ -15,9 +15,9 @@ const generationSchema = z.object({
|
||||
text: z.string().min(1, '').max(50000),
|
||||
language: z.enum(LANGUAGE_CODES as [LanguageCode, ...LanguageCode[]]),
|
||||
seed: z.number().int().optional(),
|
||||
modelSize: z.enum(['1.7B', '0.6B']).optional(),
|
||||
modelSize: z.enum(['1.7B', '0.6B', '1B', '3B']).optional(),
|
||||
instruct: z.string().max(500).optional(),
|
||||
engine: z.enum(['qwen', 'luxtts', 'chatterbox', 'chatterbox_turbo']).optional(),
|
||||
engine: z.enum(['qwen', 'luxtts', 'chatterbox', 'chatterbox_turbo', 'tada']).optional(),
|
||||
});
|
||||
|
||||
export type GenerationFormValues = z.infer<typeof generationSchema>;
|
||||
@@ -79,7 +79,11 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
|
||||
? 'chatterbox-tts'
|
||||
: engine === 'chatterbox_turbo'
|
||||
? 'chatterbox-turbo'
|
||||
: `qwen-tts-${data.modelSize}`;
|
||||
: engine === 'tada'
|
||||
? data.modelSize === '3B'
|
||||
? 'tada-3b-ml'
|
||||
: 'tada-1b'
|
||||
: `qwen-tts-${data.modelSize}`;
|
||||
const displayName =
|
||||
engine === 'luxtts'
|
||||
? 'LuxTTS'
|
||||
@@ -87,9 +91,13 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
|
||||
? 'Chatterbox TTS'
|
||||
: engine === 'chatterbox_turbo'
|
||||
? 'Chatterbox Turbo'
|
||||
: data.modelSize === '1.7B'
|
||||
? 'Qwen TTS 1.7B'
|
||||
: 'Qwen TTS 0.6B';
|
||||
: engine === 'tada'
|
||||
? data.modelSize === '3B'
|
||||
? 'TADA 3B Multilingual'
|
||||
: 'TADA 1B'
|
||||
: data.modelSize === '1.7B'
|
||||
? 'Qwen TTS 1.7B'
|
||||
: 'Qwen TTS 0.6B';
|
||||
|
||||
// Check if model needs downloading
|
||||
try {
|
||||
@@ -104,7 +112,7 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
|
||||
console.error('Failed to check model status:', error);
|
||||
}
|
||||
|
||||
const isQwen = engine === 'qwen';
|
||||
const hasModelSizes = engine === 'qwen' || engine === 'tada';
|
||||
const effectsChain = options.getEffectsChain?.();
|
||||
// This now returns immediately with status="generating"
|
||||
const result = await generation.mutateAsync({
|
||||
@@ -112,9 +120,9 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
|
||||
text: data.text,
|
||||
language: data.language,
|
||||
seed: data.seed,
|
||||
model_size: isQwen ? data.modelSize : undefined,
|
||||
model_size: hasModelSizes ? data.modelSize : undefined,
|
||||
engine,
|
||||
instruct: isQwen ? data.instruct || undefined : undefined,
|
||||
instruct: engine === 'qwen' ? data.instruct || undefined : undefined,
|
||||
max_chunk_chars: maxChunkChars,
|
||||
crossfade_ms: crossfadeMs,
|
||||
normalize: normalizeAudio,
|
||||
|
||||
+1
-1
@@ -1,3 +1,3 @@
|
||||
# Backend package
|
||||
|
||||
__version__ = "0.3.0"
|
||||
__version__ = "0.3.1"
|
||||
|
||||
@@ -166,6 +166,7 @@ TTS_ENGINES = {
|
||||
"luxtts": "LuxTTS",
|
||||
"chatterbox": "Chatterbox TTS",
|
||||
"chatterbox_turbo": "Chatterbox Turbo",
|
||||
"tada": "TADA",
|
||||
}
|
||||
|
||||
|
||||
@@ -259,6 +260,24 @@ def _get_non_qwen_tts_configs() -> list[ModelConfig]:
|
||||
needs_trim=True,
|
||||
languages=["en"],
|
||||
),
|
||||
ModelConfig(
|
||||
model_name="tada-1b",
|
||||
display_name="TADA 1B (English)",
|
||||
engine="tada",
|
||||
hf_repo_id="HumeAI/tada-1b",
|
||||
model_size="1B",
|
||||
size_mb=4000,
|
||||
languages=["en"],
|
||||
),
|
||||
ModelConfig(
|
||||
model_name="tada-3b-ml",
|
||||
display_name="TADA 3B Multilingual",
|
||||
engine="tada",
|
||||
hf_repo_id="HumeAI/tada-3b-ml",
|
||||
model_size="3B",
|
||||
size_mb=8000,
|
||||
languages=["en", "ar", "zh", "de", "es", "fr", "it", "ja", "pl", "pt"],
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
@@ -339,10 +358,12 @@ def engine_has_model_sizes(engine: str) -> bool:
|
||||
|
||||
|
||||
async def load_engine_model(engine: str, model_size: str = "default") -> None:
|
||||
"""Load a model for the given engine, handling the Qwen model_size special case."""
|
||||
"""Load a model for the given engine, handling engines with multiple model sizes."""
|
||||
backend = get_tts_backend_for_engine(engine)
|
||||
if engine == "qwen":
|
||||
await backend.load_model_async(model_size)
|
||||
elif engine == "tada":
|
||||
await backend.load_model(model_size)
|
||||
else:
|
||||
await backend.load_model()
|
||||
|
||||
@@ -358,7 +379,7 @@ async def ensure_model_cached_or_raise(engine: str, model_size: str = "default")
|
||||
cfg = c
|
||||
break
|
||||
|
||||
if engine == "qwen":
|
||||
if engine in ("qwen", "tada"):
|
||||
if not backend._is_model_cached(model_size):
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -490,6 +511,10 @@ def get_tts_backend_for_engine(engine: str) -> TTSBackend:
|
||||
from .chatterbox_turbo_backend import ChatterboxTurboTTSBackend
|
||||
|
||||
backend = ChatterboxTurboTTSBackend()
|
||||
elif engine == "tada":
|
||||
from .hume_backend import HumeTadaBackend
|
||||
|
||||
backend = HumeTadaBackend()
|
||||
else:
|
||||
raise ValueError(f"Unknown TTS engine: {engine}. Supported: {list(TTS_ENGINES.keys())}")
|
||||
|
||||
|
||||
@@ -0,0 +1,347 @@
|
||||
"""
|
||||
HumeAI TADA TTS backend implementation.
|
||||
|
||||
Wraps HumeAI's TADA (Text-Acoustic Dual Alignment) model for
|
||||
high-quality voice cloning. Two model variants:
|
||||
- tada-1b: English-only, ~2B params (Llama 3.2 1B base)
|
||||
- tada-3b-ml: Multilingual, ~4B params (Llama 3.2 3B base)
|
||||
|
||||
Both use a shared encoder/codec (HumeAI/tada-codec). The encoder
|
||||
produces 1:1 aligned token embeddings from reference audio, and the
|
||||
causal LM generates speech via flow-matching diffusion.
|
||||
|
||||
24kHz output, bf16 inference on CUDA, fp32 on CPU.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import threading
|
||||
from typing import ClassVar, List, Optional, Tuple
|
||||
|
||||
import numpy as np
|
||||
|
||||
from . import TTSBackend
|
||||
from .base import (
|
||||
is_model_cached,
|
||||
get_torch_device,
|
||||
combine_voice_prompts as _combine_voice_prompts,
|
||||
model_load_progress,
|
||||
)
|
||||
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# HuggingFace repos
|
||||
TADA_CODEC_REPO = "HumeAI/tada-codec"
|
||||
TADA_1B_REPO = "HumeAI/tada-1b"
|
||||
TADA_3B_ML_REPO = "HumeAI/tada-3b-ml"
|
||||
|
||||
TADA_MODEL_REPOS = {
|
||||
"1B": TADA_1B_REPO,
|
||||
"3B": TADA_3B_ML_REPO,
|
||||
}
|
||||
|
||||
# Key weight files for cache detection
|
||||
_TADA_MODEL_WEIGHT_FILES = [
|
||||
"model.safetensors",
|
||||
]
|
||||
|
||||
_TADA_CODEC_WEIGHT_FILES = [
|
||||
"encoder/model.safetensors",
|
||||
]
|
||||
|
||||
|
||||
class HumeTadaBackend:
|
||||
"""HumeAI TADA TTS backend for high-quality voice cloning."""
|
||||
|
||||
_load_lock: ClassVar[threading.Lock] = threading.Lock()
|
||||
|
||||
def __init__(self):
|
||||
self.model = None
|
||||
self.encoder = None
|
||||
self.model_size = "1B" # default to 1B
|
||||
self._device = None
|
||||
self._model_load_lock = asyncio.Lock()
|
||||
|
||||
def _get_device(self) -> str:
|
||||
# Force CPU on macOS — MPS has issues with flow matching
|
||||
# and large vocab lm_head (>65536 output channels)
|
||||
return get_torch_device(force_cpu_on_mac=True)
|
||||
|
||||
def is_loaded(self) -> bool:
|
||||
return self.model is not None
|
||||
|
||||
def _get_model_path(self, model_size: str = "1B") -> str:
|
||||
return TADA_MODEL_REPOS.get(model_size, TADA_1B_REPO)
|
||||
|
||||
def _is_model_cached(self, model_size: str = "1B") -> bool:
|
||||
repo = TADA_MODEL_REPOS.get(model_size, TADA_1B_REPO)
|
||||
model_cached = is_model_cached(repo, required_files=_TADA_MODEL_WEIGHT_FILES)
|
||||
codec_cached = is_model_cached(TADA_CODEC_REPO, required_files=_TADA_CODEC_WEIGHT_FILES)
|
||||
return model_cached and codec_cached
|
||||
|
||||
async def load_model(self, model_size: str = "1B") -> None:
|
||||
"""Load the TADA model and encoder."""
|
||||
if self.model is not None and self.model_size == model_size:
|
||||
return
|
||||
async with self._model_load_lock:
|
||||
if self.model is not None and self.model_size == model_size:
|
||||
return
|
||||
# Unload existing model if switching sizes
|
||||
if self.model is not None:
|
||||
self.unload_model()
|
||||
self.model_size = model_size
|
||||
await asyncio.to_thread(self._load_model_sync, model_size)
|
||||
|
||||
def _load_model_sync(self, model_size: str = "1B"):
|
||||
"""Synchronous model loading with progress tracking."""
|
||||
model_name = f"tada-{model_size.lower()}"
|
||||
is_cached = self._is_model_cached(model_size)
|
||||
repo = TADA_MODEL_REPOS.get(model_size, TADA_1B_REPO)
|
||||
|
||||
with model_load_progress(model_name, is_cached):
|
||||
# Install DAC shim before importing tada — tada's encoder/decoder
|
||||
# import dac.nn.layers.Snake1d which requires the descript-audio-codec
|
||||
# package. The real package pulls in onnx/tensorboard/matplotlib via
|
||||
# descript-audiotools, so we use a lightweight shim instead.
|
||||
from ..utils.dac_shim import install_dac_shim
|
||||
install_dac_shim()
|
||||
|
||||
import torch
|
||||
from huggingface_hub import snapshot_download
|
||||
|
||||
device = self._get_device()
|
||||
self._device = device
|
||||
logger.info(f"Loading HumeAI TADA {model_size} on {device}...")
|
||||
|
||||
# Download codec (encoder + decoder) if not cached
|
||||
logger.info("Downloading TADA codec...")
|
||||
snapshot_download(
|
||||
repo_id=TADA_CODEC_REPO,
|
||||
token=None,
|
||||
allow_patterns=["*.safetensors", "*.json", "*.txt", "*.bin"],
|
||||
)
|
||||
|
||||
# Download model weights if not cached
|
||||
logger.info(f"Downloading TADA {model_size} model...")
|
||||
snapshot_download(
|
||||
repo_id=repo,
|
||||
token=None,
|
||||
allow_patterns=["*.safetensors", "*.json", "*.txt", "*.bin", "*.model"],
|
||||
)
|
||||
|
||||
# TADA hardcodes "meta-llama/Llama-3.2-1B" as the tokenizer
|
||||
# source in its Aligner and TadaForCausalLM.from_pretrained().
|
||||
# That repo is gated (requires Meta license acceptance).
|
||||
# Download the tokenizer from an ungated mirror and get its
|
||||
# local cache path so we can point TADA at it directly.
|
||||
logger.info("Downloading Llama tokenizer (ungated mirror)...")
|
||||
tokenizer_path = snapshot_download(
|
||||
repo_id="unsloth/Llama-3.2-1B",
|
||||
token=None,
|
||||
allow_patterns=["tokenizer*", "special_tokens*"],
|
||||
)
|
||||
|
||||
# Determine dtype — use bf16 on CUDA for ~50% memory savings
|
||||
if device == "cuda" and torch.cuda.is_bf16_supported():
|
||||
model_dtype = torch.bfloat16
|
||||
else:
|
||||
model_dtype = torch.float32
|
||||
|
||||
# Patch the Aligner config class to use the local tokenizer
|
||||
# path instead of the gated "meta-llama/Llama-3.2-1B" default.
|
||||
# This avoids monkey-patching AutoTokenizer.from_pretrained
|
||||
# which corrupts the classmethod descriptor for other engines.
|
||||
from tada.modules.aligner import AlignerConfig
|
||||
AlignerConfig.tokenizer_name = tokenizer_path
|
||||
|
||||
# Load encoder (only needed for voice prompt encoding)
|
||||
from tada.modules.encoder import Encoder
|
||||
logger.info("Loading TADA encoder...")
|
||||
self.encoder = Encoder.from_pretrained(
|
||||
TADA_CODEC_REPO, subfolder="encoder"
|
||||
).to(device)
|
||||
self.encoder.eval()
|
||||
|
||||
# Load the causal LM (includes decoder for wav generation).
|
||||
# TadaForCausalLM.from_pretrained() calls
|
||||
# getattr(config, "tokenizer_name", "meta-llama/Llama-3.2-1B")
|
||||
# which hits the gated repo. Pre-load the config from HF,
|
||||
# inject the local tokenizer path, then pass it in.
|
||||
from tada.modules.tada import TadaForCausalLM, TadaConfig
|
||||
logger.info(f"Loading TADA {model_size} model...")
|
||||
config = TadaConfig.from_pretrained(repo)
|
||||
config.tokenizer_name = tokenizer_path
|
||||
self.model = TadaForCausalLM.from_pretrained(
|
||||
repo, config=config, torch_dtype=model_dtype
|
||||
).to(device)
|
||||
self.model.eval()
|
||||
|
||||
logger.info(f"HumeAI TADA {model_size} loaded successfully on {device}")
|
||||
|
||||
def unload_model(self) -> None:
|
||||
"""Unload model and encoder to free memory."""
|
||||
if self.model is not None:
|
||||
del self.model
|
||||
self.model = None
|
||||
if self.encoder is not None:
|
||||
del self.encoder
|
||||
self.encoder = None
|
||||
|
||||
self._device = None
|
||||
|
||||
import torch
|
||||
if torch.cuda.is_available():
|
||||
torch.cuda.empty_cache()
|
||||
|
||||
logger.info("HumeAI TADA unloaded")
|
||||
|
||||
async def create_voice_prompt(
|
||||
self,
|
||||
audio_path: str,
|
||||
reference_text: str,
|
||||
use_cache: bool = True,
|
||||
) -> Tuple[dict, bool]:
|
||||
"""
|
||||
Create voice prompt from reference audio using TADA's encoder.
|
||||
|
||||
TADA's encoder performs forced alignment between audio and text tokens,
|
||||
producing an EncoderOutput with 1:1 token-audio alignment. If no
|
||||
reference_text is provided, the encoder uses built-in ASR (English only).
|
||||
|
||||
We serialize the EncoderOutput to a dict for caching.
|
||||
"""
|
||||
await self.load_model(self.model_size)
|
||||
|
||||
cache_key = (
|
||||
"tada_" + get_cache_key(audio_path, reference_text)
|
||||
) if use_cache else None
|
||||
|
||||
if cache_key:
|
||||
cached = get_cached_voice_prompt(cache_key)
|
||||
if cached is not None and isinstance(cached, dict):
|
||||
return cached, True
|
||||
|
||||
def _encode_sync():
|
||||
import torch
|
||||
import soundfile as sf
|
||||
|
||||
device = self._device
|
||||
|
||||
# Load audio with soundfile (torchaudio 2.10+ requires torchcodec)
|
||||
audio_np, sr = sf.read(str(audio_path), dtype="float32")
|
||||
audio = torch.from_numpy(audio_np).float()
|
||||
if audio.ndim == 1:
|
||||
audio = audio.unsqueeze(0) # (samples,) -> (1, samples)
|
||||
else:
|
||||
audio = audio.T # (samples, channels) -> (channels, samples)
|
||||
audio = audio.to(device)
|
||||
|
||||
# Encode with forced alignment
|
||||
text_arg = [reference_text] if reference_text else None
|
||||
prompt = self.encoder(
|
||||
audio, text=text_arg, sample_rate=sr
|
||||
)
|
||||
|
||||
# Serialize EncoderOutput to a dict of CPU tensors for caching
|
||||
prompt_dict = {}
|
||||
for field_name in prompt.__dataclass_fields__:
|
||||
val = getattr(prompt, field_name)
|
||||
if isinstance(val, torch.Tensor):
|
||||
prompt_dict[field_name] = val.detach().cpu()
|
||||
elif isinstance(val, list):
|
||||
prompt_dict[field_name] = val
|
||||
elif isinstance(val, (int, float)):
|
||||
prompt_dict[field_name] = val
|
||||
else:
|
||||
prompt_dict[field_name] = val
|
||||
return prompt_dict
|
||||
|
||||
encoded = await asyncio.to_thread(_encode_sync)
|
||||
|
||||
if cache_key:
|
||||
cache_voice_prompt(cache_key, encoded)
|
||||
|
||||
return encoded, False
|
||||
|
||||
async def combine_voice_prompts(
|
||||
self,
|
||||
audio_paths: List[str],
|
||||
reference_texts: List[str],
|
||||
) -> Tuple[np.ndarray, str]:
|
||||
return await _combine_voice_prompts(audio_paths, reference_texts, sample_rate=24000)
|
||||
|
||||
async def generate(
|
||||
self,
|
||||
text: str,
|
||||
voice_prompt: dict,
|
||||
language: str = "en",
|
||||
seed: Optional[int] = None,
|
||||
instruct: Optional[str] = None,
|
||||
) -> Tuple[np.ndarray, int]:
|
||||
"""
|
||||
Generate audio from text using HumeAI TADA.
|
||||
|
||||
Args:
|
||||
text: Text to synthesize
|
||||
voice_prompt: Serialized EncoderOutput dict from create_voice_prompt()
|
||||
language: Language code (en, ar, de, es, fr, it, ja, pl, pt, zh)
|
||||
seed: Random seed for reproducibility
|
||||
instruct: Not supported by TADA (ignored)
|
||||
|
||||
Returns:
|
||||
Tuple of (audio_array, sample_rate=24000)
|
||||
"""
|
||||
await self.load_model(self.model_size)
|
||||
|
||||
def _generate_sync():
|
||||
import torch
|
||||
from tada.modules.encoder import EncoderOutput
|
||||
|
||||
if seed is not None:
|
||||
torch.manual_seed(seed)
|
||||
if torch.cuda.is_available():
|
||||
torch.cuda.manual_seed(seed)
|
||||
|
||||
device = self._device
|
||||
|
||||
# Reconstruct EncoderOutput from the cached dict
|
||||
restored = {}
|
||||
for k, v in voice_prompt.items():
|
||||
if isinstance(v, torch.Tensor):
|
||||
# Move to device and match model dtype for float tensors
|
||||
if v.is_floating_point():
|
||||
model_dtype = next(self.model.parameters()).dtype
|
||||
restored[k] = v.to(device=device, dtype=model_dtype)
|
||||
else:
|
||||
restored[k] = v.to(device=device)
|
||||
else:
|
||||
restored[k] = v
|
||||
|
||||
prompt = EncoderOutput(**restored)
|
||||
|
||||
# For non-English with the 3B-ML model, we could reload the
|
||||
# encoder with the language-specific aligner. However, the
|
||||
# generation itself is language-agnostic — only the encoder's
|
||||
# aligner changes. Since we encode at create_voice_prompt time,
|
||||
# the language is already baked in. For simplicity, we don't
|
||||
# reload the encoder here.
|
||||
|
||||
logger.info(f"[TADA] Generating ({language}), text length: {len(text)}")
|
||||
|
||||
output = self.model.generate(
|
||||
prompt=prompt,
|
||||
text=text,
|
||||
)
|
||||
|
||||
# output.audio is a list of tensors (one per batch item)
|
||||
if output.audio and output.audio[0] is not None:
|
||||
audio_tensor = output.audio[0]
|
||||
audio = audio_tensor.detach().cpu().numpy().squeeze().astype(np.float32)
|
||||
else:
|
||||
logger.warning("[TADA] Generation produced no audio")
|
||||
audio = np.zeros(24000, dtype=np.float32)
|
||||
|
||||
return audio, 24000
|
||||
|
||||
return await asyncio.to_thread(_generate_sync)
|
||||
+43
-1
@@ -34,9 +34,15 @@ def build_server(cuda=False):
|
||||
binary_name = "voicebox-server-cuda" if cuda else "voicebox-server"
|
||||
|
||||
# PyInstaller arguments
|
||||
# CUDA builds use --onedir so we can split the output into two archives:
|
||||
# 1. Server core (~200-400MB) — versioned with the app
|
||||
# 2. CUDA libs (~2GB) — versioned independently (only redownloaded on
|
||||
# CUDA toolkit / torch major version changes)
|
||||
# CPU builds remain --onefile for simplicity.
|
||||
pack_mode = "--onedir" if cuda else "--onefile"
|
||||
args = [
|
||||
"server.py", # Use server.py as entry point instead of main.py
|
||||
"--onefile",
|
||||
pack_mode,
|
||||
"--name",
|
||||
binary_name,
|
||||
]
|
||||
@@ -186,6 +192,42 @@ def build_server(cuda=False):
|
||||
# needed by LuxTTS for text-to-phoneme conversion
|
||||
"--collect-all",
|
||||
"piper_phonemize",
|
||||
# HumeAI TADA — speech-language model using Llama + flow matching
|
||||
"--hidden-import",
|
||||
"backend.backends.hume_backend",
|
||||
"--hidden-import",
|
||||
"tada",
|
||||
"--hidden-import",
|
||||
"tada.modules",
|
||||
"--hidden-import",
|
||||
"tada.modules.tada",
|
||||
"--hidden-import",
|
||||
"tada.modules.encoder",
|
||||
"--hidden-import",
|
||||
"tada.modules.decoder",
|
||||
"--hidden-import",
|
||||
"tada.modules.aligner",
|
||||
"--hidden-import",
|
||||
"tada.modules.acoustic_spkr_verf",
|
||||
"--hidden-import",
|
||||
"tada.nn",
|
||||
"--hidden-import",
|
||||
"tada.nn.vibevoice",
|
||||
"--hidden-import",
|
||||
"tada.utils",
|
||||
"--hidden-import",
|
||||
"tada.utils.gray_code",
|
||||
"--hidden-import",
|
||||
"tada.utils.text",
|
||||
# DAC shim — provides dac.nn.layers.Snake1d without the real
|
||||
# descript-audio-codec package (which pulls onnx/tensorboard via
|
||||
# descript-audiotools). The shim is in backend/utils/dac_shim.py.
|
||||
"--hidden-import",
|
||||
"backend.utils.dac_shim",
|
||||
"--hidden-import",
|
||||
"torchaudio",
|
||||
"--collect-submodules",
|
||||
"tada",
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
+2
-2
@@ -66,9 +66,9 @@ class GenerationRequest(BaseModel):
|
||||
text: str = Field(..., min_length=1, max_length=50000)
|
||||
language: str = Field(default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it|he|ar|da|el|fi|hi|ms|nl|no|pl|sv|sw|tr)$")
|
||||
seed: Optional[int] = Field(None, ge=0)
|
||||
model_size: Optional[str] = Field(default="1.7B", pattern="^(1\\.7B|0\\.6B)$")
|
||||
model_size: Optional[str] = Field(default="1.7B", pattern="^(1\\.7B|0\\.6B|1B|3B)$")
|
||||
instruct: Optional[str] = Field(None, max_length=500)
|
||||
engine: Optional[str] = Field(default="qwen", pattern="^(qwen|luxtts|chatterbox|chatterbox_turbo)$")
|
||||
engine: Optional[str] = Field(default="qwen", pattern="^(qwen|luxtts|chatterbox|chatterbox_turbo|tada)$")
|
||||
max_chunk_chars: int = Field(
|
||||
default=800, ge=100, le=5000, description="Max characters per chunk for long text splitting"
|
||||
)
|
||||
|
||||
@@ -33,6 +33,13 @@ s3tokenizer
|
||||
spacy-pkuseg
|
||||
pyloudnorm
|
||||
|
||||
# HumeAI TADA sub-dependencies (hume-tada itself is installed
|
||||
# --no-deps in the setup script because it pins torch>=2.7,<2.8.
|
||||
# descript-audio-codec is NOT installed — it pulls onnx/tensorboard
|
||||
# via descript-audiotools. A lightweight shim in utils/dac_shim.py
|
||||
# provides the only class TADA uses: Snake1d.)
|
||||
torchaudio
|
||||
|
||||
# Audio processing
|
||||
librosa>=0.10.0
|
||||
soundfile>=0.12.0
|
||||
|
||||
+266
-119
@@ -1,16 +1,22 @@
|
||||
"""
|
||||
CUDA backend binary download, assembly, and verification.
|
||||
CUDA backend download, assembly, and verification.
|
||||
|
||||
Downloads split parts of the CUDA-enabled voicebox-server binary from
|
||||
GitHub Releases, reassembles them, verifies integrity via SHA-256,
|
||||
and places the binary in the app's data directory for use on next
|
||||
backend restart.
|
||||
Downloads two archives from GitHub Releases:
|
||||
1. Server core (voicebox-server-cuda.tar.gz) — the exe + non-NVIDIA deps,
|
||||
versioned with the app.
|
||||
2. CUDA libs (cuda-libs-{version}.tar.gz) — NVIDIA runtime libraries,
|
||||
versioned independently (only redownloaded on CUDA toolkit bump).
|
||||
|
||||
Both archives are extracted into {data_dir}/backends/cuda/ which forms the
|
||||
complete PyInstaller --onedir directory structure that torch expects.
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import tarfile
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
@@ -24,6 +30,10 @@ GITHUB_RELEASES_URL = "https://github.com/jamiepine/voicebox/releases/download"
|
||||
|
||||
PROGRESS_KEY = "cuda-backend"
|
||||
|
||||
# The current expected CUDA libs version. Bump this when we change the
|
||||
# CUDA toolkit version or torch's CUDA dependency changes (e.g. cu126 -> cu128).
|
||||
CUDA_LIBS_VERSION = "cu126-v1"
|
||||
|
||||
|
||||
def get_backends_dir() -> Path:
|
||||
"""Directory where downloaded backend binaries are stored."""
|
||||
@@ -32,21 +42,46 @@ def get_backends_dir() -> Path:
|
||||
return d
|
||||
|
||||
|
||||
def get_cuda_binary_name() -> str:
|
||||
"""Platform-specific CUDA binary filename."""
|
||||
def get_cuda_dir() -> Path:
|
||||
"""Directory where the CUDA backend (onedir) is extracted."""
|
||||
d = get_backends_dir() / "cuda"
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
return d
|
||||
|
||||
|
||||
def get_cuda_exe_name() -> str:
|
||||
"""Platform-specific CUDA executable filename."""
|
||||
if sys.platform == "win32":
|
||||
return "voicebox-server-cuda.exe"
|
||||
return "voicebox-server-cuda"
|
||||
|
||||
|
||||
def get_cuda_binary_path() -> Optional[Path]:
|
||||
"""Return path to CUDA binary if it exists."""
|
||||
p = get_backends_dir() / get_cuda_binary_name()
|
||||
"""Return path to the CUDA executable if it exists inside the onedir."""
|
||||
p = get_cuda_dir() / get_cuda_exe_name()
|
||||
if p.exists():
|
||||
return p
|
||||
return None
|
||||
|
||||
|
||||
def get_cuda_libs_manifest_path() -> Path:
|
||||
"""Path to the cuda-libs.json manifest inside the CUDA dir."""
|
||||
return get_cuda_dir() / "cuda-libs.json"
|
||||
|
||||
|
||||
def get_installed_cuda_libs_version() -> Optional[str]:
|
||||
"""Read the installed CUDA libs version from cuda-libs.json, or None."""
|
||||
manifest_path = get_cuda_libs_manifest_path()
|
||||
if not manifest_path.exists():
|
||||
return None
|
||||
try:
|
||||
data = json.loads(manifest_path.read_text())
|
||||
return data.get("version")
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not read cuda-libs.json: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def is_cuda_active() -> bool:
|
||||
"""Check if the current process is the CUDA binary.
|
||||
|
||||
@@ -60,25 +95,151 @@ def get_cuda_status() -> dict:
|
||||
progress_manager = get_progress_manager()
|
||||
cuda_path = get_cuda_binary_path()
|
||||
progress = progress_manager.get_progress(PROGRESS_KEY)
|
||||
cuda_libs_version = get_installed_cuda_libs_version()
|
||||
|
||||
return {
|
||||
"available": cuda_path is not None,
|
||||
"active": is_cuda_active(),
|
||||
"binary_path": str(cuda_path) if cuda_path else None,
|
||||
"cuda_libs_version": cuda_libs_version,
|
||||
"downloading": progress is not None and progress.get("status") == "downloading",
|
||||
"download_progress": progress,
|
||||
}
|
||||
|
||||
|
||||
async def download_cuda_binary(version: Optional[str] = None):
|
||||
"""Download the CUDA backend binary from GitHub Releases.
|
||||
def _needs_server_download(version: Optional[str] = None) -> bool:
|
||||
"""Check if the server core archive needs to be (re)downloaded."""
|
||||
cuda_path = get_cuda_binary_path()
|
||||
if not cuda_path:
|
||||
return True
|
||||
# Check if the binary version matches the expected app version
|
||||
installed = get_cuda_binary_version()
|
||||
expected = version or __version__
|
||||
if expected.startswith("v"):
|
||||
expected = expected[1:]
|
||||
return installed != expected
|
||||
|
||||
Downloads split parts listed in a manifest file, concatenates them,
|
||||
and verifies the SHA-256 checksum for integrity. Atomic write
|
||||
(temp file -> rename).
|
||||
|
||||
def _needs_cuda_libs_download() -> bool:
|
||||
"""Check if the CUDA libs archive needs to be (re)downloaded."""
|
||||
installed = get_installed_cuda_libs_version()
|
||||
if installed is None:
|
||||
return True
|
||||
return installed != CUDA_LIBS_VERSION
|
||||
|
||||
|
||||
async def _download_and_extract_archive(
|
||||
client,
|
||||
url: str,
|
||||
sha256_url: Optional[str],
|
||||
dest_dir: Path,
|
||||
label: str,
|
||||
progress_offset: int,
|
||||
total_size: int,
|
||||
):
|
||||
"""Download a .tar.gz archive and extract it into dest_dir.
|
||||
|
||||
Args:
|
||||
version: Version tag (e.g. "v0.2.0"). Defaults to current app version.
|
||||
client: httpx.AsyncClient
|
||||
url: URL of the .tar.gz archive
|
||||
sha256_url: URL of the .sha256 checksum file (optional)
|
||||
dest_dir: Directory to extract into
|
||||
label: Human-readable label for progress updates
|
||||
progress_offset: Byte offset for progress reporting (when downloading
|
||||
multiple archives sequentially)
|
||||
total_size: Total bytes across all downloads (for progress bar)
|
||||
"""
|
||||
progress = get_progress_manager()
|
||||
temp_path = dest_dir / f".download-{label.replace(' ', '-')}.tmp"
|
||||
|
||||
# Clean up leftover partial download
|
||||
if temp_path.exists():
|
||||
temp_path.unlink()
|
||||
|
||||
# Fetch expected checksum (fail-fast: never extract an unverified archive)
|
||||
expected_sha = None
|
||||
if sha256_url:
|
||||
try:
|
||||
sha_resp = await client.get(sha256_url)
|
||||
sha_resp.raise_for_status()
|
||||
expected_sha = sha_resp.text.strip().split()[0]
|
||||
logger.info(f"{label}: expected SHA-256: {expected_sha[:16]}...")
|
||||
except Exception as e:
|
||||
raise RuntimeError(f"{label}: failed to fetch checksum from {sha256_url}") from e
|
||||
|
||||
# Stream download, verify, and extract — always clean up temp file
|
||||
downloaded = 0
|
||||
try:
|
||||
async with client.stream("GET", url) as response:
|
||||
response.raise_for_status()
|
||||
with open(temp_path, "wb") as f:
|
||||
async for chunk in response.aiter_bytes(chunk_size=1024 * 1024):
|
||||
f.write(chunk)
|
||||
downloaded += len(chunk)
|
||||
progress.update_progress(
|
||||
PROGRESS_KEY,
|
||||
current=progress_offset + downloaded,
|
||||
total=total_size,
|
||||
filename=f"Downloading {label}",
|
||||
status="downloading",
|
||||
)
|
||||
|
||||
# Verify integrity
|
||||
if expected_sha:
|
||||
progress.update_progress(
|
||||
PROGRESS_KEY,
|
||||
current=progress_offset + downloaded,
|
||||
total=total_size,
|
||||
filename=f"Verifying {label}...",
|
||||
status="downloading",
|
||||
)
|
||||
sha256 = hashlib.sha256()
|
||||
with open(temp_path, "rb") as f:
|
||||
while True:
|
||||
data = f.read(1024 * 1024)
|
||||
if not data:
|
||||
break
|
||||
sha256.update(data)
|
||||
actual = sha256.hexdigest()
|
||||
if actual != expected_sha:
|
||||
raise ValueError(
|
||||
f"{label} integrity check failed: expected {expected_sha[:16]}..., got {actual[:16]}..."
|
||||
)
|
||||
logger.info(f"{label}: integrity verified")
|
||||
|
||||
# Extract (use data filter for path traversal protection on Python 3.12+)
|
||||
progress.update_progress(
|
||||
PROGRESS_KEY,
|
||||
current=progress_offset + downloaded,
|
||||
total=total_size,
|
||||
filename=f"Extracting {label}...",
|
||||
status="downloading",
|
||||
)
|
||||
with tarfile.open(temp_path, "r:gz") as tar:
|
||||
if sys.version_info >= (3, 12):
|
||||
tar.extractall(path=dest_dir, filter="data")
|
||||
else:
|
||||
tar.extractall(path=dest_dir)
|
||||
|
||||
logger.info(f"{label}: extracted to {dest_dir}")
|
||||
finally:
|
||||
if temp_path.exists():
|
||||
temp_path.unlink()
|
||||
return downloaded
|
||||
|
||||
|
||||
async def download_cuda_binary(version: Optional[str] = None):
|
||||
"""Download the CUDA backend (server core + CUDA libs if needed).
|
||||
|
||||
Downloads both archives from GitHub Releases, extracts them into
|
||||
{data_dir}/backends/cuda/, and writes the cuda-libs.json manifest.
|
||||
|
||||
Only downloads what's needed:
|
||||
- Server core: always redownloaded (versioned with app)
|
||||
- CUDA libs: only if missing or version mismatch
|
||||
|
||||
Args:
|
||||
version: Version tag (e.g. "v0.3.0"). Defaults to current app version.
|
||||
"""
|
||||
import httpx
|
||||
|
||||
@@ -86,114 +247,91 @@ async def download_cuda_binary(version: Optional[str] = None):
|
||||
version = f"v{__version__}"
|
||||
|
||||
progress = get_progress_manager()
|
||||
binary_name = get_cuda_binary_name()
|
||||
dest_dir = get_backends_dir()
|
||||
final_path = dest_dir / binary_name
|
||||
temp_path = dest_dir / f"{binary_name}.download"
|
||||
cuda_dir = get_cuda_dir()
|
||||
|
||||
# Clean up any leftover partial download
|
||||
if temp_path.exists():
|
||||
temp_path.unlink()
|
||||
need_server = _needs_server_download(version)
|
||||
need_libs = _needs_cuda_libs_download()
|
||||
|
||||
logger.info(f"Starting CUDA backend download for {version}")
|
||||
if not need_server and not need_libs:
|
||||
logger.info("CUDA backend is up to date, nothing to download")
|
||||
return
|
||||
|
||||
logger.info(
|
||||
f"Starting CUDA backend download for {version} "
|
||||
f"(server={'yes' if need_server else 'cached'}, "
|
||||
f"libs={'yes' if need_libs else 'cached'})"
|
||||
)
|
||||
progress.update_progress(
|
||||
PROGRESS_KEY, current=0, total=0,
|
||||
filename="Fetching manifest...", status="downloading",
|
||||
PROGRESS_KEY,
|
||||
current=0,
|
||||
total=0,
|
||||
filename="Preparing download...",
|
||||
status="downloading",
|
||||
)
|
||||
|
||||
base_url = f"{GITHUB_RELEASES_URL}/{version}"
|
||||
stem = Path(binary_name).stem # voicebox-server-cuda
|
||||
server_archive = "voicebox-server-cuda.tar.gz"
|
||||
libs_archive = f"cuda-libs-{CUDA_LIBS_VERSION}.tar.gz"
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(follow_redirects=True, timeout=30.0) as client:
|
||||
# Fetch the manifest (list of split part filenames)
|
||||
manifest_url = f"{base_url}/{stem}.manifest"
|
||||
manifest_resp = await client.get(manifest_url)
|
||||
manifest_resp.raise_for_status()
|
||||
parts = [p.strip() for p in manifest_resp.text.strip().splitlines() if p.strip()]
|
||||
|
||||
if not parts:
|
||||
raise ValueError("Empty manifest — no split parts found")
|
||||
|
||||
logger.info(f"Found {len(parts)} split parts to download")
|
||||
|
||||
# Fetch expected checksum (optional — for integrity verification)
|
||||
expected_sha = None
|
||||
try:
|
||||
sha_url = f"{base_url}/{stem}.sha256"
|
||||
sha_resp = await client.get(sha_url)
|
||||
if sha_resp.status_code == 200:
|
||||
# Format: "sha256hex filename\n"
|
||||
expected_sha = sha_resp.text.strip().split()[0]
|
||||
logger.info(f"Expected SHA-256: {expected_sha[:16]}...")
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not fetch checksum file — skipping verification: {e}")
|
||||
|
||||
# Get total size across all parts by issuing HEAD requests
|
||||
# Estimate total download size
|
||||
total_size = 0
|
||||
for part_name in parts:
|
||||
if need_server:
|
||||
try:
|
||||
head_resp = await client.head(f"{base_url}/{part_name}")
|
||||
content_length = int(head_resp.headers.get("content-length", 0))
|
||||
total_size += content_length
|
||||
head = await client.head(f"{base_url}/{server_archive}")
|
||||
total_size += int(head.headers.get("content-length", 0))
|
||||
except Exception:
|
||||
pass
|
||||
if need_libs:
|
||||
try:
|
||||
head = await client.head(f"{base_url}/{libs_archive}")
|
||||
total_size += int(head.headers.get("content-length", 0))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
logger.info(f"Total download size: {total_size / 1024 / 1024:.1f} MB")
|
||||
|
||||
# Download and concatenate parts
|
||||
total_downloaded = 0
|
||||
with open(temp_path, "wb") as f:
|
||||
for i, part_name in enumerate(parts):
|
||||
part_url = f"{base_url}/{part_name}"
|
||||
logger.info(f"Downloading part {i + 1}/{len(parts)}: {part_name}")
|
||||
offset = 0
|
||||
|
||||
async with client.stream("GET", part_url) as response:
|
||||
response.raise_for_status()
|
||||
async for chunk in response.aiter_bytes(chunk_size=1024 * 1024):
|
||||
f.write(chunk)
|
||||
total_downloaded += len(chunk)
|
||||
progress.update_progress(
|
||||
PROGRESS_KEY, current=total_downloaded, total=total_size,
|
||||
filename=f"Downloading CUDA backend ({i + 1}/{len(parts)})",
|
||||
status="downloading",
|
||||
)
|
||||
|
||||
# Verify integrity if checksum was available
|
||||
if expected_sha:
|
||||
progress.update_progress(
|
||||
PROGRESS_KEY, current=total_downloaded, total=total_downloaded,
|
||||
filename="Verifying integrity...", status="downloading",
|
||||
)
|
||||
sha256 = hashlib.sha256()
|
||||
with open(temp_path, "rb") as f:
|
||||
while True:
|
||||
chunk = f.read(1024 * 1024)
|
||||
if not chunk:
|
||||
break
|
||||
sha256.update(chunk)
|
||||
|
||||
actual = sha256.hexdigest()
|
||||
if actual != expected_sha:
|
||||
raise ValueError(
|
||||
f"Integrity check failed: expected {expected_sha[:16]}..., "
|
||||
f"got {actual[:16]}..."
|
||||
# Download server core
|
||||
if need_server:
|
||||
server_downloaded = await _download_and_extract_archive(
|
||||
client,
|
||||
url=f"{base_url}/{server_archive}",
|
||||
sha256_url=f"{base_url}/{server_archive}.sha256",
|
||||
dest_dir=cuda_dir,
|
||||
label="CUDA server",
|
||||
progress_offset=offset,
|
||||
total_size=total_size,
|
||||
)
|
||||
logger.info(f"Integrity verified: {actual[:16]}...")
|
||||
offset += server_downloaded
|
||||
|
||||
# Atomic move into place (replace handles existing target on all platforms)
|
||||
temp_path.replace(final_path)
|
||||
# Make executable on Unix
|
||||
exe_path = cuda_dir / get_cuda_exe_name()
|
||||
if sys.platform != "win32" and exe_path.exists():
|
||||
exe_path.chmod(0o755)
|
||||
|
||||
# Make executable on Unix
|
||||
if sys.platform != "win32":
|
||||
final_path.chmod(0o755)
|
||||
# Download CUDA libs
|
||||
if need_libs:
|
||||
await _download_and_extract_archive(
|
||||
client,
|
||||
url=f"{base_url}/{libs_archive}",
|
||||
sha256_url=f"{base_url}/{libs_archive}.sha256",
|
||||
dest_dir=cuda_dir,
|
||||
label="CUDA libraries",
|
||||
progress_offset=offset,
|
||||
total_size=total_size,
|
||||
)
|
||||
|
||||
logger.info(f"CUDA backend downloaded to {final_path}")
|
||||
# Write local cuda-libs.json manifest
|
||||
manifest = {"version": CUDA_LIBS_VERSION}
|
||||
get_cuda_libs_manifest_path().write_text(json.dumps(manifest, indent=2) + "\n")
|
||||
|
||||
logger.info(f"CUDA backend ready at {cuda_dir}")
|
||||
progress.mark_complete(PROGRESS_KEY)
|
||||
|
||||
except Exception as e:
|
||||
# Clean up on failure
|
||||
if temp_path.exists():
|
||||
temp_path.unlink()
|
||||
logger.error(f"CUDA backend download failed: {e}")
|
||||
progress.mark_error(PROGRESS_KEY, str(e))
|
||||
raise
|
||||
@@ -202,15 +340,19 @@ async def download_cuda_binary(version: Optional[str] = None):
|
||||
def get_cuda_binary_version() -> Optional[str]:
|
||||
"""Get the version of the installed CUDA binary, or None if not installed."""
|
||||
import subprocess
|
||||
|
||||
cuda_path = get_cuda_binary_path()
|
||||
if not cuda_path:
|
||||
return None
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[str(cuda_path), "--version"],
|
||||
capture_output=True, text=True, timeout=30,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
cwd=str(cuda_path.parent), # Run from the onedir directory
|
||||
)
|
||||
# Output format: "voicebox-server 0.2.0"
|
||||
# Output format: "voicebox-server 0.3.0"
|
||||
for line in result.stdout.strip().splitlines():
|
||||
if "voicebox-server" in line:
|
||||
return line.split()[-1]
|
||||
@@ -222,26 +364,29 @@ def get_cuda_binary_version() -> Optional[str]:
|
||||
async def check_and_update_cuda_binary():
|
||||
"""Check if the CUDA binary is outdated and auto-download if so.
|
||||
|
||||
Called on server startup. If a CUDA binary exists but its version
|
||||
doesn't match the current app version, triggers a background download
|
||||
of the updated CUDA binary. The download progress is visible to the
|
||||
frontend via the existing SSE progress endpoint.
|
||||
Called on server startup. Checks both server version and CUDA libs
|
||||
version. Downloads only what's needed.
|
||||
"""
|
||||
cuda_path = get_cuda_binary_path()
|
||||
if not cuda_path:
|
||||
return # No CUDA binary installed, nothing to update
|
||||
|
||||
cuda_version = get_cuda_binary_version()
|
||||
current_version = __version__
|
||||
need_server = _needs_server_download()
|
||||
need_libs = _needs_cuda_libs_download()
|
||||
|
||||
if cuda_version == current_version:
|
||||
logger.info(f"CUDA binary is up to date (v{current_version})")
|
||||
if not need_server and not need_libs:
|
||||
logger.info(f"CUDA binary is up to date (server=v{__version__}, libs={get_installed_cuda_libs_version()})")
|
||||
return
|
||||
|
||||
logger.info(
|
||||
f"CUDA binary version mismatch: binary=v{cuda_version}, app=v{current_version}. "
|
||||
f"Auto-downloading updated CUDA backend..."
|
||||
)
|
||||
reasons = []
|
||||
if need_server:
|
||||
cuda_version = get_cuda_binary_version()
|
||||
reasons.append(f"server v{cuda_version} != v{__version__}")
|
||||
if need_libs:
|
||||
installed_libs = get_installed_cuda_libs_version()
|
||||
reasons.append(f"libs {installed_libs} != {CUDA_LIBS_VERSION}")
|
||||
|
||||
logger.info(f"CUDA backend needs update ({', '.join(reasons)}). Auto-downloading...")
|
||||
|
||||
try:
|
||||
await download_cuda_binary()
|
||||
@@ -250,10 +395,12 @@ async def check_and_update_cuda_binary():
|
||||
|
||||
|
||||
async def delete_cuda_binary() -> bool:
|
||||
"""Delete the downloaded CUDA binary. Returns True if deleted."""
|
||||
path = get_cuda_binary_path()
|
||||
if path and path.exists():
|
||||
path.unlink()
|
||||
logger.info(f"Deleted CUDA binary: {path}")
|
||||
"""Delete the downloaded CUDA backend directory. Returns True if deleted."""
|
||||
import shutil
|
||||
|
||||
cuda_dir = get_cuda_dir()
|
||||
if cuda_dir.exists() and any(cuda_dir.iterdir()):
|
||||
shutil.rmtree(cuda_dir)
|
||||
logger.info(f"Deleted CUDA backend directory: {cuda_dir}")
|
||||
return True
|
||||
return False
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
"""
|
||||
Minimal shim for descript-audio-codec (DAC).
|
||||
|
||||
TADA only imports Snake1d from dac.nn.layers and dac.model.dac.
|
||||
The real DAC package pulls in descript-audiotools which depends on
|
||||
onnx, tensorboard, protobuf, matplotlib, pystoi, etc. — none of
|
||||
which are needed for TADA's runtime use of Snake1d.
|
||||
|
||||
This shim provides the exact Snake1d implementation (MIT-licensed,
|
||||
from https://github.com/descriptinc/descript-audio-codec) so we can
|
||||
avoid the entire audiotools dependency chain.
|
||||
|
||||
If the real DAC package is installed, this module is never used —
|
||||
Python's import system will find the site-packages version first.
|
||||
Install this shim only when descript-audio-codec is NOT installed.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import types
|
||||
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
|
||||
|
||||
# ── Snake activation (from dac/nn/layers.py) ────────────────────────
|
||||
|
||||
# NOTE: The original DAC code uses @torch.jit.script here for a 1.4x
|
||||
# speedup. We omit it because TorchScript calls inspect.getsource()
|
||||
# which fails inside a PyInstaller frozen binary (no .py source files).
|
||||
def snake(x: torch.Tensor, alpha: torch.Tensor) -> torch.Tensor:
|
||||
shape = x.shape
|
||||
x = x.reshape(shape[0], shape[1], -1)
|
||||
x = x + (alpha + 1e-9).reciprocal() * torch.sin(alpha * x).pow(2)
|
||||
x = x.reshape(shape)
|
||||
return x
|
||||
|
||||
|
||||
class Snake1d(nn.Module):
|
||||
def __init__(self, channels: int):
|
||||
super().__init__()
|
||||
self.alpha = nn.Parameter(torch.ones(1, channels, 1))
|
||||
|
||||
def forward(self, x: torch.Tensor) -> torch.Tensor:
|
||||
return snake(x, self.alpha)
|
||||
|
||||
|
||||
# ── Register as dac.nn.layers and dac.model.dac ─────────────────────
|
||||
|
||||
def install_dac_shim() -> None:
|
||||
"""Register fake dac package modules in sys.modules.
|
||||
|
||||
Only installs the shim if 'dac' is not already importable
|
||||
(i.e. the real descript-audio-codec is not installed).
|
||||
"""
|
||||
try:
|
||||
import dac # noqa: F401 — real package exists, do nothing
|
||||
return
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
# Create the module tree: dac -> dac.nn -> dac.nn.layers
|
||||
# -> dac.model -> dac.model.dac
|
||||
dac_pkg = types.ModuleType("dac")
|
||||
dac_pkg.__path__ = [] # make it a package
|
||||
dac_pkg.__package__ = "dac"
|
||||
|
||||
dac_nn = types.ModuleType("dac.nn")
|
||||
dac_nn.__path__ = []
|
||||
dac_nn.__package__ = "dac.nn"
|
||||
|
||||
dac_nn_layers = types.ModuleType("dac.nn.layers")
|
||||
dac_nn_layers.__package__ = "dac.nn"
|
||||
dac_nn_layers.Snake1d = Snake1d
|
||||
dac_nn_layers.snake = snake
|
||||
|
||||
dac_model = types.ModuleType("dac.model")
|
||||
dac_model.__path__ = []
|
||||
dac_model.__package__ = "dac.model"
|
||||
|
||||
dac_model_dac = types.ModuleType("dac.model.dac")
|
||||
dac_model_dac.__package__ = "dac.model"
|
||||
dac_model_dac.Snake1d = Snake1d
|
||||
|
||||
# Wire up submodules
|
||||
dac_pkg.nn = dac_nn
|
||||
dac_pkg.model = dac_model
|
||||
dac_nn.layers = dac_nn_layers
|
||||
dac_model.dac = dac_model_dac
|
||||
|
||||
# Register in sys.modules
|
||||
sys.modules["dac"] = dac_pkg
|
||||
sys.modules["dac.nn"] = dac_nn
|
||||
sys.modules["dac.nn.layers"] = dac_nn_layers
|
||||
sys.modules["dac.model"] = dac_model
|
||||
sys.modules["dac.model.dac"] = dac_model_dac
|
||||
@@ -159,12 +159,14 @@ Tauri looks for `voicebox-server-${PLATFORM}` in `src-tauri/binaries/` and bundl
|
||||
|
||||
The `build-cuda-windows` job runs separately:
|
||||
|
||||
1. Install PyTorch with CUDA 12.1
|
||||
2. Build with `build_binary.py --cuda`
|
||||
3. Split binary with `scripts/split_binary.py`
|
||||
4. Upload parts as release artifacts
|
||||
1. Install PyTorch with CUDA 12.6
|
||||
2. Build with `build_binary.py --cuda` (produces `--onedir` output)
|
||||
3. Package with `scripts/package_cuda.py` into two archives:
|
||||
- `voicebox-server-cuda.tar.gz` — server core (~945 MB)
|
||||
- `cuda-libs-cu126-v1.tar.gz` — NVIDIA runtime libraries (~1.7 GB, cached independently)
|
||||
4. Upload archives as release artifacts
|
||||
|
||||
This binary is downloaded on-demand by users who enable CUDA in settings.
|
||||
This binary is downloaded on-demand by users who enable CUDA in settings. The CUDA libs archive is only re-downloaded when the CUDA toolkit version changes, not on every app update.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
|
||||
@@ -3,8 +3,12 @@ title: "TTS Engines"
|
||||
description: "How to add new text-to-speech engines to Voicebox"
|
||||
---
|
||||
|
||||
> **For humans:** This doc is optimized for AI agents to implement new TTS engines autonomously. It's structured as a phased workflow with explicit gates and a checklist so an agent can do the full integration — dependency research, backend, frontend, bundling — and hand you a draft release or prod build to test locally. It's also a useful reference if you're doing it yourself.
|
||||
|
||||
Adding an engine touches ~10 files across 4 layers. The backend protocol work is straightforward — the real time sink is dependency hell, upstream library bugs, and PyInstaller bundling.
|
||||
|
||||
**Do not start writing code until you complete Phase 0.** The v0.2.3 release was three patch releases of PyInstaller fixes because dependency research was skipped. Every issue — `inspect.getsource()` failures, missing native data files, metadata lookups, dtype mismatches — was discoverable by reading the model library's source code before integration began.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
The backend is split into layers:
|
||||
@@ -18,6 +22,133 @@ The backend is split into layers:
|
||||
|
||||
New engines only need to touch `backends/` and `models.py` on the backend side — the route and service layers use a model config registry that handles dispatch automatically.
|
||||
|
||||
## Phase 0: Dependency Research
|
||||
|
||||
**This phase is mandatory.** Clone the model library and its key dependencies into a temporary directory and inspect them before writing any integration code. The goal is to produce a dependency audit that identifies every PyInstaller-incompatible pattern, every native data file, and every upstream bug you'll need to work around.
|
||||
|
||||
### 0.1 Clone and Inspect the Model Library
|
||||
|
||||
```bash
|
||||
# Create a throwaway workspace
|
||||
mkdir /tmp/engine-research && cd /tmp/engine-research
|
||||
|
||||
# Clone the model library
|
||||
git clone https://github.com/org/model-library.git
|
||||
cd model-library
|
||||
```
|
||||
|
||||
**Read these files first, in order:**
|
||||
|
||||
1. **`setup.py` / `setup.cfg` / `pyproject.toml`** — Check pinned dependency versions. If the library pins `torch==2.6.0` or `numpy<1.26`, you'll need `--no-deps` installation and manual sub-dependency listing (this is what happened with `chatterbox-tts`).
|
||||
|
||||
2. **`__init__.py` and the main model class** — Trace the import chain. Look for:
|
||||
- `from_pretrained()` — does it call `huggingface_hub` internally? Does it pass `token=True` (which crashes without a stored HF token)?
|
||||
- `from_local()` — does it exist? You may need manual `snapshot_download()` + `from_local()` to bypass download bugs.
|
||||
- Device handling — does it default to CUDA? Does it support MPS? Many libraries crash on MPS with unsupported operators.
|
||||
|
||||
3. **All `import` statements** — Recursively trace what the library imports. You're looking for:
|
||||
- `inspect.getsource()` anywhere in the chain (search all `.py` files)
|
||||
- `typeguard` / `@typechecked` decorators (these call `inspect.getsource()` at import time)
|
||||
- `importlib.metadata.version()` or `pkg_resources.get_distribution()` (need `--copy-metadata`)
|
||||
- `lazy_loader` (needs `--collect-all` to bundle `.pyi` stubs)
|
||||
|
||||
### 0.2 Scan for PyInstaller-Incompatible Patterns
|
||||
|
||||
Run these searches against the cloned library **and** its transitive dependencies:
|
||||
|
||||
```bash
|
||||
# inspect.getsource — will crash in frozen binary without --collect-all
|
||||
grep -r "inspect.getsource\|getsource(" .
|
||||
|
||||
# typeguard / @typechecked — calls inspect.getsource at import time
|
||||
grep -r "@typechecked\|from typeguard" .
|
||||
|
||||
# importlib.metadata — needs --copy-metadata
|
||||
grep -r "importlib.metadata\|pkg_resources.get_distribution\|pkg_resources.require" .
|
||||
|
||||
# Data files loaded at runtime — need --collect-all or --collect-data
|
||||
grep -r "Path(__file__).parent\|os.path.dirname(__file__)\|resources_path\|pkg_resources.resource_filename" .
|
||||
|
||||
# Native library paths — may need env var override in frozen builds
|
||||
grep -r "/usr/share\|/usr/lib\|/usr/local\|espeak\|phonemize" .
|
||||
|
||||
# torch.load without map_location — will crash on CPU-only builds
|
||||
grep -r "torch.load(" . | grep -v "map_location"
|
||||
|
||||
# HuggingFace token bugs
|
||||
grep -r 'token=True\|token=os.getenv' .
|
||||
|
||||
# Float64/Float32 assumptions — librosa returns float64, many models assume float32
|
||||
grep -r "torch.from_numpy\|\.double()\|float64" .
|
||||
|
||||
# @torch.jit.script — calls inspect.getsource(), crashes in frozen builds
|
||||
grep -r "@torch.jit.script\|torch.jit.script" .
|
||||
|
||||
# torchaudio.load — requires torchcodec in torchaudio 2.10+, use soundfile.read() instead
|
||||
grep -r "torchaudio.load\|torchaudio.save" .
|
||||
|
||||
# Gated HuggingFace repos — models that hardcode gated repos as tokenizer/config sources
|
||||
grep -r "from_pretrained\|tokenizer_name\|AutoTokenizer" . | grep -i "llama\|meta-llama\|gated"
|
||||
```
|
||||
|
||||
### 0.3 Install and Trace in a Throwaway Venv
|
||||
|
||||
```bash
|
||||
# Create isolated venv
|
||||
python -m venv /tmp/engine-venv
|
||||
source /tmp/engine-venv/bin/activate
|
||||
|
||||
# Install the package (try normally first)
|
||||
pip install model-package
|
||||
|
||||
# Check if it conflicts with our stack
|
||||
pip install model-package torch==2.10 transformers==4.57.3 numpy>=1.26
|
||||
# If this fails, you need --no-deps:
|
||||
pip install --no-deps model-package
|
||||
|
||||
# Get the full dependency tree
|
||||
pip show model-package # Check Requires: field
|
||||
pip show -f model-package # List all installed files (look for data files)
|
||||
|
||||
# Check for non-PyPI dependencies
|
||||
pip install model-package 2>&1 | grep -i "no matching distribution"
|
||||
```
|
||||
|
||||
### 0.4 Test Model Loading on CPU
|
||||
|
||||
Before writing any integration code, verify the model works on CPU in a plain Python script:
|
||||
|
||||
```python
|
||||
import torch
|
||||
# Force CPU to catch map_location bugs early
|
||||
model = ModelClass.from_pretrained("org/model", device="cpu")
|
||||
|
||||
# Test with a float32 audio array (not float64)
|
||||
import numpy as np
|
||||
audio = np.random.randn(16000).astype(np.float32)
|
||||
output = model.generate("Hello world", audio)
|
||||
print(f"Output shape: {output.shape}, dtype: {output.dtype}, sample rate: {model.sample_rate}")
|
||||
```
|
||||
|
||||
If this crashes, you've found a bug you'll need to monkey-patch. Common ones:
|
||||
- `RuntimeError: expected scalar type Float but found Double` → needs float32 cast
|
||||
- `RuntimeError: map_location` → needs `torch.load` patch
|
||||
- `RuntimeError: Unsupported operator aten::...` → needs MPS skip
|
||||
|
||||
### 0.5 Produce a Dependency Audit
|
||||
|
||||
Before proceeding to Phase 1, write down:
|
||||
|
||||
1. **PyPI vs non-PyPI deps** — which packages need `--find-links`, `git+https://`, or `--no-deps`?
|
||||
2. **PyInstaller directives needed** — which packages need `--collect-all`, `--copy-metadata`, `--hidden-import`?
|
||||
3. **Runtime data files** — which packages ship data files (YAML, pretrained weights, phoneme tables, shader libraries) that must be bundled?
|
||||
4. **Native library paths** — which packages look for data at system paths that won't exist in a frozen binary?
|
||||
5. **Monkey-patches needed** — `torch.load` map_location, float64→float32 casts, MPS skip, HF token bypass, etc.
|
||||
6. **Sample rate** — what does the engine output? (24kHz, 44.1kHz, 48kHz)
|
||||
7. **Model download method** — `from_pretrained()` with library-managed download, or manual `snapshot_download()` + `from_local()`?
|
||||
|
||||
This audit becomes your implementation plan for Phases 1, 4, and 5.
|
||||
|
||||
## Phase 1: Backend Implementation
|
||||
|
||||
### 1.1 Create the Backend File
|
||||
@@ -148,6 +279,8 @@ In `app/src/lib/hooks/useGenerationForm.ts`:
|
||||
- Add engine-to-model-name mapping
|
||||
- Update payload construction for engine-specific fields
|
||||
|
||||
**Watch out for model naming inconsistencies.** The HuggingFace repo name, the model size label, and the API model name don't always follow predictable patterns. For example, TADA's 3B model is named `tada-3b-ml` (not `tada-3b`), because it's a multilingual variant. Always check the actual repo names and build the frontend model name mapping from those, not from assumptions like `{engine}-{size}`.
|
||||
|
||||
### 3.5 Model Management
|
||||
|
||||
In `app/src/components/ServerSettings/ModelManagement.tsx`:
|
||||
@@ -155,54 +288,173 @@ In `app/src/components/ServerSettings/ModelManagement.tsx`:
|
||||
|
||||
## Phase 4: Dependencies
|
||||
|
||||
Use the dependency audit from Phase 0 to drive this phase. You should already know what packages are needed, which conflict, and which require special installation.
|
||||
|
||||
### 4.1 Python Dependencies
|
||||
|
||||
Add to `backend/requirements.txt`. Watch for:
|
||||
Add to `backend/requirements.txt`. There are three installation patterns, depending on what Phase 0 revealed:
|
||||
|
||||
**Pinned dependency conflicts** — If the model package pins old versions, install with `--no-deps`:
|
||||
**Normal PyPI packages:**
|
||||
```
|
||||
some-model-package>=1.0.0
|
||||
```
|
||||
|
||||
**Pinned dependency conflicts (`--no-deps`)** — If the model package pins old versions of torch/numpy/transformers, install with `--no-deps` and list sub-dependencies manually. This is the pattern used for `chatterbox-tts`:
|
||||
```bash
|
||||
# In justfile / CI setup:
|
||||
pip install --no-deps chatterbox-tts
|
||||
|
||||
# In requirements.txt — list each actual sub-dependency:
|
||||
conformer>=0.3.2
|
||||
diffusers>=0.31.0
|
||||
omegaconf>=2.3.0
|
||||
resemble-perth>=0.0.2
|
||||
s3tokenizer>=0.1.6
|
||||
```
|
||||
|
||||
Then list sub-dependencies manually in `requirements.txt`.
|
||||
To identify sub-deps: `pip show chatterbox-tts` → `Requires:` field, then cross-reference against existing `requirements.txt` to avoid duplicates.
|
||||
|
||||
**Non-PyPI packages:**
|
||||
```
|
||||
linacodec @ git+https://github.com/user/repo.git
|
||||
**Non-PyPI packages** — Some libraries only exist on GitHub or require custom indexes:
|
||||
```
|
||||
# Git-only packages (no PyPI release)
|
||||
linacodec @ git+https://github.com/ysharma3501/LinaCodec.git
|
||||
Zipvoice @ git+https://github.com/ysharma3501/LuxTTS.git
|
||||
|
||||
**Custom package indexes:**
|
||||
```
|
||||
# Custom package indexes (C extensions with platform-specific wheels)
|
||||
--find-links https://k2-fsa.github.io/icefall/piper_phonemize.html
|
||||
piper-phonemize>=1.2.0
|
||||
```
|
||||
|
||||
### 4.2 Identifying Hidden Sub-Dependencies
|
||||
### 4.2 Dependency Conflict Resolution
|
||||
|
||||
1. Install the package normally in a throwaway venv
|
||||
2. Run `pip show <package>` to get its `Requires:` list
|
||||
3. Cross-reference against existing requirements.txt
|
||||
4. Test that the engine loads and generates
|
||||
Check for conflicts with the existing stack before adding anything:
|
||||
|
||||
## Phase 5: PyInstaller Bundling
|
||||
```bash
|
||||
# Our current stack pins (approximate):
|
||||
# Python 3.12+, torch>=2.10, transformers>=4.57, numpy>=1.26
|
||||
|
||||
This is where most of the pain lives. Common issues:
|
||||
# Test compatibility
|
||||
pip install model-package torch==2.10 transformers==4.57.3 numpy>=1.26
|
||||
|
||||
| Issue | Symptom | Fix |
|
||||
|-------|---------|-----|
|
||||
| `inspect.getsource()` at import | "could not get source code" | `--collect-all <package>` |
|
||||
| Data files (yaml, .pth.tar) | FileNotFoundError at runtime | `--collect-all <package>` |
|
||||
| Native data paths (espeak-ng) | Library looks at `/usr/share/...` | Set env var in frozen builds |
|
||||
| `importlib.metadata` lookups | "No package metadata found" | `--copy-metadata <package>` |
|
||||
| Dynamic imports | ModuleNotFoundError | `--hidden-import <module>` |
|
||||
# If it fails, check what the package pins:
|
||||
pip show model-package | grep Requires
|
||||
# Look at setup.py/pyproject.toml for version constraints
|
||||
```
|
||||
|
||||
### Testing Frozen Builds
|
||||
**Known incompatible patterns in the wild:**
|
||||
- `torch==2.6.0` — many older packages pin this
|
||||
- `numpy<1.26` — conflicts with Python 3.12+
|
||||
- `transformers==4.46.3` — many packages pin old transformers
|
||||
- `onnxruntime` pinned versions — often conflict with torch
|
||||
|
||||
You can't skip this. Models that work in `python -m uvicorn` will break in the PyInstaller binary.
|
||||
### 4.3 Update Installation Scripts
|
||||
|
||||
Dependencies must be added in multiple places:
|
||||
|
||||
| File | What to add |
|
||||
|------|------------|
|
||||
| `backend/requirements.txt` | Package and version constraint |
|
||||
| `justfile` | `--no-deps` install line if needed (in `setup-python` and `setup-python-release` targets) |
|
||||
| `.github/workflows/release.yml` | Same `--no-deps` line in CI build steps |
|
||||
| `Dockerfile` | Same install commands for Docker builds |
|
||||
|
||||
## Phase 5: PyInstaller Bundling (`build_binary.py`)
|
||||
|
||||
This is where most of the pain lives. **The v0.2.3 release was entirely dedicated to fixing bundling issues** — every new engine that shipped in v0.2.1 (LuxTTS, Chatterbox, Chatterbox Turbo) worked in dev but failed in production builds. Don't skip this phase.
|
||||
|
||||
### 5.1 Register Your Engine in `build_binary.py`
|
||||
|
||||
Every new engine needs entries in `backend/build_binary.py`. This file drives PyInstaller and is the single most common source of "works in dev, breaks in prod" bugs. You need to decide which PyInstaller directives your engine's dependencies require:
|
||||
|
||||
| Directive | What It Does | When You Need It |
|
||||
|-----------|-------------|-----------------|
|
||||
| `--hidden-import <module>` | Includes a module PyInstaller can't detect via static analysis | Dynamic imports, lazy imports, plugin architectures |
|
||||
| `--collect-all <package>` | Bundles source `.py` files, data files, AND native libraries | Packages that call `inspect.getsource()` at import time (e.g. `inflect` via `typeguard`'s `@typechecked`), or that ship pretrained model files (e.g. `perth` ships `.pth.tar` + `hparams.yaml`) |
|
||||
| `--collect-data <package>` | Bundles only data files (not source or native libs) | Packages with YAML configs, vocab files, etc. |
|
||||
| `--collect-submodules <package>` | Bundles all submodules | Packages with deep module trees that PyInstaller misses |
|
||||
| `--copy-metadata <package>` | Copies `importlib.metadata` info | Packages that call `importlib.metadata.version()` or `pkg_resources.get_distribution()` at runtime. Already required for: `requests`, `transformers`, `huggingface-hub`, `tokenizers`, `safetensors`, `tqdm` |
|
||||
|
||||
**Example: adding hidden imports and collect-all for a new engine:**
|
||||
|
||||
```python
|
||||
# In build_binary.py, inside the args list:
|
||||
"--hidden-import",
|
||||
"backend.backends.your_engine_backend",
|
||||
"--hidden-import",
|
||||
"your_engine_package",
|
||||
"--hidden-import",
|
||||
"your_engine_package.inference",
|
||||
"--collect-all",
|
||||
"some_dependency_that_uses_inspect_getsource",
|
||||
"--copy-metadata",
|
||||
"some_dependency_that_checks_its_own_version",
|
||||
```
|
||||
|
||||
### 5.2 Lessons from v0.2.3 — Real Failures and Their Fixes
|
||||
|
||||
These are actual production failures from shipping new engines. Every one of these passed `python -m uvicorn` in dev:
|
||||
|
||||
| Engine | Failure | Root Cause | Fix |
|
||||
|--------|---------|-----------|-----|
|
||||
| LuxTTS | `"could not get source code"` on import | `inflect` uses `typeguard`'s `@typechecked` which calls `inspect.getsource()` — needs `.py` source files, not just bytecode | `--collect-all inflect` |
|
||||
| LuxTTS | `espeak-ng-data` not found | `piper_phonemize` C library looks for data at `/usr/share/espeak-ng-data/` which doesn't exist in the bundle | `--collect-all piper_phonemize` + set `ESPEAK_DATA_PATH` env var at runtime (see 5.3) |
|
||||
| LuxTTS | `inspect.getsource` error in Vocos codec | `linacodec` and `zipvoice` use source introspection | `--collect-all linacodec` + `--collect-all zipvoice` |
|
||||
| Chatterbox | `FileNotFoundError` for watermark model | `perth` ships pretrained model files (`hparams.yaml`, `.pth.tar`) that PyInstaller doesn't bundle by default | `--collect-all perth` |
|
||||
| All engines | `importlib.metadata` failures | Frozen binary doesn't include package metadata for `huggingface-hub`, `transformers`, etc. | `--copy-metadata` for each affected package |
|
||||
| All engines | Download progress bars stuck at 0% | `huggingface_hub` silently disables tqdm progress bars based on logger level in frozen builds — our progress tracker never receives byte updates | Force-enable tqdm's internal counter in `HFProgressTracker` |
|
||||
| TADA | `inspect.getsource` error in DAC's `Snake1d` | `@torch.jit.script` calls `inspect.getsource()` which fails without `.py` source files | Wrote a lightweight shim (`dac_shim.py`) reimplementing `Snake1d` without `@torch.jit.script`, registered fake `dac.*` modules in `sys.modules` |
|
||||
| All engines | `NameError: name 'obj' is not defined` on macOS | Python 3.12.0 has a [CPython bug](https://github.com/pyinstaller/pyinstaller/issues/7992) that corrupts bytecode when PyInstaller rewrites code objects | Upgrade to Python 3.12.13+ |
|
||||
| All engines | `resource_tracker` subprocess crash | `multiprocessing` in frozen binaries needs `freeze_support()` called before anything else | Added to `server.py` entry point |
|
||||
|
||||
### 5.3 Runtime Frozen-Build Handling (`server.py`)
|
||||
|
||||
Some fixes can't live in `build_binary.py` — they need runtime detection. The entry point `backend/server.py` handles these before any heavy imports:
|
||||
|
||||
```python
|
||||
# 1. freeze_support() — MUST be called before any multiprocessing use
|
||||
import multiprocessing
|
||||
multiprocessing.freeze_support()
|
||||
|
||||
# 2. Native data paths — redirect C libraries to bundled data
|
||||
if getattr(sys, 'frozen', False):
|
||||
_meipass = getattr(sys, '_MEIPASS', os.path.dirname(sys.executable))
|
||||
_espeak_data = os.path.join(_meipass, 'piper_phonemize', 'espeak-ng-data')
|
||||
if os.path.isdir(_espeak_data):
|
||||
os.environ.setdefault('ESPEAK_DATA_PATH', _espeak_data)
|
||||
|
||||
# 3. stdout/stderr safety — PyInstaller --noconsole on Windows sets these to None
|
||||
if not _is_writable(sys.stdout):
|
||||
sys.stdout = open(os.devnull, 'w')
|
||||
```
|
||||
|
||||
If your engine's dependencies include native libraries that look for data at system paths (like espeak-ng does), you'll need to add a similar `os.environ.setdefault()` block here.
|
||||
|
||||
### 5.4 CUDA vs CPU Build Branching
|
||||
|
||||
`build_binary.py` produces two different binaries:
|
||||
|
||||
- **`voicebox-server`** (CPU) — excludes all `nvidia.*` packages to avoid bundling ~3 GB of CUDA DLLs
|
||||
- **`voicebox-server-cuda`** — includes `torch.cuda` and `torch.backends.cudnn`
|
||||
|
||||
On Windows, if the build environment has CUDA torch installed but you're building the CPU binary, the script temporarily swaps to CPU-only torch and restores CUDA torch afterward. This prevents PyInstaller from accidentally bundling CUDA libraries into the CPU build.
|
||||
|
||||
New engine imports go in the **common section** (not the CUDA or MLX conditional blocks) unless your engine has platform-specific dependencies.
|
||||
|
||||
### 5.5 MLX Conditional Inclusion
|
||||
|
||||
Apple Silicon builds conditionally include MLX hidden imports and `--collect-all mlx` / `--collect-all mlx_audio`. If your engine has an MLX-specific backend variant, add its imports inside the `if is_apple_silicon() and not cuda:` block.
|
||||
|
||||
### 5.6 Testing Frozen Builds
|
||||
|
||||
You can't skip this. Models that work in `python -m uvicorn` will break in the PyInstaller binary. The v0.2.3 release required **three patch releases** (v0.2.1 → v0.2.2 → v0.2.3) to get all engines working in production.
|
||||
|
||||
1. Build: `just build`
|
||||
2. Run and try download + load + generate
|
||||
3. Check stderr for the actual error
|
||||
4. Fix, rebuild, repeat
|
||||
2. Launch the binary directly (not via `python -m`)
|
||||
3. Test the **full chain**: download → load → generate → progress tracking
|
||||
4. Check stderr for the actual error (logs go to stderr for Tauri sidecar capture)
|
||||
5. Fix, rebuild, repeat
|
||||
|
||||
**Common gotcha:** testing only generation with a pre-cached model from your dev install. Always test with a clean model cache to verify downloads work too.
|
||||
|
||||
## Phase 6: Common Upstream Workarounds
|
||||
|
||||
@@ -240,6 +492,90 @@ def _get_device(self):
|
||||
return "cpu" # Skip MPS
|
||||
```
|
||||
|
||||
### Gated HuggingFace repos as hardcoded config sources
|
||||
|
||||
Some models hardcode a gated HuggingFace repo as their tokenizer or config source (e.g., TADA hardcodes `"meta-llama/Llama-3.2-1B"` in both its `AlignerConfig` and `TadaConfig`). This silently fails without HF authentication.
|
||||
|
||||
**Fix:** Download from an ungated mirror and patch the config objects directly:
|
||||
|
||||
```python
|
||||
# Download tokenizer from ungated mirror
|
||||
UNGATED_TOKENIZER = "unsloth/Llama-3.2-1B"
|
||||
tokenizer_path = snapshot_download(UNGATED_TOKENIZER, token=None)
|
||||
|
||||
# Patch the model config to use the local path instead of the gated repo
|
||||
config = ModelConfig.from_pretrained(model_path)
|
||||
config.tokenizer_name = tokenizer_path
|
||||
model = ModelClass.from_pretrained(model_path, config=config)
|
||||
```
|
||||
|
||||
**Do NOT monkey-patch `AutoTokenizer.from_pretrained`** — it's a classmethod, and replacing it corrupts the descriptor, which breaks other engines that use different tokenizers (e.g., Qwen uses a Qwen tokenizer via `AutoTokenizer`). Always patch at the config level, not the class method level.
|
||||
|
||||
### `torchaudio.load()` requires `torchcodec` in 2.10+
|
||||
|
||||
As of `torchaudio>=2.10`, `torchaudio.load()` requires the `torchcodec` package for audio I/O. If your engine or backend code uses `torchaudio.load()`, replace it with `soundfile`:
|
||||
|
||||
```python
|
||||
# Before (breaks without torchcodec):
|
||||
import torchaudio
|
||||
waveform, sr = torchaudio.load("audio.wav")
|
||||
|
||||
# After:
|
||||
import soundfile as sf
|
||||
import torch
|
||||
data, sr = sf.read("audio.wav", dtype="float32")
|
||||
waveform = torch.from_numpy(data).unsqueeze(0)
|
||||
```
|
||||
|
||||
Note: `torchaudio.functional.resample()` and other pure-PyTorch math functions work fine without `torchcodec` — only the I/O functions are affected.
|
||||
|
||||
### `@torch.jit.script` breaks in frozen builds
|
||||
|
||||
`torch.jit.script` calls `inspect.getsource()` to parse the decorated function's source code. In a PyInstaller binary, `.py` source files aren't available, so this crashes at import time.
|
||||
|
||||
**Fix:** Remove or avoid `@torch.jit.script` decorators. If the decorated function comes from an upstream dependency, write a shim that reimplements the function without the decorator (see "Toxic dependency chains" below).
|
||||
|
||||
### Toxic dependency chains — the shim pattern
|
||||
|
||||
Sometimes a model library depends on a package with a massive, hostile transitive dependency tree, but only uses a tiny piece of it. When the dependency chain is unbuildable or would pull in dozens of unwanted packages, the right move is to write a lightweight shim.
|
||||
|
||||
**Example:** TADA depends on `descript-audio-codec` (DAC), which pulls in `descript-audiotools` -> `onnx`, `tensorboard`, `protobuf`, `matplotlib`, `pystoi`, etc. The `onnx` package fails to build from source on macOS. But TADA only uses `Snake1d` from DAC — a 7-line PyTorch module.
|
||||
|
||||
**Solution:** Create a shim at `backend/utils/dac_shim.py` that registers fake modules in `sys.modules`:
|
||||
|
||||
```python
|
||||
import sys
|
||||
import types
|
||||
import torch
|
||||
from torch import nn
|
||||
|
||||
def snake(x, alpha):
|
||||
"""Snake activation — reimplemented without @torch.jit.script."""
|
||||
return x + (1.0 / (alpha + 1e-9)) * torch.sin(alpha * x).pow(2)
|
||||
|
||||
class Snake1d(nn.Module):
|
||||
def __init__(self, channels):
|
||||
super().__init__()
|
||||
self.alpha = nn.Parameter(torch.ones(1, channels, 1))
|
||||
def forward(self, x):
|
||||
return snake(x, self.alpha)
|
||||
|
||||
# Register fake dac.* modules so "from dac.nn.layers import Snake1d" works
|
||||
_nn = types.ModuleType("dac.nn")
|
||||
_layers = types.ModuleType("dac.nn.layers")
|
||||
_layers.Snake1d = Snake1d
|
||||
_nn.layers = _layers
|
||||
|
||||
for name, mod in [("dac", types.ModuleType("dac")),
|
||||
("dac.nn", _nn), ("dac.nn.layers", _layers)]:
|
||||
sys.modules[name] = mod
|
||||
```
|
||||
|
||||
**Key rules for shims:**
|
||||
- Import the shim **before** importing the model library (so it finds the fake modules first)
|
||||
- Do NOT use `@torch.jit.script` in the shim (see above)
|
||||
- Only reimplement what the model actually uses — check the import chain carefully
|
||||
|
||||
## Upcoming Engines
|
||||
|
||||
Based on the current model landscape, these are candidates for future integration:
|
||||
@@ -250,8 +586,83 @@ Based on the current model landscape, these are candidates for future integratio
|
||||
| **Fish Speech** | 50+ | Medium | Word-level control via inline text | Ready |
|
||||
| **Kokoro-82M** | English | 82M | CPU realtime, Apache 2.0 | Ready |
|
||||
| **XTTS-v2** | 17+ | Medium | Zero-shot cloning | Ready |
|
||||
| **HumeAI TADA** | EN (1B), Multi (3B) | Medium | 700s+ coherent audio, synced transcripts | Needs vetting |
|
||||
| **MOSS-TTS** | Multilingual | Medium | Text-to-voice design, multi-speaker dialogue | Needs vetting |
|
||||
| **Pocket TTS** | English | ~100M | CPU-first, >1× realtime | Needs vetting |
|
||||
|
||||
The multi-engine architecture is now in place, making new model integration straightforward (~1 day for a well-documented model with a PyPI package).
|
||||
## Implementation Checklist
|
||||
|
||||
Use this as a gate between phases. Do not proceed to the next phase until every item in the current phase is checked.
|
||||
|
||||
### Phase 0: Dependency Research
|
||||
- [ ] Cloned model library source into a temp directory
|
||||
- [ ] Read `setup.py` / `pyproject.toml` — noted pinned dependency versions
|
||||
- [ ] Traced all imports from the model class through to leaf dependencies
|
||||
- [ ] Searched for `inspect.getsource`, `@typechecked`, `typeguard` in the full dependency tree
|
||||
- [ ] Searched for `importlib.metadata`, `pkg_resources.get_distribution` in the dependency tree
|
||||
- [ ] Searched for `Path(__file__).parent`, `os.path.dirname(__file__)`, hardcoded system paths
|
||||
- [ ] Searched for `torch.load` calls missing `map_location`
|
||||
- [ ] Searched for `torch.from_numpy` without `.float()` cast
|
||||
- [ ] Searched for `token=True` or `token=os.getenv("HF_TOKEN")` in HuggingFace calls
|
||||
- [ ] Searched for `@torch.jit.script` / `torch.jit.script` (crashes in frozen builds)
|
||||
- [ ] Searched for `torchaudio.load` / `torchaudio.save` (requires `torchcodec` in 2.10+)
|
||||
- [ ] Searched for hardcoded gated HuggingFace repo names (e.g., `meta-llama/*`)
|
||||
- [ ] Evaluated whether any dependency is used minimally enough to shim instead of install
|
||||
- [ ] Tested model loading and generation on CPU in a throwaway venv
|
||||
- [ ] Tested with a clean HuggingFace cache (no pre-downloaded models)
|
||||
- [ ] Produced a written dependency audit documenting all findings
|
||||
|
||||
### Phase 1: Backend Implementation
|
||||
- [ ] Created `backend/backends/<engine>_backend.py` implementing `TTSBackend` protocol
|
||||
- [ ] Chose voice prompt pattern (pre-computed tensors vs deferred file paths)
|
||||
- [ ] Implemented all monkey-patches identified in Phase 0
|
||||
- [ ] Used `get_torch_device()` from `backends/base.py` for device selection
|
||||
- [ ] Used `model_load_progress()` from `backends/base.py` for download/load tracking
|
||||
- [ ] Tested: model downloads correctly
|
||||
- [ ] Tested: model loads on CPU
|
||||
- [ ] Tested: generation produces valid audio
|
||||
- [ ] Tested: voice cloning from reference audio works
|
||||
- [ ] Registered `ModelConfig` in `backends/__init__.py`
|
||||
- [ ] Added to `TTS_ENGINES` dict
|
||||
- [ ] Added factory branch in `get_tts_backend_for_engine()`
|
||||
- [ ] Updated engine regex in `backend/models.py`
|
||||
|
||||
### Phase 2–3: Route, Service, and Frontend
|
||||
- [ ] Confirmed zero changes needed in routes/services (or documented why custom behavior is needed)
|
||||
- [ ] Added engine to TypeScript union type in `app/src/lib/api/types.ts`
|
||||
- [ ] Added language map entry in `app/src/lib/constants/languages.ts`
|
||||
- [ ] Added to `ENGINE_OPTIONS` and `ENGINE_DESCRIPTIONS` in `EngineModelSelector.tsx`
|
||||
- [ ] Added to Zod schema and model-name mapping in `useGenerationForm.ts`
|
||||
- [ ] Added description in `ModelManagement.tsx`
|
||||
|
||||
### Phase 4: Dependencies
|
||||
- [ ] Added packages to `backend/requirements.txt`
|
||||
- [ ] If `--no-deps` needed: listed sub-dependencies explicitly
|
||||
- [ ] If git-only packages: added `@ git+https://...` entries
|
||||
- [ ] If custom index needed: added `--find-links` line
|
||||
- [ ] Updated `justfile` setup targets
|
||||
- [ ] Updated `.github/workflows/release.yml` build steps
|
||||
- [ ] Updated `Dockerfile` if applicable
|
||||
- [ ] Verified `pip install` succeeds in a clean venv with existing requirements
|
||||
|
||||
### Phase 5: PyInstaller Bundling
|
||||
- [ ] Added `--hidden-import` entries in `build_binary.py` for:
|
||||
- [ ] `backend.backends.<engine>_backend`
|
||||
- [ ] The model package and its key submodules
|
||||
- [ ] Added `--collect-all` for any packages that:
|
||||
- [ ] Use `inspect.getsource()` / `@typechecked`
|
||||
- [ ] Ship pretrained model data files (`.pth.tar`, `.yaml`, etc.)
|
||||
- [ ] Ship native data files (phoneme tables, shader libraries, etc.)
|
||||
- [ ] Added `--copy-metadata` for any packages that use `importlib.metadata`
|
||||
- [ ] If engine has native data paths: added `os.environ.setdefault()` in `server.py`
|
||||
- [ ] Built frozen binary with `just build`
|
||||
- [ ] Tested in frozen binary with **clean model cache** (not pre-cached from dev):
|
||||
- [ ] Model download works with real-time progress
|
||||
- [ ] Model loading works
|
||||
- [ ] Generation produces valid audio
|
||||
- [ ] No errors in stderr logs
|
||||
|
||||
### Phase 6: Final Verification
|
||||
- [ ] Engine works in dev mode (`just dev`)
|
||||
- [ ] Engine works in frozen binary (`just build` → run binary directly)
|
||||
- [ ] Tested on target platform (macOS for MLX, Windows/Linux for CUDA)
|
||||
- [ ] No regressions in existing engines
|
||||
|
||||
@@ -3,12 +3,12 @@ title: "Voicebox Documentation"
|
||||
description: "Voicebox is a local-first voice cloning studio -- a free and open-source alternative to ElevenLabs."
|
||||
---
|
||||
|
||||
Voicebox is a **local-first voice cloning studio** -- a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
|
||||
Voicebox is a **local-first voice cloning studio** -- a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 5 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
|
||||
|
||||

|
||||
|
||||
- **Complete privacy** -- models and voice data stay on your machine
|
||||
- **4 TTS engines** -- Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo
|
||||
- **5 TTS engines** -- Qwen3-TTS, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo, and HumeAI TADA
|
||||
- **23 languages** -- from English to Arabic, Japanese, Hindi, Swahili, and more
|
||||
- **Post-processing effects** -- pitch shift, reverb, delay, chorus, compression, and filters
|
||||
- **Expressive speech** -- paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo
|
||||
|
||||
@@ -5,10 +5,10 @@ description: "Voicebox is a local-first voice cloning studio -- a free and open-
|
||||
|
||||
## What is Voicebox?
|
||||
|
||||
Voicebox is a **local-first voice cloning studio** -- a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
|
||||
Voicebox is a **local-first voice cloning studio** -- a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 5 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
|
||||
|
||||
- **Complete privacy** -- models and voice data stay on your machine
|
||||
- **4 TTS engines** -- Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo
|
||||
- **5 TTS engines** -- Qwen3-TTS, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo, and HumeAI TADA
|
||||
- **23 languages** -- from English to Arabic, Japanese, Hindi, Swahili, and more
|
||||
- **Post-processing effects** -- pitch shift, reverb, delay, chorus, compression, and filters
|
||||
- **Expressive speech** -- paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo
|
||||
@@ -20,7 +20,7 @@ Voicebox is a **local-first voice cloning studio** -- a free and open-source alt
|
||||
|
||||
## TTS Engines
|
||||
|
||||
Four engines with different strengths, switchable per-generation:
|
||||
Five engines with different strengths, switchable per-generation:
|
||||
|
||||
| Engine | Languages | Strengths |
|
||||
|--------|-----------|-----------|
|
||||
@@ -28,6 +28,7 @@ Four engines with different strengths, switchable per-generation:
|
||||
| **LuxTTS** | English | Lightweight (~1GB VRAM), 48kHz output, 150x realtime on CPU |
|
||||
| **Chatterbox Multilingual** | 23 | Broadest language coverage |
|
||||
| **Chatterbox Turbo** | English | Fast 350M model with paralinguistic emotion/sound tags |
|
||||
| **TADA** (1B / 3B) | 10 | HumeAI speech-language model -- 700s+ coherent audio |
|
||||
|
||||
## GPU Support
|
||||
|
||||
@@ -56,7 +57,7 @@ Four engines with different strengths, switchable per-generation:
|
||||
| Frontend | React, TypeScript, Tailwind CSS |
|
||||
| State | Zustand, React Query |
|
||||
| Backend | FastAPI (Python) |
|
||||
| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo |
|
||||
| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo, TADA |
|
||||
| Effects | Pedalboard (Spotify) |
|
||||
| Transcription | Whisper / Whisper Turbo (PyTorch or MLX) |
|
||||
| Inference | MLX (Apple Silicon) / PyTorch (CUDA/ROCm/XPU/CPU) |
|
||||
|
||||
@@ -36,6 +36,10 @@
|
||||
│ │ │ │ Qwen3-TTS│ │LuxTTS │ │Chatterbox │ │ │ │
|
||||
│ │ │ │(Py/MLX) │ │ │ │(MTL+Turbo)│ │ │ │
|
||||
│ │ │ └──────────┘ └───────┘ └───────────┘ │ │ │
|
||||
│ │ │ ┌──────────┐ │ │ │
|
||||
│ │ │ │ TADA │ │ │ │
|
||||
│ │ │ │(1B / 3B) │ │ │ │
|
||||
│ │ │ └──────────┘ │ │ │
|
||||
│ │ └─────────────────────────────────────────┘ │ │
|
||||
│ │ ┌───────────┐ ┌─────────┐ │ │
|
||||
│ │ │ STTBackend│ │ Profiles│ │ │
|
||||
@@ -59,6 +63,7 @@
|
||||
| LuxTTS | `backend/backends/luxtts_backend.py` | LuxTTS — fast, CPU-friendly |
|
||||
| Chatterbox MTL | `backend/backends/chatterbox_backend.py` | Chatterbox Multilingual — 23 languages |
|
||||
| Chatterbox Turbo | `backend/backends/chatterbox_turbo_backend.py` | Chatterbox Turbo — English, paralinguistic tags |
|
||||
| TADA | `backend/backends/hume_backend.py` | HumeAI TADA — 1B English + 3B Multilingual |
|
||||
| Platform detect | `backend/platform_detect.py` | Apple Silicon → MLX, else → PyTorch |
|
||||
| API types | `backend/models.py` | Pydantic request/response models |
|
||||
| HF progress | `backend/utils/hf_progress.py` | HFProgressTracker (tqdm patching for download progress) |
|
||||
@@ -78,7 +83,7 @@
|
||||
```
|
||||
POST /generate
|
||||
1. Look up voice profile from DB
|
||||
2. Resolve engine from request (qwen | luxtts | chatterbox | chatterbox_turbo)
|
||||
2. Resolve engine from request (qwen | luxtts | chatterbox | chatterbox_turbo | tada)
|
||||
3. Get backend: get_tts_backend_for_engine(engine) # thread-safe singleton per engine
|
||||
4. Check model cache → if missing, trigger background download, return HTTP 202
|
||||
5. Load model (lazy): tts_backend.load_model(model_size)
|
||||
@@ -104,7 +109,8 @@ POST /generate
|
||||
- LuxTTS integration — fast, CPU-friendly English TTS (PR #254)
|
||||
- Chatterbox Multilingual TTS — 23 languages including Hebrew (PR #257)
|
||||
- Instruct parameter UI exists but is non-functional across all backends (see #224, Known Limitations)
|
||||
- Single flat model dropdown (Qwen 1.7B, Qwen 0.6B, LuxTTS, Chatterbox, Chatterbox Turbo)
|
||||
- HumeAI TADA integration — 1B English + 3B Multilingual speech-language model (PR #296)
|
||||
- Single flat model dropdown (Qwen 1.7B, Qwen 0.6B, LuxTTS, Chatterbox, Chatterbox Turbo, TADA 1B, TADA 3B)
|
||||
- Centralized model config registry (`ModelConfig` dataclass) — no per-engine dispatch maps in `main.py`
|
||||
- Shared `EngineModelSelector` component — engine/model dropdown defined once, used in both generation forms
|
||||
|
||||
@@ -136,6 +142,8 @@ POST /generate
|
||||
| LuxTTS | `luxtts` | English | ~300 MB | CPU-friendly, 48 kHz, fast | None |
|
||||
| Chatterbox | `chatterbox-tts` | 23 (incl. Hebrew, Arabic, Hindi, etc.) | ~3.2 GB | Zero-shot cloning, multilingual | Partial — `exaggeration` float (0-1) for expressiveness |
|
||||
| Chatterbox Turbo | `chatterbox-turbo` | English | ~1.5 GB | Paralinguistic tags ([laugh], [cough]), 350M params, low latency | Partial — inline tags only, no separate instruct param |
|
||||
| TADA 1B | `tada-1b` | English | ~4 GB | HumeAI speech-language model, 700s+ coherent audio | None |
|
||||
| TADA 3B Multilingual | `tada-3b-ml` | 10 (en, ar, zh, de, es, fr, it, ja, pl, pt) | ~8 GB | Multilingual, text-acoustic dual alignment | None |
|
||||
|
||||
### Multi-Engine Architecture (Shipped)
|
||||
|
||||
@@ -143,7 +151,7 @@ The singleton TTS backend blocker described in the previous version of this doc
|
||||
|
||||
- **Thread-safe backend registry** (`_tts_backends` dict + `_tts_backends_lock`) with double-checked locking
|
||||
- **Per-engine backend instances** — each engine gets its own singleton, loaded lazily
|
||||
- **Engine field on GenerationRequest** — frontend sends `engine: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo'`
|
||||
- **Engine field on GenerationRequest** — frontend sends `engine: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo' | 'tada'`
|
||||
- **Per-engine language filtering** — `ENGINE_LANGUAGES` map in frontend, backend regex accepts all languages
|
||||
- **Per-engine voice prompts** — `create_voice_prompt_for_profile()` dispatches to the correct backend
|
||||
- **Trim post-processing** — `trim_tts_output()` for Chatterbox engines (cuts trailing silence/hallucination)
|
||||
@@ -337,7 +345,7 @@ Notable requests:
|
||||
| **CosyVoice2-0.5B** | 3-10s zero-shot | Very fast | 24 kHz | Multilingual | Low | **Yes** — `inference_instruct2()`, works with cloning | Ready | Best instruct candidate |
|
||||
| **Fish Speech** | 10-30s few-shot | Real-time | 24-44 kHz | 50+ | Medium | **Yes** — inline text descriptions, word-level control | Ready | Multi-engine arch in place |
|
||||
| **MOSS-TTS Family** | Zero-shot | — | — | Multilingual | Medium | **Yes** — text prompts for style + timbre design | Needs vetting | Apache 2.0, multi-speaker dialogue |
|
||||
| **HumeAI TADA 1B/3B** | Zero-shot | 5× faster than LLM-TTS | — | EN (1B), Multilingual (3B) | Medium | Partial — automatic prosody from text context | Needs vetting | MIT, 700s+ coherent, synced transcript output |
|
||||
| **HumeAI TADA 1B/3B** | Zero-shot | 5× faster than LLM-TTS | 24 kHz | EN (1B), Multilingual (3B) | Medium | Partial — automatic prosody from text context | **Shipped** | PR #296, MIT, 700s+ coherent |
|
||||
| **VoxCPM 1.5** | Zero-shot (seconds) | ~0.15 RTF streaming | — | Bilingual (EN/ZH) | Medium | Partial — automatic context-aware prosody | Needs vetting | Apache 2.0, tokenizer-free continuous diffusion |
|
||||
| **Kokoro-82M** | 3s instant | CPU realtime | 24 kHz | English | Tiny (82M) | Partial — automatic style inference | Ready | Apache 2.0, multi-engine arch in place |
|
||||
| **XTTS-v2** | 6s zero-shot | Mid-GPU | 24 kHz | 17+ | Medium | Partial — style transfer from ref audio only | Ready | Multi-engine arch in place |
|
||||
@@ -475,7 +483,7 @@ The generation form now uses a flat model dropdown with engine-based routing. Pe
|
||||
| `/history/{id}/export` | GET | Export generation ZIP |
|
||||
| `/history/{id}/export-audio` | GET | Export audio only |
|
||||
| `/transcribe` | POST | Transcribe audio (Whisper) |
|
||||
| `/models/status` | GET | All model statuses (Qwen, LuxTTS, Chatterbox, Chatterbox Turbo, Whisper) |
|
||||
| `/models/status` | GET | All model statuses (Qwen, LuxTTS, Chatterbox, Chatterbox Turbo, TADA, Whisper) |
|
||||
| `/models/download` | POST | Trigger model download |
|
||||
| `/models/download/cancel` | POST | Cancel/dismiss download |
|
||||
| `/models/{name}` | DELETE | Delete downloaded model |
|
||||
|
||||
@@ -0,0 +1,173 @@
|
||||
# CUDA Libs as a Bolt-On Addon
|
||||
|
||||
## Problem
|
||||
|
||||
Every time we bump `__version__` (even for a UI tweak or bugfix), the exact-match version check in both `main.rs:222` and `cuda.py:237` invalidates the user's ~2.4GB CUDA binary, forcing a full redownload. The CUDA binary is the entire server rebuilt with NVIDIA libs included -- there's no separation between app logic and the CUDA runtime.
|
||||
|
||||
## Why This Is Hard With `--onefile`
|
||||
|
||||
The core tension is PyInstaller `--onefile` mode (`build_binary.py:39`). In onefile mode, everything -- Python code, all dependencies, torch, the NVIDIA `.dll`/`.so` files -- gets packed into a single self-extracting archive. There's no concept of "swap out one part." The binary IS the server.
|
||||
|
||||
## Options
|
||||
|
||||
### Option A: Switch to `--onedir` for the CUDA Build (Recommended)
|
||||
|
||||
Instead of `--onefile`, build the CUDA variant as a directory (a folder with the exe + all the shared libs alongside it). Then split the distribution into two archives:
|
||||
|
||||
1. **`voicebox-server-cuda` executable + non-NVIDIA deps** (~200-400MB) -- versioned with the app, redownloaded on every app update.
|
||||
2. **`cuda-libs-cu126.tar.gz`** (~2GB) -- the `nvidia.*` packages (cublas, cudnn, cuda_runtime, etc.), versioned independently (e.g., `cuda-libs-cu126-v1`). Only redownloaded when we bump the CUDA toolkit version or torch's CUDA dependency changes.
|
||||
|
||||
#### How it would work at runtime
|
||||
|
||||
- Tauri downloads the server binary archive and extracts it to `{data_dir}/backends/cuda/`
|
||||
- On first CUDA setup (or when cuda-libs version bumps), downloads and extracts the libs archive into the same directory
|
||||
- The CUDA server exe finds the `.dll`/`.so` files next to it (standard PyInstaller onedir behavior)
|
||||
- Version check becomes two checks: server version + cuda-libs version
|
||||
|
||||
#### Independent versioning
|
||||
|
||||
Add a `cuda-libs.json` manifest:
|
||||
|
||||
```json
|
||||
{"version": "cu126-v1", "torch_compat": ">=2.6.0,<2.8.0"}
|
||||
```
|
||||
|
||||
The server checks this on startup. The Tauri side checks it before launching. Only bump `cu126-v1` -> `cu126-v2` when we actually change the CUDA toolkit or torch major version.
|
||||
|
||||
#### Build pipeline changes
|
||||
|
||||
The CI `build-cuda-windows` job would build with `--onedir`, then separate the output into two archives. The CUDA libs archive could be built less frequently (only when torch/CUDA version changes) and stored as a pinned release asset.
|
||||
|
||||
#### Download experience
|
||||
|
||||
- First-time CUDA setup: ~2.4GB total (same as today)
|
||||
- Subsequent app updates: ~200-400MB for the server, CUDA libs stay cached
|
||||
- CUDA toolkit bump: ~2GB for just the libs
|
||||
|
||||
#### Pros
|
||||
|
||||
- PyInstaller `--onedir` natively produces this structure -- NVIDIA DLLs end up as discrete files in the output directory
|
||||
- The separation is natural: PyInstaller puts torch's NVIDIA deps in predictable paths (`nvidia/cublas/lib/`, etc.)
|
||||
- CUDA libs are highly stable -- only rebundle when changing CUDA toolkit version (e.g., cu126 -> cu128) or major torch version
|
||||
- Server updates become ~200-400MB instead of ~2.4GB
|
||||
- No library path hacking needed -- torch finds NVIDIA DLLs because they're in the same directory tree
|
||||
|
||||
#### Cons
|
||||
|
||||
- Onedir means a folder with hundreds of files instead of a single exe -- more complex to manage, extract, and clean up
|
||||
- Need to modify download/assembly logic in `cuda.py` to handle two separate archives
|
||||
- The Tauri side (`main.rs`) needs to point at an exe inside a directory rather than a standalone binary
|
||||
- Users who manually manage the file may find the folder structure confusing
|
||||
|
||||
#### TTS engine compatibility
|
||||
|
||||
No issues. The TTS engines are pure Python + torch. They don't care whether NVIDIA libs are inside the binary or sitting next to it -- torch's dynamic loader finds them either way.
|
||||
|
||||
---
|
||||
|
||||
### Option B: Keep `--onefile` but Externalize CUDA Libs via Library Path
|
||||
|
||||
Keep the server as a single `--onefile` binary (with NVIDIA packages excluded, same as the CPU build). Ship the CUDA libs as a separate download that gets extracted to `{data_dir}/backends/cuda-libs/`. Before launching, set the library search path to include that directory.
|
||||
|
||||
**Important caveat:** The CPU torch wheel (`whl/cpu`) doesn't have CUDA kernels compiled in -- it's a fundamentally different build. So the binary would need to be built with CUDA-compiled torch but with the NVIDIA runtime libraries excluded. The runtime libs (cublas, cudnn, etc.) would be provided externally.
|
||||
|
||||
#### How it would work
|
||||
|
||||
- Build ONE "CUDA-ready" server binary with CUDA-compiled torch but NVIDIA runtime packages excluded
|
||||
- Ship `cuda-libs-cu126-v1.tar.gz` separately (~2GB of `.dll`/`.so` files)
|
||||
- When launching, Tauri sets `PATH` (Windows) or `LD_LIBRARY_PATH` (Linux) to include the cuda-libs directory
|
||||
|
||||
#### Pros
|
||||
|
||||
- Single server binary for both CPU and CUDA users -- simplifies build pipeline enormously
|
||||
- True bolt-on CUDA libs with fully independent versioning
|
||||
- Server updates are always small (~150MB for the onefile binary)
|
||||
|
||||
#### Cons
|
||||
|
||||
- **Fragile on Windows.** PyInstaller `--onefile` extracts to a temp directory at runtime and the internal torch may not find externally-placed NVIDIA libs. DLL resolution on Windows is notoriously unreliable in this scenario.
|
||||
- `os.add_dll_directory()` only affects `LoadLibraryEx` with `LOAD_LIBRARY_SEARCH_USER_DIRS` flag -- not all DLL loads go through this path
|
||||
- PyInstaller's onefile bootloader may configure DLL search paths before Python code runs
|
||||
- Could work on Linux but is fragile on Windows
|
||||
|
||||
---
|
||||
|
||||
### Option C: Hybrid -- `--onefile` Server + Dynamic CUDA Lib Loading at Runtime
|
||||
|
||||
Build the server as `--onefile` with CUDA-compiled torch but with NVIDIA packages excluded. At startup, before torch initializes CUDA, explicitly load the NVIDIA shared libraries using `ctypes.CDLL` or `os.add_dll_directory()`.
|
||||
|
||||
In `server.py`, before any torch imports:
|
||||
|
||||
```python
|
||||
cuda_libs_dir = os.environ.get("VOICEBOX_CUDA_LIBS")
|
||||
if cuda_libs_dir and os.path.isdir(cuda_libs_dir):
|
||||
if sys.platform == "win32":
|
||||
os.add_dll_directory(cuda_libs_dir)
|
||||
os.environ["PATH"] = cuda_libs_dir + os.pathsep + os.environ.get("PATH", "")
|
||||
else:
|
||||
os.environ["LD_LIBRARY_PATH"] = cuda_libs_dir + ":" + os.environ.get("LD_LIBRARY_PATH", "")
|
||||
```
|
||||
|
||||
#### Pros
|
||||
|
||||
- Single server binary, true bolt-on CUDA libs
|
||||
- Clean separation of concerns
|
||||
- Independent versioning
|
||||
|
||||
#### Cons
|
||||
|
||||
- Needs careful testing with each torch version -- CUDA initialization happens deep in C++ extension layer
|
||||
- On Windows, `os.add_dll_directory()` may not cover all DLL load paths
|
||||
- PyInstaller's onefile bootloader may have already configured DLL search paths before Python code runs
|
||||
- Most complex to get right and maintain
|
||||
|
||||
## Recommendation
|
||||
|
||||
**Option A (`--onedir` with split archives)** is the most reliable path:
|
||||
|
||||
1. **It actually works.** `--onedir` puts all files on disk as regular files. Torch finds NVIDIA DLLs because they're in the same directory tree, exactly as they would be in a normal pip install.
|
||||
2. **Natural separation.** PyInstaller's `--onedir` output already separates the NVIDIA `.dll`/`.so` files into `nvidia/` subdirectories. We can split the output directory into "core" and "nvidia-libs" archives after building.
|
||||
3. **Independent versioning is straightforward.** A `cuda-libs.json` manifest controls when redownloads are needed.
|
||||
4. **Build pipeline simplification.** Build CUDA libs archive less frequently, store as a pinned release asset.
|
||||
|
||||
The main cost is managing a directory instead of a single file, but we already have sophisticated download/assembly infrastructure in `cuda.py` with manifests and split parts. Extending that to handle two archives is incremental work.
|
||||
|
||||
## Tauri Compatibility (Validated)
|
||||
|
||||
Tauri handles PyInstaller `--onedir` with no issues. The key insight is that we're **not** using a static sidecar for CUDA -- we're downloading and extracting at runtime (the existing `cuda.py` + `main.rs` flow). For runtime-launched processes, Tauri's `tauri::shell::Command` supports arbitrary directories natively.
|
||||
|
||||
### The critical change in `main.rs`
|
||||
|
||||
The only Tauri-side change needed is adding `.current_dir()` when spawning the CUDA backend:
|
||||
|
||||
```rust
|
||||
let cuda_dir = data_dir.join("backends/cuda");
|
||||
let exe_path = cuda_dir.join("voicebox-server-cuda.exe");
|
||||
|
||||
let mut cmd = app.shell().command(exe_path.to_str().unwrap());
|
||||
cmd = cmd.current_dir(&cuda_dir); // PyInstaller finds all DLLs relative to exe
|
||||
cmd = cmd.args(["--data-dir", &data_dir_str, "--port", &port_str, "--parent-pid", &parent_pid_str]);
|
||||
```
|
||||
|
||||
`.current_dir()` tells the PyInstaller bootloader that everything (DLLs, `nvidia/cublas/lib/`, `_internal/`, torch extensions, etc.) lives relative to the exe. Torch finds the NVIDIA libs exactly as it does in a normal `pip install` or dev environment -- no `LD_LIBRARY_PATH` hacks, no `os.add_dll_directory` gymnastics.
|
||||
|
||||
### Community evidence
|
||||
|
||||
- Multiple Tauri users run this exact pattern: Nuitka folders (exe + pythonXX.dll + supporting files), multi-file .NET apps, and PyInstaller onedir backends (GitHub issues #5719, discussion #5206).
|
||||
- The shell plugin explicitly supports `cwd` in both Rust and JS APIs.
|
||||
- No reports of torch/CUDA-specific breakage -- the onedir layout is identical to what PyInstaller produces in normal usage.
|
||||
|
||||
### Known gotcha: process termination on Windows
|
||||
|
||||
PyInstaller onedir creates a parent bootloader + child Python process on Windows. `child.kill()` only hits the outer process in some cases (Tauri issue #11686). Mitigation: keep a reference to the parent PID or use `taskkill /F /T` for clean shutdown. This is not a blocker -- our existing `--parent-pid` watchdog mechanism in `server.py` already handles orphan cleanup.
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Prototype: Build the current CUDA binary with `--onedir` and verify torch CUDA works from the output directory
|
||||
2. Measure the size split: how much is NVIDIA libs vs everything else
|
||||
3. Design the two-archive download flow and dual version checking
|
||||
4. Update `cuda.py` for dual-archive extraction (server core + cuda-libs)
|
||||
5. Update `main.rs`: change launch path to `backends/cuda/` dir + add `.current_dir()`
|
||||
6. Add `ensure_cuda_structure()` helper in Rust to verify exe + nvidia/ subdirs exist before spawning
|
||||
7. Update CI pipeline: `build-cuda-windows` produces two archives instead of split parts
|
||||
8. ~~Update `split_binary.py` or replace with archive-based distribution~~ Done: replaced with `package_cuda.py`
|
||||
@@ -46,6 +46,8 @@ setup-python:
|
||||
{{ pip }} install -r {{ backend_dir }}/requirements.txt
|
||||
# Chatterbox pins numpy<1.26 / torch==2.6 which break on Python 3.12+
|
||||
{{ pip }} install --no-deps chatterbox-tts
|
||||
# HumeAI TADA pins torch>=2.7,<2.8 which conflicts with our torch>=2.1
|
||||
{{ pip }} install --no-deps hume-tada
|
||||
# Apple Silicon: install MLX backend
|
||||
if [ "$(uname -m)" = "arm64" ] && [ "$(uname)" = "Darwin" ]; then
|
||||
echo "Detected Apple Silicon — installing MLX dependencies..."
|
||||
@@ -74,6 +76,7 @@ setup-python:
|
||||
}
|
||||
& "{{ pip }}" install -r {{ backend_dir }}/requirements.txt
|
||||
& "{{ pip }}" install --no-deps chatterbox-tts
|
||||
& "{{ pip }}" install --no-deps hume-tada
|
||||
& "{{ pip }}" install git+https://github.com/QwenLM/Qwen3-TTS.git
|
||||
& "{{ pip }}" install pyinstaller ruff pytest pytest-asyncio -q
|
||||
Write-Host "Python environment ready."
|
||||
@@ -205,10 +208,11 @@ build-server-cuda: _ensure-venv
|
||||
$env:PATH = "{{ venv_bin }};$env:PATH"; \
|
||||
& "{{ python }}" backend/build_binary.py --cuda; \
|
||||
if ($LASTEXITCODE -ne 0) { throw "build_binary.py --cuda failed with exit code $LASTEXITCODE" }; \
|
||||
$dest = "$env:APPDATA/com.voicebox.app/backends"; \
|
||||
$dest = "$env:APPDATA/sh.voicebox.app/backends/cuda"; \
|
||||
if (Test-Path $dest) { Remove-Item -Recurse -Force $dest }; \
|
||||
New-Item -ItemType Directory -Path $dest -Force | Out-Null; \
|
||||
Copy-Item "backend/dist/voicebox-server-cuda.exe" "$dest/voicebox-server-cuda.exe" -Force; \
|
||||
Write-Host "Copied CUDA binary to $dest"
|
||||
Copy-Item "backend/dist/voicebox-server-cuda/*" $dest -Recurse -Force; \
|
||||
Write-Host "Copied CUDA backend to $dest"
|
||||
|
||||
# Build everything locally: CPU server + CUDA server + installable Tauri app
|
||||
[windows]
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@voicebox/landing",
|
||||
"version": "0.3.0",
|
||||
"version": "0.3.1",
|
||||
"description": "Landing page for voicebox.sh",
|
||||
"scripts": {
|
||||
"dev": "bun --bun next dev --turbo",
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "voicebox",
|
||||
"version": "0.3.0",
|
||||
"version": "0.3.1",
|
||||
"private": true,
|
||||
"workspaces": [
|
||||
"app",
|
||||
|
||||
@@ -0,0 +1,232 @@
|
||||
"""
|
||||
Package the PyInstaller --onedir CUDA build into two archives.
|
||||
|
||||
Takes the PyInstaller --onedir output directory and splits it into:
|
||||
1. voicebox-server-cuda.tar.gz — server core (exe + non-NVIDIA deps)
|
||||
2. cuda-libs-cu126.tar.gz — NVIDIA runtime libraries only
|
||||
3. cuda-libs.json — version manifest for the CUDA libs
|
||||
|
||||
Usage:
|
||||
python scripts/package_cuda.py backend/dist/voicebox-server-cuda/
|
||||
python scripts/package_cuda.py backend/dist/voicebox-server-cuda/ --output release-assets/
|
||||
python scripts/package_cuda.py backend/dist/voicebox-server-cuda/ --cuda-libs-version cu126-v1
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import sys
|
||||
import tarfile
|
||||
from pathlib import Path
|
||||
|
||||
# DLL name prefixes that identify NVIDIA CUDA runtime libraries.
|
||||
# These DLLs may appear in different locations depending on the torch
|
||||
# and PyInstaller version:
|
||||
# - nvidia/ subdirectories (older torch with separate nvidia-* packages)
|
||||
# - _internal/torch/lib/ (torch 2.10+ bundles NVIDIA DLLs directly)
|
||||
# - Top-level directory (some PyInstaller versions)
|
||||
NVIDIA_DLL_PREFIXES = (
|
||||
"cublas",
|
||||
"cublaslt",
|
||||
"cudart",
|
||||
"cudnn",
|
||||
"cufft",
|
||||
"cufftw",
|
||||
"curand",
|
||||
"cusolver",
|
||||
"cusolvermg",
|
||||
"cusparse",
|
||||
"nvjitlink",
|
||||
"nvrtc",
|
||||
"nccl",
|
||||
"caffe2_nvrtc",
|
||||
)
|
||||
|
||||
# Files to keep in the server core even if they match NVIDIA prefixes.
|
||||
# These are small Python modules or stubs, not the large runtime DLLs.
|
||||
NVIDIA_KEEP_IN_CORE = {
|
||||
"torch/cuda/nccl.py",
|
||||
"torch/_inductor/codegen/cuda/cutlass_lib_extensions/cutlass_mock_imports/cuda/cudart.py",
|
||||
}
|
||||
|
||||
|
||||
def is_nvidia_file(rel_path: str) -> bool:
|
||||
"""Check if a relative path belongs to the NVIDIA CUDA libs.
|
||||
|
||||
Identifies large NVIDIA runtime DLLs (.dll/.so) regardless of where
|
||||
PyInstaller placed them. Excludes small Python stubs that happen to
|
||||
share NVIDIA-related names.
|
||||
"""
|
||||
rel_lower = rel_path.lower().replace("\\", "/")
|
||||
|
||||
# Never split out Python source files or small stubs
|
||||
if rel_lower in NVIDIA_KEEP_IN_CORE:
|
||||
return False
|
||||
|
||||
# Files under nvidia/ subdirectory tree (older torch layout)
|
||||
if rel_lower.startswith("nvidia/") or "/nvidia/" in rel_lower:
|
||||
# Only DLLs/shared objects — not .py, .dist-info, etc.
|
||||
if rel_lower.endswith((".dll", ".so")):
|
||||
return True
|
||||
# Include entire nvidia/ namespace package tree
|
||||
for part in rel_lower.split("/"):
|
||||
if part == "nvidia":
|
||||
return True
|
||||
|
||||
# NVIDIA DLLs anywhere in the tree (e.g. _internal/torch/lib/cublas64_12.dll)
|
||||
name = rel_lower.rsplit("/", 1)[-1]
|
||||
if name.endswith(".dll") or name.endswith(".so"):
|
||||
name_no_ext = name.rsplit(".", 1)[0]
|
||||
for prefix in NVIDIA_DLL_PREFIXES:
|
||||
if name_no_ext.startswith(prefix):
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def sha256_file(path: Path) -> str:
|
||||
"""Compute SHA-256 hex digest of a file."""
|
||||
h = hashlib.sha256()
|
||||
with open(path, "rb") as f:
|
||||
while True:
|
||||
chunk = f.read(1024 * 1024)
|
||||
if not chunk:
|
||||
break
|
||||
h.update(chunk)
|
||||
return h.hexdigest()
|
||||
|
||||
|
||||
def package(
|
||||
onedir_path: Path,
|
||||
output_dir: Path,
|
||||
cuda_libs_version: str,
|
||||
torch_compat: str,
|
||||
):
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Collect all files in the onedir output, split into core vs nvidia
|
||||
core_files = []
|
||||
nvidia_files = []
|
||||
|
||||
for item in sorted(onedir_path.rglob("*")):
|
||||
if item.is_dir():
|
||||
continue
|
||||
rel = item.relative_to(onedir_path)
|
||||
rel_str = str(rel)
|
||||
if is_nvidia_file(rel_str):
|
||||
nvidia_files.append((rel_str, item))
|
||||
else:
|
||||
core_files.append((rel_str, item))
|
||||
|
||||
core_size = sum(f.stat().st_size for _, f in core_files)
|
||||
nvidia_size = sum(f.stat().st_size for _, f in nvidia_files)
|
||||
|
||||
print(f"Input directory: {onedir_path}")
|
||||
print(f"Core files: {len(core_files)} ({core_size / (1024**2):.1f} MB)")
|
||||
print(f"NVIDIA files: {len(nvidia_files)} ({nvidia_size / (1024**2):.1f} MB)")
|
||||
|
||||
if not nvidia_files:
|
||||
print(
|
||||
f"ERROR: No NVIDIA files found in {onedir_path}. "
|
||||
"Refusing to create an empty CUDA libs archive.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
print(
|
||||
"Make sure you built with --cuda and the NVIDIA packages are present.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
# Create server core archive
|
||||
# Files are stored relative to the archive root (no parent directory prefix)
|
||||
# so extracting to backends/cuda/ puts everything at the right level.
|
||||
server_archive = output_dir / "voicebox-server-cuda.tar.gz"
|
||||
print(f"\nCreating server core archive: {server_archive.name}")
|
||||
with tarfile.open(server_archive, "w:gz") as tar:
|
||||
for rel_str, full_path in core_files:
|
||||
tar.add(full_path, arcname=rel_str)
|
||||
server_sha = sha256_file(server_archive)
|
||||
(output_dir / "voicebox-server-cuda.tar.gz.sha256").write_text(
|
||||
f"{server_sha} voicebox-server-cuda.tar.gz\n"
|
||||
)
|
||||
print(f" Size: {server_archive.stat().st_size / (1024**2):.1f} MB")
|
||||
print(f" SHA-256: {server_sha[:16]}...")
|
||||
|
||||
# Create CUDA libs archive
|
||||
cuda_libs_archive = output_dir / f"cuda-libs-{cuda_libs_version}.tar.gz"
|
||||
print(f"\nCreating CUDA libs archive: {cuda_libs_archive.name}")
|
||||
with tarfile.open(cuda_libs_archive, "w:gz") as tar:
|
||||
for rel_str, full_path in nvidia_files:
|
||||
tar.add(full_path, arcname=rel_str)
|
||||
cuda_sha = sha256_file(cuda_libs_archive)
|
||||
(output_dir / f"cuda-libs-{cuda_libs_version}.tar.gz.sha256").write_text(
|
||||
f"{cuda_sha} cuda-libs-{cuda_libs_version}.tar.gz\n"
|
||||
)
|
||||
print(f" Size: {cuda_libs_archive.stat().st_size / (1024**2):.1f} MB")
|
||||
print(f" SHA-256: {cuda_sha[:16]}...")
|
||||
|
||||
# Write cuda-libs.json manifest
|
||||
manifest = {
|
||||
"version": cuda_libs_version,
|
||||
"torch_compat": torch_compat,
|
||||
"archive": cuda_libs_archive.name,
|
||||
"sha256": cuda_sha,
|
||||
}
|
||||
manifest_path = output_dir / "cuda-libs.json"
|
||||
manifest_path.write_text(json.dumps(manifest, indent=2) + "\n")
|
||||
print(f"\nManifest: {manifest_path.name}")
|
||||
print(json.dumps(manifest, indent=2))
|
||||
|
||||
# Summary
|
||||
total_input = core_size + nvidia_size
|
||||
total_output = server_archive.stat().st_size + cuda_libs_archive.stat().st_size
|
||||
print(f"\nTotal input: {total_input / (1024**3):.2f} GB")
|
||||
print(f"Total output: {total_output / (1024**3):.2f} GB (compressed)")
|
||||
print(
|
||||
f"Server core: {server_archive.stat().st_size / (1024**2):.1f} MB (redownloaded on app update)"
|
||||
)
|
||||
print(
|
||||
f"CUDA libs: {cuda_libs_archive.stat().st_size / (1024**2):.1f} MB (cached until CUDA toolkit bump)"
|
||||
)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Package PyInstaller --onedir CUDA build into server + CUDA libs archives"
|
||||
)
|
||||
parser.add_argument(
|
||||
"input",
|
||||
type=Path,
|
||||
help="Path to PyInstaller --onedir output directory (e.g. backend/dist/voicebox-server-cuda/)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="Output directory for archives (default: same as input parent)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--cuda-libs-version",
|
||||
type=str,
|
||||
default="cu126-v1",
|
||||
help="Version string for the CUDA libs archive (default: cu126-v1)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--torch-compat",
|
||||
type=str,
|
||||
default=">=2.6.0,<2.11.0",
|
||||
help="Torch version compatibility range (default: >=2.6.0,<2.11.0)",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
if not args.input.is_dir():
|
||||
print(f"Error: {args.input} is not a directory", file=sys.stderr)
|
||||
print("Expected a PyInstaller --onedir output directory.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
output_dir = args.output or args.input.parent
|
||||
package(args.input, output_dir, args.cuda_libs_version, args.torch_compat)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,82 +0,0 @@
|
||||
"""
|
||||
Split a large binary into chunks for GitHub Releases (<2 GB each).
|
||||
|
||||
Usage:
|
||||
python scripts/split_binary.py backend/dist/voicebox-server-cuda.exe
|
||||
python scripts/split_binary.py backend/dist/voicebox-server-cuda.exe --chunk-size 1900000000
|
||||
python scripts/split_binary.py backend/dist/voicebox-server-cuda.exe --output release-assets/
|
||||
|
||||
The script produces:
|
||||
- voicebox-server-cuda.part00.exe, .part01.exe, ... (binary chunks)
|
||||
- voicebox-server-cuda.sha256 (SHA-256 checksum of the complete file)
|
||||
- voicebox-server-cuda.manifest (ordered list of part filenames)
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def split(input_path: Path, chunk_size: int, output_dir: Path):
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
data = input_path.read_bytes()
|
||||
total_size = len(data)
|
||||
|
||||
# Write SHA-256 of the complete file
|
||||
sha256 = hashlib.sha256(data).hexdigest()
|
||||
checksum_file = output_dir / f"{input_path.stem}.sha256"
|
||||
checksum_file.write_text(f"{sha256} {input_path.name}\n")
|
||||
|
||||
# Split into chunks
|
||||
parts = []
|
||||
for i in range(0, total_size, chunk_size):
|
||||
part_index = len(parts)
|
||||
part_name = f"{input_path.stem}.part{part_index:02d}{input_path.suffix}"
|
||||
part_path = output_dir / part_name
|
||||
part_path.write_bytes(data[i:i + chunk_size])
|
||||
parts.append(part_name)
|
||||
|
||||
# Write manifest (ordered list of part filenames)
|
||||
manifest_file = output_dir / f"{input_path.stem}.manifest"
|
||||
manifest_file.write_text("\n".join(parts) + "\n")
|
||||
|
||||
print(f"Input: {input_path} ({total_size / (1024**3):.2f} GB)")
|
||||
print(f"Output: {output_dir}/")
|
||||
print(f"Parts: {len(parts)} (chunk size: {chunk_size / (1024**3):.2f} GB)")
|
||||
print(f"SHA-256: {sha256}")
|
||||
print(f"Manifest: {manifest_file.name}")
|
||||
for p in parts:
|
||||
size = (output_dir / p).stat().st_size
|
||||
print(f" {p} ({size / (1024**3):.2f} GB)")
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Split a large binary into chunks for GitHub Releases"
|
||||
)
|
||||
parser.add_argument("input", type=Path, help="Path to the binary file to split")
|
||||
parser.add_argument(
|
||||
"--chunk-size",
|
||||
type=int,
|
||||
default=1_900_000_000, # 1.9 GB — safely under 2 GB GitHub limit
|
||||
help="Maximum chunk size in bytes (default: 1.9 GB)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="Output directory (default: same directory as input)",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
if not args.input.exists():
|
||||
print(f"Error: {args.input} does not exist", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
output_dir = args.output or args.input.parent
|
||||
split(args.input, args.chunk_size, output_dir)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "@voicebox/tauri",
|
||||
"private": true,
|
||||
"version": "0.3.0",
|
||||
"version": "0.3.1",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "voicebox"
|
||||
version = "0.3.0"
|
||||
version = "0.3.1"
|
||||
description = "A production-quality desktop app for Qwen3-TTS voice cloning and generation"
|
||||
authors = ["you"]
|
||||
license = ""
|
||||
|
||||
+16
-10
@@ -197,22 +197,24 @@ async fn start_server(
|
||||
println!("Data directory: {:?}", data_dir);
|
||||
println!("Remote mode: {}", remote.unwrap_or(false));
|
||||
|
||||
// Check for CUDA backend binary in data directory
|
||||
// Check for CUDA backend in data directory (onedir layout: backends/cuda/)
|
||||
let cuda_binary = {
|
||||
let backends_dir = data_dir.join("backends");
|
||||
let cuda_dir = data_dir.join("backends").join("cuda");
|
||||
let cuda_name = if cfg!(windows) {
|
||||
"voicebox-server-cuda.exe"
|
||||
} else {
|
||||
"voicebox-server-cuda"
|
||||
};
|
||||
let path = backends_dir.join(cuda_name);
|
||||
if path.exists() {
|
||||
println!("Found CUDA backend binary at {:?}", path);
|
||||
let exe_path = cuda_dir.join(cuda_name);
|
||||
if exe_path.exists() {
|
||||
println!("Found CUDA backend at {:?}", cuda_dir);
|
||||
|
||||
// Version check: run --version and compare to app version
|
||||
// Version check: run --version from the onedir directory so
|
||||
// PyInstaller can find its support files for the fast --version path
|
||||
let app_version = app.config().version.clone().unwrap_or_default();
|
||||
let version_ok = match std::process::Command::new(&path)
|
||||
let version_ok = match std::process::Command::new(&exe_path)
|
||||
.arg("--version")
|
||||
.current_dir(&cuda_dir)
|
||||
.output()
|
||||
{
|
||||
Ok(output) => {
|
||||
@@ -237,7 +239,7 @@ async fn start_server(
|
||||
};
|
||||
|
||||
if version_ok {
|
||||
Some(path)
|
||||
Some(exe_path)
|
||||
} else {
|
||||
None
|
||||
}
|
||||
@@ -300,10 +302,14 @@ async fn start_server(
|
||||
println!("Custom models directory: {}", dir);
|
||||
}
|
||||
|
||||
// If CUDA binary exists, launch it directly instead of the bundled sidecar
|
||||
// If CUDA binary exists, launch it from the onedir directory.
|
||||
// .current_dir() is critical: PyInstaller onedir expects all DLLs and
|
||||
// support files (nvidia/, _internal/, etc.) relative to the exe.
|
||||
let spawn_result = if let Some(ref cuda_path) = cuda_binary {
|
||||
println!("Launching CUDA backend: {:?}", cuda_path);
|
||||
let cuda_dir = cuda_path.parent().unwrap();
|
||||
println!("Launching CUDA backend: {:?} (cwd: {:?})", cuda_path, cuda_dir);
|
||||
let mut cmd = app.shell().command(cuda_path.to_str().unwrap());
|
||||
cmd = cmd.current_dir(cuda_dir);
|
||||
cmd = cmd.args(["--data-dir", &data_dir_str, "--port", &port_str, "--parent-pid", &parent_pid_str]);
|
||||
if is_remote {
|
||||
cmd = cmd.args(["--host", "0.0.0.0"]);
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"$schema": "https://schema.tauri.app/config/2",
|
||||
"productName": "Voicebox",
|
||||
"version": "0.3.0",
|
||||
"version": "0.3.1",
|
||||
"identifier": "sh.voicebox.app",
|
||||
"build": {
|
||||
"beforeDevCommand": "bun run dev",
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "@voicebox/web",
|
||||
"private": true,
|
||||
"version": "0.3.0",
|
||||
"version": "0.3.1",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
|
||||
Reference in New Issue
Block a user