mirror of
https://github.com/jamiepine/voicebox.git
synced 2026-09-18 06:10:43 -07:00
docs: address PR review feedback
- architecture.mdx: fix backends/ file list (remove nonexistent qwen_backend.py, rename tada_backend.py → hume_backend.py) - model-management.mdx: Kokoro language count 9 → 8 (matches ModelConfig) - model-management.mdx: ProgressManager path services/ → utils/ - tts-generation.mdx: ModelConfig example uses field(default_factory=...) — mutable default would raise at runtime - tts-generation.mdx: "1080p samples" → "on CUDA" (1080p is video, not audio) - PROJECT_STATUS.md: replace ASCII architecture diagram with prose (matches no-ASCII-art rule) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.7
parent
3412dca662
commit
b91d5d74cb
+14
-30
@@ -19,36 +19,20 @@
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────┐
|
||||
│ Tauri Shell (Rust) │
|
||||
│ ┌───────────────────────────────────────────────┐ │
|
||||
│ │ React Frontend (app/) │ │
|
||||
│ │ Zustand stores · API client · Generation UI │ │
|
||||
│ │ Stories Editor · Voice Profiles · Model Mgmt │ │
|
||||
│ └──────────────────────┬────────────────────────┘ │
|
||||
│ │ HTTP :17493 │
|
||||
│ ┌──────────────────────▼────────────────────────┐ │
|
||||
│ │ FastAPI Backend (backend/) │ │
|
||||
│ │ ┌─────────────────────────────────────────┐ │ │
|
||||
│ │ │ TTSBackend Protocol │ │ │
|
||||
│ │ │ ┌──────────┐ ┌───────┐ ┌───────────┐ │ │ │
|
||||
│ │ │ │ Qwen3-TTS│ │LuxTTS │ │Chatterbox │ │ │ │
|
||||
│ │ │ │(Py/MLX) │ │ │ │(MTL+Turbo)│ │ │ │
|
||||
│ │ │ └──────────┘ └───────┘ └───────────┘ │ │ │
|
||||
│ │ │ ┌──────────┐ ┌────────┐ ┌──────────┐ │ │ │
|
||||
│ │ │ │ TADA │ │Kokoro │ │Qwen │ │ │ │
|
||||
│ │ │ │(1B / 3B) │ │(82M) │ │CustomVoice│ │ │ │
|
||||
│ │ │ └──────────┘ └────────┘ └──────────┘ │ │ │
|
||||
│ │ └─────────────────────────────────────────┘ │ │
|
||||
│ │ ┌───────────┐ ┌─────────┐ │ │
|
||||
│ │ │ STTBackend│ │ Profiles│ │ │
|
||||
│ │ │ (Whisper) │ │ History │ │ │
|
||||
│ │ └───────────┘ │ Stories │ │ │
|
||||
│ │ └─────────┘ │ │
|
||||
│ └───────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────┘
|
||||
```
|
||||
**Tauri shell (Rust)** hosts a **React frontend** (`app/`) that talks over HTTP on `localhost:17493` to a **FastAPI backend** (`backend/`).
|
||||
|
||||
The backend exposes:
|
||||
|
||||
- **`TTSBackend` Protocol** with seven concrete engine implementations:
|
||||
- Qwen3-TTS (PyTorch or MLX depending on platform)
|
||||
- Qwen CustomVoice (predefined speakers with instruct)
|
||||
- LuxTTS (fast, CPU-friendly)
|
||||
- Chatterbox Multilingual (23 languages)
|
||||
- Chatterbox Turbo (English, paralinguistic tags)
|
||||
- TADA (1B English, 3B multilingual via HumeAI)
|
||||
- Kokoro 82M (pre-built voices, CPU realtime)
|
||||
- **`STTBackend` Protocol** for Whisper (PyTorch or MLX-Whisper)
|
||||
- **Profiles / History / Stories** services for persistence and timeline editing
|
||||
|
||||
### Key Files
|
||||
|
||||
|
||||
@@ -83,12 +83,11 @@ These two layers communicate via HTTP on `localhost:17493`, with the frontend ma
|
||||
<File name="base.py" />
|
||||
<File name="pytorch_backend.py" />
|
||||
<File name="mlx_backend.py" />
|
||||
<File name="qwen_backend.py" />
|
||||
<File name="qwen_custom_voice_backend.py" />
|
||||
<File name="luxtts_backend.py" />
|
||||
<File name="chatterbox_backend.py" />
|
||||
<File name="chatterbox_turbo_backend.py" />
|
||||
<File name="tada_backend.py" />
|
||||
<File name="hume_backend.py" />
|
||||
<File name="kokoro_backend.py" />
|
||||
</Folder>
|
||||
<Folder name="database">
|
||||
|
||||
@@ -26,7 +26,7 @@ Every model is described by a `ModelConfig` entry in `backend/backends/__init__.
|
||||
| **Chatterbox Turbo** | `chatterbox_turbo` | `ResembleAI/chatterbox-turbo` | 1.5 GB | ~1.5 GB | English |
|
||||
| **TADA 1B** | `tada` | `HumeAI/tada-1b` | 4 GB | ~4 GB | English |
|
||||
| **TADA 3B Multilingual** | `tada` | `HumeAI/tada-3b-ml` | 8 GB | ~8 GB | 10 |
|
||||
| **Kokoro 82M** | `kokoro` | `hexgrad/Kokoro-82M` | 350 MB | ~150 MB | 9 |
|
||||
| **Kokoro 82M** | `kokoro` | `hexgrad/Kokoro-82M` | 350 MB | ~150 MB | 8 |
|
||||
|
||||
On Apple Silicon, Qwen TTS uses MLX-optimized repos from `mlx-community` instead of the PyTorch repos. The backend picks automatically via `get_backend_type()`.
|
||||
|
||||
@@ -59,7 +59,7 @@ Set `VOICEBOX_MODELS_DIR` to override.
|
||||
|
||||
Downloads stream progress to the frontend via Server-Sent Events. The progress pipeline has three pieces:
|
||||
|
||||
**`ProgressManager`** (`backend/services/progress.py`) — in-memory map of `model_name → {current, total, filename, status}`.
|
||||
**`ProgressManager`** (`backend/utils/progress.py`) — in-memory map of `model_name → {current, total, filename, status}`.
|
||||
|
||||
**`HFProgressTracker`** — context manager that intercepts HuggingFace Hub downloads to emit byte-level progress. Needed because `huggingface_hub` silently disables tqdm in frozen PyInstaller builds.
|
||||
|
||||
|
||||
@@ -50,7 +50,7 @@ class ModelConfig:
|
||||
size_mb: int = 0
|
||||
needs_trim: bool = False
|
||||
supports_instruct: bool = False
|
||||
languages: list[str] = ["en"]
|
||||
languages: list[str] = field(default_factory=lambda: ["en"])
|
||||
```
|
||||
|
||||
Registry helpers in `backends/__init__.py` replace what used to be per-engine `if/elif` chains:
|
||||
@@ -223,7 +223,7 @@ The model management API (`/models/load`, `/models/unload`) lets users free VRAM
|
||||
- **Voice prompt caching** saves ~1-2s on repeated generations from the same profile.
|
||||
- **Model pinning** — the first load is slow (download + load), subsequent generations reuse the cached model in memory.
|
||||
|
||||
### Per-engine VRAM (approximate, 1080p samples)
|
||||
### Per-engine VRAM (approximate, on CUDA)
|
||||
|
||||
| Engine | VRAM |
|
||||
|--------|------|
|
||||
|
||||
Reference in New Issue
Block a user