diff --git a/docs/PROJECT_STATUS.md b/docs/PROJECT_STATUS.md index 6aea31ab..d6a1daff 100644 --- a/docs/PROJECT_STATUS.md +++ b/docs/PROJECT_STATUS.md @@ -19,36 +19,20 @@ ## Architecture Overview -``` -┌─────────────────────────────────────────────────────┐ -│ Tauri Shell (Rust) │ -│ ┌───────────────────────────────────────────────┐ │ -│ │ React Frontend (app/) │ │ -│ │ Zustand stores · API client · Generation UI │ │ -│ │ Stories Editor · Voice Profiles · Model Mgmt │ │ -│ └──────────────────────┬────────────────────────┘ │ -│ │ HTTP :17493 │ -│ ┌──────────────────────▼────────────────────────┐ │ -│ │ FastAPI Backend (backend/) │ │ -│ │ ┌─────────────────────────────────────────┐ │ │ -│ │ │ TTSBackend Protocol │ │ │ -│ │ │ ┌──────────┐ ┌───────┐ ┌───────────┐ │ │ │ -│ │ │ │ Qwen3-TTS│ │LuxTTS │ │Chatterbox │ │ │ │ -│ │ │ │(Py/MLX) │ │ │ │(MTL+Turbo)│ │ │ │ -│ │ │ └──────────┘ └───────┘ └───────────┘ │ │ │ -│ │ │ ┌──────────┐ ┌────────┐ ┌──────────┐ │ │ │ -│ │ │ │ TADA │ │Kokoro │ │Qwen │ │ │ │ -│ │ │ │(1B / 3B) │ │(82M) │ │CustomVoice│ │ │ │ -│ │ │ └──────────┘ └────────┘ └──────────┘ │ │ │ -│ │ └─────────────────────────────────────────┘ │ │ -│ │ ┌───────────┐ ┌─────────┐ │ │ -│ │ │ STTBackend│ │ Profiles│ │ │ -│ │ │ (Whisper) │ │ History │ │ │ -│ │ └───────────┘ │ Stories │ │ │ -│ │ └─────────┘ │ │ -│ └───────────────────────────────────────────────┘ │ -└─────────────────────────────────────────────────────┘ -``` +**Tauri shell (Rust)** hosts a **React frontend** (`app/`) that talks over HTTP on `localhost:17493` to a **FastAPI backend** (`backend/`). + +The backend exposes: + +- **`TTSBackend` Protocol** with seven concrete engine implementations: + - Qwen3-TTS (PyTorch or MLX depending on platform) + - Qwen CustomVoice (predefined speakers with instruct) + - LuxTTS (fast, CPU-friendly) + - Chatterbox Multilingual (23 languages) + - Chatterbox Turbo (English, paralinguistic tags) + - TADA (1B English, 3B multilingual via HumeAI) + - Kokoro 82M (pre-built voices, CPU realtime) +- **`STTBackend` Protocol** for Whisper (PyTorch or MLX-Whisper) +- **Profiles / History / Stories** services for persistence and timeline editing ### Key Files diff --git a/docs/content/docs/developer/architecture.mdx b/docs/content/docs/developer/architecture.mdx index c7090b58..dda492f5 100644 --- a/docs/content/docs/developer/architecture.mdx +++ b/docs/content/docs/developer/architecture.mdx @@ -83,12 +83,11 @@ These two layers communicate via HTTP on `localhost:17493`, with the frontend ma - - + diff --git a/docs/content/docs/developer/model-management.mdx b/docs/content/docs/developer/model-management.mdx index a05bfdf7..93c87e57 100644 --- a/docs/content/docs/developer/model-management.mdx +++ b/docs/content/docs/developer/model-management.mdx @@ -26,7 +26,7 @@ Every model is described by a `ModelConfig` entry in `backend/backends/__init__. | **Chatterbox Turbo** | `chatterbox_turbo` | `ResembleAI/chatterbox-turbo` | 1.5 GB | ~1.5 GB | English | | **TADA 1B** | `tada` | `HumeAI/tada-1b` | 4 GB | ~4 GB | English | | **TADA 3B Multilingual** | `tada` | `HumeAI/tada-3b-ml` | 8 GB | ~8 GB | 10 | -| **Kokoro 82M** | `kokoro` | `hexgrad/Kokoro-82M` | 350 MB | ~150 MB | 9 | +| **Kokoro 82M** | `kokoro` | `hexgrad/Kokoro-82M` | 350 MB | ~150 MB | 8 | On Apple Silicon, Qwen TTS uses MLX-optimized repos from `mlx-community` instead of the PyTorch repos. The backend picks automatically via `get_backend_type()`. @@ -59,7 +59,7 @@ Set `VOICEBOX_MODELS_DIR` to override. Downloads stream progress to the frontend via Server-Sent Events. The progress pipeline has three pieces: -**`ProgressManager`** (`backend/services/progress.py`) — in-memory map of `model_name → {current, total, filename, status}`. +**`ProgressManager`** (`backend/utils/progress.py`) — in-memory map of `model_name → {current, total, filename, status}`. **`HFProgressTracker`** — context manager that intercepts HuggingFace Hub downloads to emit byte-level progress. Needed because `huggingface_hub` silently disables tqdm in frozen PyInstaller builds. diff --git a/docs/content/docs/developer/tts-generation.mdx b/docs/content/docs/developer/tts-generation.mdx index 6dc4ed58..5b902755 100644 --- a/docs/content/docs/developer/tts-generation.mdx +++ b/docs/content/docs/developer/tts-generation.mdx @@ -50,7 +50,7 @@ class ModelConfig: size_mb: int = 0 needs_trim: bool = False supports_instruct: bool = False - languages: list[str] = ["en"] + languages: list[str] = field(default_factory=lambda: ["en"]) ``` Registry helpers in `backends/__init__.py` replace what used to be per-engine `if/elif` chains: @@ -223,7 +223,7 @@ The model management API (`/models/load`, `/models/unload`) lets users free VRAM - **Voice prompt caching** saves ~1-2s on repeated generations from the same profile. - **Model pinning** — the first load is slow (download + load), subsequent generations reuse the cached model in memory. -### Per-engine VRAM (approximate, 1080p samples) +### Per-engine VRAM (approximate, on CUDA) | Engine | VRAM | |--------|------|