From 8ac202aa58a14626e04d2d51737822e8f19267ef Mon Sep 17 00:00:00 2001 From: James Pine Date: Tue, 17 Mar 2026 01:13:30 -0700 Subject: [PATCH] docs for adding new engines --- .agents/skills/add-tts-engine/SKILL.md | 120 +++++++ docs/content/docs/developer/tts-engines.mdx | 368 ++++++++++++++++++-- 2 files changed, 460 insertions(+), 28 deletions(-) create mode 100644 .agents/skills/add-tts-engine/SKILL.md diff --git a/.agents/skills/add-tts-engine/SKILL.md b/.agents/skills/add-tts-engine/SKILL.md new file mode 100644 index 00000000..b28a73f8 --- /dev/null +++ b/.agents/skills/add-tts-engine/SKILL.md @@ -0,0 +1,120 @@ +--- +name: add-tts-engine +description: Use this skill to add a new TTS engine to Voicebox. It walks through dependency research, backend implementation, frontend wiring, PyInstaller bundling, and frozen-build testing. Always start with Phase 0 (dependency audit) before writing any code. +--- + +# Add TTS Engine + +## Goal + +Integrate a new text-to-speech engine into Voicebox end-to-end: dependency research, backend protocol implementation, frontend UI wiring, PyInstaller bundling, and frozen-build verification. The user should only need to test the final build locally. + +## Reference Doc + +The full phased guide lives at `docs/content/docs/developer/tts-engines.mdx`. **Read this file in its entirety before starting.** It contains: + +- Phase 0: Dependency research (mandatory before writing code) +- Phase 1: Backend implementation (`TTSBackend` protocol) +- Phase 2: Route and service integration (usually zero changes) +- Phase 3: Frontend integration (5 files) +- Phase 4: Dependencies (`requirements.txt`, justfile, CI, Docker) +- Phase 5: PyInstaller bundling (`build_binary.py` + `server.py`) +- Phase 6: Common upstream workarounds +- Implementation checklist (gate between phases) + +## Workflow + +### 1. Read the guide + +```bash +# Read the full TTS engines doc +cat docs/content/docs/developer/tts-engines.mdx +``` + +Internalize all phases, especially Phase 0 and Phase 5. The v0.2.3 release was three patch releases because Phase 0 was skipped. + +### 2. Dependency research (Phase 0) + +Clone the model library into a temporary directory and audit it. Do NOT skip this. + +```bash +mkdir /tmp/engine-research && cd /tmp/engine-research +git clone +``` + +Run the grep searches from Phase 0.2 in the guide against the cloned source and its transitive dependencies. Produce a written dependency audit covering: + +1. PyPI vs non-PyPI packages +2. PyInstaller directives needed (`--collect-all`, `--copy-metadata`, `--hidden-import`) +3. Runtime data files that must be bundled +4. Native library paths that need env var overrides in frozen builds +5. Monkey-patches needed (`torch.load`, float64, MPS, HF token) +6. Sample rate +7. Model download method (`from_pretrained` vs `snapshot_download` + `from_local`) + +Test model loading and generation on CPU in the throwaway venv before proceeding. + +### 3. Implement (Phases 1–4) + +Follow the guide's phases in order. Key files to modify: + +**Backend (Phase 1):** +- Create `backend/backends/_backend.py` +- Register in `backend/backends/__init__.py` (ModelConfig + TTS_ENGINES + factory) +- Update regex in `backend/models.py` + +**Frontend (Phase 3):** +- `app/src/lib/api/types.ts` — engine union type +- `app/src/lib/constants/languages.ts` — ENGINE_LANGUAGES +- `app/src/components/Generation/EngineModelSelector.tsx` — ENGINE_OPTIONS, ENGINE_DESCRIPTIONS +- `app/src/lib/hooks/useGenerationForm.ts` — Zod schema, model-name mapping +- `app/src/components/ServerSettings/ModelManagement.tsx` — MODEL_DESCRIPTIONS + +**Dependencies (Phase 4):** +- `backend/requirements.txt` +- `justfile` (setup-python, setup-python-release targets) +- `.github/workflows/release.yml` +- `Dockerfile` (if applicable) + +### 4. PyInstaller bundling (Phase 5) + +Register the engine in `backend/build_binary.py`: +- `--hidden-import` for the backend module and model package +- `--collect-all` for packages using `inspect.getsource`, shipping data files, or native libraries +- `--copy-metadata` for packages using `importlib.metadata` + +If the engine has native data paths, add `os.environ.setdefault()` in `backend/server.py` inside the `if getattr(sys, 'frozen', False):` block. + +### 5. Verify in dev mode + +```bash +just dev +``` + +Test the full chain: model download → load → generate → voice cloning. + +### 6. Use the checklist + +Walk through the Implementation Checklist at the bottom of `tts-engines.mdx`. Every item must be checked before handing the build to the user. + +## Key Lessons (from v0.2.3) + +These are the most common failure modes. Phase 0 research catches all of them: + +| Pattern | Symptom in Frozen Build | Fix | +|---------|------------------------|-----| +| `@typechecked` / `inspect.getsource()` | "could not get source code" | `--collect-all ` | +| Package ships pretrained model files | `FileNotFoundError` for `.pth.tar`, `.yaml` | `--collect-all ` | +| C library with hardcoded system paths | `FileNotFoundError` for `/usr/share/...` | `--collect-all` + env var in `server.py` | +| `importlib.metadata.version()` | "No package metadata found" | `--copy-metadata ` | +| `torch.load` without `map_location` | CUDA device not available on CPU build | Monkey-patch `torch.load` | +| `torch.from_numpy` on float64 data | dtype mismatch RuntimeError | Cast to `.float()` | +| `token=True` in HF download calls | Auth failure without stored HF token | Use `snapshot_download(token=None)` + `from_local()` | + +## Notes + +- The route and service layers have zero per-engine dispatch points. `main.py` requires zero changes. +- The model config registry in `backends/__init__.py` handles all dispatch automatically. +- Use `get_torch_device()` and `model_load_progress()` from `backends/base.py` — don't reimplement device detection or progress tracking. +- Always test with a **clean HuggingFace cache** (no pre-downloaded models from dev). +- Do NOT push or create a release. Hand the build to the user for local testing. diff --git a/docs/content/docs/developer/tts-engines.mdx b/docs/content/docs/developer/tts-engines.mdx index 179ac2c8..f95e7a71 100644 --- a/docs/content/docs/developer/tts-engines.mdx +++ b/docs/content/docs/developer/tts-engines.mdx @@ -3,8 +3,12 @@ title: "TTS Engines" description: "How to add new text-to-speech engines to Voicebox" --- +> **For humans:** This doc is optimized for AI agents to implement new TTS engines autonomously. It's structured as a phased workflow with explicit gates and a checklist so an agent can do the full integration — dependency research, backend, frontend, bundling — and hand you a draft release or prod build to test locally. It's also a useful reference if you're doing it yourself. + Adding an engine touches ~10 files across 4 layers. The backend protocol work is straightforward — the real time sink is dependency hell, upstream library bugs, and PyInstaller bundling. +**Do not start writing code until you complete Phase 0.** The v0.2.3 release was three patch releases of PyInstaller fixes because dependency research was skipped. Every issue — `inspect.getsource()` failures, missing native data files, metadata lookups, dtype mismatches — was discoverable by reading the model library's source code before integration began. + ## Architecture Overview The backend is split into layers: @@ -18,6 +22,124 @@ The backend is split into layers: New engines only need to touch `backends/` and `models.py` on the backend side — the route and service layers use a model config registry that handles dispatch automatically. +## Phase 0: Dependency Research + +**This phase is mandatory.** Clone the model library and its key dependencies into a temporary directory and inspect them before writing any integration code. The goal is to produce a dependency audit that identifies every PyInstaller-incompatible pattern, every native data file, and every upstream bug you'll need to work around. + +### 0.1 Clone and Inspect the Model Library + +```bash +# Create a throwaway workspace +mkdir /tmp/engine-research && cd /tmp/engine-research + +# Clone the model library +git clone https://github.com/org/model-library.git +cd model-library +``` + +**Read these files first, in order:** + +1. **`setup.py` / `setup.cfg` / `pyproject.toml`** — Check pinned dependency versions. If the library pins `torch==2.6.0` or `numpy<1.26`, you'll need `--no-deps` installation and manual sub-dependency listing (this is what happened with `chatterbox-tts`). + +2. **`__init__.py` and the main model class** — Trace the import chain. Look for: + - `from_pretrained()` — does it call `huggingface_hub` internally? Does it pass `token=True` (which crashes without a stored HF token)? + - `from_local()` — does it exist? You may need manual `snapshot_download()` + `from_local()` to bypass download bugs. + - Device handling — does it default to CUDA? Does it support MPS? Many libraries crash on MPS with unsupported operators. + +3. **All `import` statements** — Recursively trace what the library imports. You're looking for: + - `inspect.getsource()` anywhere in the chain (search all `.py` files) + - `typeguard` / `@typechecked` decorators (these call `inspect.getsource()` at import time) + - `importlib.metadata.version()` or `pkg_resources.get_distribution()` (need `--copy-metadata`) + - `lazy_loader` (needs `--collect-all` to bundle `.pyi` stubs) + +### 0.2 Scan for PyInstaller-Incompatible Patterns + +Run these searches against the cloned library **and** its transitive dependencies: + +```bash +# inspect.getsource — will crash in frozen binary without --collect-all +grep -r "inspect.getsource\|getsource(" . + +# typeguard / @typechecked — calls inspect.getsource at import time +grep -r "@typechecked\|from typeguard" . + +# importlib.metadata — needs --copy-metadata +grep -r "importlib.metadata\|pkg_resources.get_distribution\|pkg_resources.require" . + +# Data files loaded at runtime — need --collect-all or --collect-data +grep -r "Path(__file__).parent\|os.path.dirname(__file__)\|resources_path\|pkg_resources.resource_filename" . + +# Native library paths — may need env var override in frozen builds +grep -r "/usr/share\|/usr/lib\|/usr/local\|espeak\|phonemize" . + +# torch.load without map_location — will crash on CPU-only builds +grep -r "torch.load(" . | grep -v "map_location" + +# HuggingFace token bugs +grep -r 'token=True\|token=os.getenv' . + +# Float64/Float32 assumptions — librosa returns float64, many models assume float32 +grep -r "torch.from_numpy\|\.double()\|float64" . +``` + +### 0.3 Install and Trace in a Throwaway Venv + +```bash +# Create isolated venv +python -m venv /tmp/engine-venv +source /tmp/engine-venv/bin/activate + +# Install the package (try normally first) +pip install model-package + +# Check if it conflicts with our stack +pip install model-package torch==2.10 transformers==4.57.3 numpy>=1.26 +# If this fails, you need --no-deps: +pip install --no-deps model-package + +# Get the full dependency tree +pip show model-package # Check Requires: field +pip show -f model-package # List all installed files (look for data files) + +# Check for non-PyPI dependencies +pip install model-package 2>&1 | grep -i "no matching distribution" +``` + +### 0.4 Test Model Loading on CPU + +Before writing any integration code, verify the model works on CPU in a plain Python script: + +```python +import torch +# Force CPU to catch map_location bugs early +model = ModelClass.from_pretrained("org/model", device="cpu") + +# Test with a float32 audio array (not float64) +import numpy as np +audio = np.random.randn(16000).astype(np.float32) +output = model.generate("Hello world", audio) +print(f"Output shape: {output.shape}, dtype: {output.dtype}, sample rate: {model.sample_rate}") +``` + +If this crashes, you've found a bug you'll need to monkey-patch. Common ones: +- `RuntimeError: expected scalar type Float but found Double` → needs float32 cast +- `RuntimeError: map_location` → needs `torch.load` patch +- `RuntimeError: Unsupported operator aten::...` → needs MPS skip + +### 0.5 Produce a Dependency Audit + +Before proceeding to Phase 1, write down: + +1. **PyPI vs non-PyPI deps** — which packages need `--find-links`, `git+https://`, or `--no-deps`? +2. **PyInstaller directives needed** — which packages need `--collect-all`, `--copy-metadata`, `--hidden-import`? +3. **Runtime data files** — which packages ship data files (YAML, pretrained weights, phoneme tables, shader libraries) that must be bundled? +4. **Native library paths** — which packages look for data at system paths that won't exist in a frozen binary? +5. **Monkey-patches needed** — `torch.load` map_location, float64→float32 casts, MPS skip, HF token bypass, etc. +6. **Sample rate** — what does the engine output? (24kHz, 44.1kHz, 48kHz) +7. **Model download method** — `from_pretrained()` with library-managed download, or manual `snapshot_download()` + `from_local()`? + +This audit becomes your implementation plan for Phases 1, 4, and 5. + ## Phase 1: Backend Implementation ### 1.1 Create the Backend File @@ -155,54 +277,172 @@ In `app/src/components/ServerSettings/ModelManagement.tsx`: ## Phase 4: Dependencies +Use the dependency audit from Phase 0 to drive this phase. You should already know what packages are needed, which conflict, and which require special installation. + ### 4.1 Python Dependencies -Add to `backend/requirements.txt`. Watch for: +Add to `backend/requirements.txt`. There are three installation patterns, depending on what Phase 0 revealed: -**Pinned dependency conflicts** — If the model package pins old versions, install with `--no-deps`: +**Normal PyPI packages:** +``` +some-model-package>=1.0.0 +``` + +**Pinned dependency conflicts (`--no-deps`)** — If the model package pins old versions of torch/numpy/transformers, install with `--no-deps` and list sub-dependencies manually. This is the pattern used for `chatterbox-tts`: ```bash +# In justfile / CI setup: pip install --no-deps chatterbox-tts + +# In requirements.txt — list each actual sub-dependency: +conformer>=0.3.2 +diffusers>=0.31.0 +omegaconf>=2.3.0 +resemble-perth>=0.0.2 +s3tokenizer>=0.1.6 ``` -Then list sub-dependencies manually in `requirements.txt`. +To identify sub-deps: `pip show chatterbox-tts` → `Requires:` field, then cross-reference against existing `requirements.txt` to avoid duplicates. -**Non-PyPI packages:** -``` -linacodec @ git+https://github.com/user/repo.git +**Non-PyPI packages** — Some libraries only exist on GitHub or require custom indexes: ``` +# Git-only packages (no PyPI release) +linacodec @ git+https://github.com/ysharma3501/LinaCodec.git +Zipvoice @ git+https://github.com/ysharma3501/LuxTTS.git -**Custom package indexes:** -``` +# Custom package indexes (C extensions with platform-specific wheels) --find-links https://k2-fsa.github.io/icefall/piper_phonemize.html +piper-phonemize>=1.2.0 ``` -### 4.2 Identifying Hidden Sub-Dependencies +### 4.2 Dependency Conflict Resolution -1. Install the package normally in a throwaway venv -2. Run `pip show ` to get its `Requires:` list -3. Cross-reference against existing requirements.txt -4. Test that the engine loads and generates +Check for conflicts with the existing stack before adding anything: -## Phase 5: PyInstaller Bundling +```bash +# Our current stack pins (approximate): +# Python 3.12+, torch>=2.10, transformers>=4.57, numpy>=1.26 -This is where most of the pain lives. Common issues: +# Test compatibility +pip install model-package torch==2.10 transformers==4.57.3 numpy>=1.26 -| Issue | Symptom | Fix | -|-------|---------|-----| -| `inspect.getsource()` at import | "could not get source code" | `--collect-all ` | -| Data files (yaml, .pth.tar) | FileNotFoundError at runtime | `--collect-all ` | -| Native data paths (espeak-ng) | Library looks at `/usr/share/...` | Set env var in frozen builds | -| `importlib.metadata` lookups | "No package metadata found" | `--copy-metadata ` | -| Dynamic imports | ModuleNotFoundError | `--hidden-import ` | +# If it fails, check what the package pins: +pip show model-package | grep Requires +# Look at setup.py/pyproject.toml for version constraints +``` -### Testing Frozen Builds +**Known incompatible patterns in the wild:** +- `torch==2.6.0` — many older packages pin this +- `numpy<1.26` — conflicts with Python 3.12+ +- `transformers==4.46.3` — many packages pin old transformers +- `onnxruntime` pinned versions — often conflict with torch -You can't skip this. Models that work in `python -m uvicorn` will break in the PyInstaller binary. +### 4.3 Update Installation Scripts + +Dependencies must be added in multiple places: + +| File | What to add | +|------|------------| +| `backend/requirements.txt` | Package and version constraint | +| `justfile` | `--no-deps` install line if needed (in `setup-python` and `setup-python-release` targets) | +| `.github/workflows/release.yml` | Same `--no-deps` line in CI build steps | +| `Dockerfile` | Same install commands for Docker builds | + +## Phase 5: PyInstaller Bundling (`build_binary.py`) + +This is where most of the pain lives. **The v0.2.3 release was entirely dedicated to fixing bundling issues** — every new engine that shipped in v0.2.1 (LuxTTS, Chatterbox, Chatterbox Turbo) worked in dev but failed in production builds. Don't skip this phase. + +### 5.1 Register Your Engine in `build_binary.py` + +Every new engine needs entries in `backend/build_binary.py`. This file drives PyInstaller and is the single most common source of "works in dev, breaks in prod" bugs. You need to decide which PyInstaller directives your engine's dependencies require: + +| Directive | What It Does | When You Need It | +|-----------|-------------|-----------------| +| `--hidden-import ` | Includes a module PyInstaller can't detect via static analysis | Dynamic imports, lazy imports, plugin architectures | +| `--collect-all ` | Bundles source `.py` files, data files, AND native libraries | Packages that call `inspect.getsource()` at import time (e.g. `inflect` via `typeguard`'s `@typechecked`), or that ship pretrained model files (e.g. `perth` ships `.pth.tar` + `hparams.yaml`) | +| `--collect-data ` | Bundles only data files (not source or native libs) | Packages with YAML configs, vocab files, etc. | +| `--collect-submodules ` | Bundles all submodules | Packages with deep module trees that PyInstaller misses | +| `--copy-metadata ` | Copies `importlib.metadata` info | Packages that call `importlib.metadata.version()` or `pkg_resources.get_distribution()` at runtime. Already required for: `requests`, `transformers`, `huggingface-hub`, `tokenizers`, `safetensors`, `tqdm` | + +**Example: adding hidden imports and collect-all for a new engine:** + +```python +# In build_binary.py, inside the args list: +"--hidden-import", +"backend.backends.your_engine_backend", +"--hidden-import", +"your_engine_package", +"--hidden-import", +"your_engine_package.inference", +"--collect-all", +"some_dependency_that_uses_inspect_getsource", +"--copy-metadata", +"some_dependency_that_checks_its_own_version", +``` + +### 5.2 Lessons from v0.2.3 — Real Failures and Their Fixes + +These are actual production failures from shipping new engines. Every one of these passed `python -m uvicorn` in dev: + +| Engine | Failure | Root Cause | Fix | +|--------|---------|-----------|-----| +| LuxTTS | `"could not get source code"` on import | `inflect` uses `typeguard`'s `@typechecked` which calls `inspect.getsource()` — needs `.py` source files, not just bytecode | `--collect-all inflect` | +| LuxTTS | `espeak-ng-data` not found | `piper_phonemize` C library looks for data at `/usr/share/espeak-ng-data/` which doesn't exist in the bundle | `--collect-all piper_phonemize` + set `ESPEAK_DATA_PATH` env var at runtime (see 5.3) | +| LuxTTS | `inspect.getsource` error in Vocos codec | `linacodec` and `zipvoice` use source introspection | `--collect-all linacodec` + `--collect-all zipvoice` | +| Chatterbox | `FileNotFoundError` for watermark model | `perth` ships pretrained model files (`hparams.yaml`, `.pth.tar`) that PyInstaller doesn't bundle by default | `--collect-all perth` | +| All engines | `importlib.metadata` failures | Frozen binary doesn't include package metadata for `huggingface-hub`, `transformers`, etc. | `--copy-metadata` for each affected package | +| All engines | Download progress bars stuck at 0% | `huggingface_hub` silently disables tqdm progress bars based on logger level in frozen builds — our progress tracker never receives byte updates | Force-enable tqdm's internal counter in `HFProgressTracker` | +| All engines | `NameError: name 'obj' is not defined` on macOS | Python 3.12.0 has a [CPython bug](https://github.com/pyinstaller/pyinstaller/issues/7992) that corrupts bytecode when PyInstaller rewrites code objects | Upgrade to Python 3.12.13+ | +| All engines | `resource_tracker` subprocess crash | `multiprocessing` in frozen binaries needs `freeze_support()` called before anything else | Added to `server.py` entry point | + +### 5.3 Runtime Frozen-Build Handling (`server.py`) + +Some fixes can't live in `build_binary.py` — they need runtime detection. The entry point `backend/server.py` handles these before any heavy imports: + +```python +# 1. freeze_support() — MUST be called before any multiprocessing use +import multiprocessing +multiprocessing.freeze_support() + +# 2. Native data paths — redirect C libraries to bundled data +if getattr(sys, 'frozen', False): + _meipass = getattr(sys, '_MEIPASS', os.path.dirname(sys.executable)) + _espeak_data = os.path.join(_meipass, 'piper_phonemize', 'espeak-ng-data') + if os.path.isdir(_espeak_data): + os.environ.setdefault('ESPEAK_DATA_PATH', _espeak_data) + +# 3. stdout/stderr safety — PyInstaller --noconsole on Windows sets these to None +if not _is_writable(sys.stdout): + sys.stdout = open(os.devnull, 'w') +``` + +If your engine's dependencies include native libraries that look for data at system paths (like espeak-ng does), you'll need to add a similar `os.environ.setdefault()` block here. + +### 5.4 CUDA vs CPU Build Branching + +`build_binary.py` produces two different binaries: + +- **`voicebox-server`** (CPU) — excludes all `nvidia.*` packages to avoid bundling ~3 GB of CUDA DLLs +- **`voicebox-server-cuda`** — includes `torch.cuda` and `torch.backends.cudnn` + +On Windows, if the build environment has CUDA torch installed but you're building the CPU binary, the script temporarily swaps to CPU-only torch and restores CUDA torch afterward. This prevents PyInstaller from accidentally bundling CUDA libraries into the CPU build. + +New engine imports go in the **common section** (not the CUDA or MLX conditional blocks) unless your engine has platform-specific dependencies. + +### 5.5 MLX Conditional Inclusion + +Apple Silicon builds conditionally include MLX hidden imports and `--collect-all mlx` / `--collect-all mlx_audio`. If your engine has an MLX-specific backend variant, add its imports inside the `if is_apple_silicon() and not cuda:` block. + +### 5.6 Testing Frozen Builds + +You can't skip this. Models that work in `python -m uvicorn` will break in the PyInstaller binary. The v0.2.3 release required **three patch releases** (v0.2.1 → v0.2.2 → v0.2.3) to get all engines working in production. 1. Build: `just build` -2. Run and try download + load + generate -3. Check stderr for the actual error -4. Fix, rebuild, repeat +2. Launch the binary directly (not via `python -m`) +3. Test the **full chain**: download → load → generate → progress tracking +4. Check stderr for the actual error (logs go to stderr for Tauri sidecar capture) +5. Fix, rebuild, repeat + +**Common gotcha:** testing only generation with a pre-cached model from your dev install. Always test with a clean model cache to verify downloads work too. ## Phase 6: Common Upstream Workarounds @@ -254,4 +494,76 @@ Based on the current model landscape, these are candidates for future integratio | **MOSS-TTS** | Multilingual | Medium | Text-to-voice design, multi-speaker dialogue | Needs vetting | | **Pocket TTS** | English | ~100M | CPU-first, >1× realtime | Needs vetting | -The multi-engine architecture is now in place, making new model integration straightforward (~1 day for a well-documented model with a PyPI package). +## Implementation Checklist + +Use this as a gate between phases. Do not proceed to the next phase until every item in the current phase is checked. + +### Phase 0: Dependency Research +- [ ] Cloned model library source into a temp directory +- [ ] Read `setup.py` / `pyproject.toml` — noted pinned dependency versions +- [ ] Traced all imports from the model class through to leaf dependencies +- [ ] Searched for `inspect.getsource`, `@typechecked`, `typeguard` in the full dependency tree +- [ ] Searched for `importlib.metadata`, `pkg_resources.get_distribution` in the dependency tree +- [ ] Searched for `Path(__file__).parent`, `os.path.dirname(__file__)`, hardcoded system paths +- [ ] Searched for `torch.load` calls missing `map_location` +- [ ] Searched for `torch.from_numpy` without `.float()` cast +- [ ] Searched for `token=True` or `token=os.getenv("HF_TOKEN")` in HuggingFace calls +- [ ] Tested model loading and generation on CPU in a throwaway venv +- [ ] Tested with a clean HuggingFace cache (no pre-downloaded models) +- [ ] Produced a written dependency audit documenting all findings + +### Phase 1: Backend Implementation +- [ ] Created `backend/backends/_backend.py` implementing `TTSBackend` protocol +- [ ] Chose voice prompt pattern (pre-computed tensors vs deferred file paths) +- [ ] Implemented all monkey-patches identified in Phase 0 +- [ ] Used `get_torch_device()` from `backends/base.py` for device selection +- [ ] Used `model_load_progress()` from `backends/base.py` for download/load tracking +- [ ] Tested: model downloads correctly +- [ ] Tested: model loads on CPU +- [ ] Tested: generation produces valid audio +- [ ] Tested: voice cloning from reference audio works +- [ ] Registered `ModelConfig` in `backends/__init__.py` +- [ ] Added to `TTS_ENGINES` dict +- [ ] Added factory branch in `get_tts_backend_for_engine()` +- [ ] Updated engine regex in `backend/models.py` + +### Phase 2–3: Route, Service, and Frontend +- [ ] Confirmed zero changes needed in routes/services (or documented why custom behavior is needed) +- [ ] Added engine to TypeScript union type in `app/src/lib/api/types.ts` +- [ ] Added language map entry in `app/src/lib/constants/languages.ts` +- [ ] Added to `ENGINE_OPTIONS` and `ENGINE_DESCRIPTIONS` in `EngineModelSelector.tsx` +- [ ] Added to Zod schema and model-name mapping in `useGenerationForm.ts` +- [ ] Added description in `ModelManagement.tsx` + +### Phase 4: Dependencies +- [ ] Added packages to `backend/requirements.txt` +- [ ] If `--no-deps` needed: listed sub-dependencies explicitly +- [ ] If git-only packages: added `@ git+https://...` entries +- [ ] If custom index needed: added `--find-links` line +- [ ] Updated `justfile` setup targets +- [ ] Updated `.github/workflows/release.yml` build steps +- [ ] Updated `Dockerfile` if applicable +- [ ] Verified `pip install` succeeds in a clean venv with existing requirements + +### Phase 5: PyInstaller Bundling +- [ ] Added `--hidden-import` entries in `build_binary.py` for: + - [ ] `backend.backends._backend` + - [ ] The model package and its key submodules +- [ ] Added `--collect-all` for any packages that: + - [ ] Use `inspect.getsource()` / `@typechecked` + - [ ] Ship pretrained model data files (`.pth.tar`, `.yaml`, etc.) + - [ ] Ship native data files (phoneme tables, shader libraries, etc.) +- [ ] Added `--copy-metadata` for any packages that use `importlib.metadata` +- [ ] If engine has native data paths: added `os.environ.setdefault()` in `server.py` +- [ ] Built frozen binary with `just build` +- [ ] Tested in frozen binary with **clean model cache** (not pre-cached from dev): + - [ ] Model download works with real-time progress + - [ ] Model loading works + - [ ] Generation produces valid audio + - [ ] No errors in stderr logs + +### Phase 6: Final Verification +- [ ] Engine works in dev mode (`just dev`) +- [ ] Engine works in frozen binary (`just build` → run binary directly) +- [ ] Tested on target platform (macOS for MLX, Windows/Linux for CUDA) +- [ ] No regressions in existing engines