Compare commits

..
Author SHA1 Message Date
James Pine 707046237c fix: complete Intel XPU support — device-aware seeding, GPU status reporting, and setup detection
Address CodeRabbit review feedback and user-reported GPU acceleration failure:

- Use shared manual_seed() in chatterbox, chatterbox_turbo, and luxtts
  backends so XPU (and future accelerators) get proper device seeding
- Add XPU branch to _get_gpu_status() so startup log reports Intel Arc
  GPUs instead of 'None (CPU only)'
- Add XPU VRAM reporting and correct backend_variant fallback in the
  /health endpoint
- Switch justfile GPU detection from Get-WmiObject to Get-CimInstance,
  simplify the Arc regex to match 'Arc' (not 'Intel.*Arc'), log
  detected GPUs, and print manual install instructions on miss

Resolves the root cause where IPEX was silently not installed due to
WMI detection failure, causing CPU-only fallback on Intel Arc systems.
2026-03-18 17:01:12 -07:00
James Pine 83ebababe7 feat: add Intel Arc (XPU) GPU support across all backends
Auto-detect Intel Arc GPUs during Windows setup and install PyTorch
with XPU support + intel-extension-for-pytorch. Enable allow_xpu=True
on all TTS backends (Chatterbox, Chatterbox Turbo, Hume TADA, LuxTTS)
that previously only supported CUDA. Add shared empty_device_cache()
and manual_seed() helpers in base.py to handle XPU memory management
and reproducible seeding alongside CUDA.
2026-03-18 11:24:51 -07:00
Jamie PineandGitHub ffc1b54812 Merge pull request #316 from jamiepine/fix/cuda-cu128-upgrade
Upgrade CUDA backend from cu126 to cu128, fix GPU settings UI
2026-03-18 07:58:12 -07:00
James Pine fc5ed1ff40 upgrade CUDA backend from cu126 to cu128 and fix GPU settings UI
Upgrade CUDA toolkit from 12.6 (cu126) to 12.8 (cu128) for proper
RTX 50-series (Blackwell) GPU support. Users with RTX 5070/5080/5090
were reporting CUDA detection failures with cu126.

Also fix the GPU Acceleration settings panel where the 'Switch to CPU
Backend' button was unreachable — it was inside a conditional block
that required !isCurrentlyCuda, making it impossible to switch back
to CPU once running on CUDA.

Closes #315
2026-03-18 07:47:39 -07:00
Jamie PineandGitHub c9f38dd496 Merge pull request #305 from jamiepine/fix/qwen-tts-pyinstaller-source-files
fix: bundle qwen_tts source files in PyInstaller build
2026-03-17 09:24:42 -07:00
James Pine 58b19e4e9f fix: bundle qwen_tts source files in PyInstaller build
Replace --collect-submodules + --collect-data with --collect-all for
qwen_tts. The qwen_tts runtime expects physical .py source files
(e.g. modeling_qwen3_tts.py) under _MEIPASS, which only --collect-all
provides. This is the same pattern used for inflect/typeguard.

Fixes #212
2026-03-17 09:23:30 -07:00
Jamie PineandGitHub 0245c31dba Merge pull request #298 from jamiepine/feat/cuda-libs-addon
feat: split CUDA backend into independently versioned server + libs archives
2026-03-17 09:17:31 -07:00
Jamie Pine 81864e831a fix: always clean up temp archive on failure and fix justfile data dir path
- Wrap download/verify/extract in try/finally so .download-*.tmp is
  always deleted, even on mid-download or extraction failures
- Fix justfile build-server-cuda to use sh.voicebox.app (production path)
2026-03-17 09:15:23 -07:00
Jamie Pine 7bd72ea9f7 Bump version: 0.3.0 → 0.3.1 2026-03-17 07:50:12 -07:00
Jamie Pine f96eae2567 fix: address PR review feedback from CodeRabbit
- Upgrade softprops/action-gh-release@v1 to @v2 (Node 16 EOL)
- Fail-fast on checksum fetch failure instead of extracting unverified archives
- Abort packaging if no NVIDIA files found (prevents empty cuda-libs archive)
- Fix nvidia/ path detection bug (list membership vs substring check)
- Fix justfile Copy-Item nesting (copy contents, not the directory itself)
2026-03-17 06:53:54 -07:00
Jamie Pine 7d53699c96 fix: update build-server-cuda to copy onedir folder instead of single exe 2026-03-17 06:12:30 -07:00
Jamie Pine 28e91ce2c1 chore: add .spec and nul to .gitignore 2026-03-17 04:58:13 -07:00
Jamie Pine 88be097b62 fix: update package_cuda.py for PyInstaller 6.18 layout and remove split_binary.py
- Fix is_nvidia_file() to match NVIDIA DLLs in _internal/torch/lib/
  (PyInstaller 6.18 + torch 2.10 no longer uses nvidia/ subdirectories)
- Remove deprecated split_binary.py (both archives are under 2GB)
- Update torch_compat range to >=2.6.0,<2.11.0
- Update build docs for new dual-archive packaging flow
2026-03-17 04:57:05 -07:00
James Pine 564d787927 feat: split CUDA backend into independently versioned server + libs archives
Switch CUDA builds from PyInstaller --onefile to --onedir and split the
output into two separately versioned archives:

1. Server core (~200-400MB) — versioned with the app, redownloaded on
   every app update
2. CUDA libs (~2GB) — versioned independently (cu126-v1), only
   redownloaded when the CUDA toolkit or torch version changes

This eliminates the ~2.4GB full redownload on every version bump.
After initial setup, most app updates only need ~200-400MB.

Closes #297
2026-03-17 04:04:17 -07:00
James Pine 2c1ee94891 docs: add TADA learnings to TTS engine guide and CUDA libs addon plan
Enrich tts-engines.mdx with patterns discovered during TADA integration:
- Phase 0.2: new greps for @torch.jit.script, torchaudio.load, gated repos
- Phase 3.4: model naming inconsistency warning
- Phase 5.2: TADA shim failure added to lessons table
- Phase 6: four new workaround sections (gated repos, torchcodec,
  torch.jit.script, toxic dependency shim pattern)
- Checklist: four new items matching the new scan patterns
- Remove TADA from upcoming engines (now shipped)

Add CUDA_LIBS_ADDON.md exploring --onedir split to avoid 2.4GB
redownloads on every version bump.
2026-03-17 03:53:40 -07:00
Jamie PineandGitHub e789c937ad Merge pull request #296 from jamiepine/feat/add-tada-tts-engine
Add HumeAI TADA TTS engine (1B English + 3B Multilingual)
2026-03-17 03:47:47 -07:00
James Pine 273483ffcf fix TorchScript error in frozen builds and update docs for TADA
Remove @torch.jit.script from the DAC shim's snake() function —
TorchScript calls inspect.getsource() which fails in PyInstaller
binaries (no .py source files).

Update all user-facing docs: 4 → 5 TTS engines, add TADA row to
every engine comparison table, mark TADA as Shipped in the upcoming
engines list, update architecture diagrams and tech stack tables.
2026-03-17 03:28:58 -07:00
James Pine 5774a168a9 fix TADA 3B model name: tada-3b -> tada-3b-ml 2026-03-17 03:17:53 -07:00
James Pine 6bf40bd2d0 fix tokenizer patch corrupting AutoTokenizer for other engines
Replace the monkey-patch on AutoTokenizer.from_pretrained (which broke
the classmethod descriptor and caused 'Tokenizer not loaded' errors
when loading Qwen after TADA) with two targeted config patches:
- Set AlignerConfig.tokenizer_name to the local ungated tokenizer path
- Pre-load TadaConfig, inject tokenizer_name, pass config= to from_pretrained

No global state is modified; other engines are unaffected.
2026-03-17 03:15:57 -07:00
James Pine 12cda2e090 fix torchcodec error by using soundfile instead of torchaudio.load
torchaudio 2.10+ switched its default audio loading backend to
torchcodec, which isn't installed. Replace torchaudio.load() with
soundfile.read() in create_voice_prompt(). TADA's internal use of
torchaudio.functional.resample() is unaffected (pure PyTorch math,
no torchcodec dependency).
2026-03-17 02:25:05 -07:00
James Pine 7a90290a76 fix gated Llama tokenizer error by redirecting to ungated mirror
TADA hardcodes 'meta-llama/Llama-3.2-1B' as its tokenizer source in
both the Aligner and TadaForCausalLM.from_pretrained(). That repo is
gated and requires accepting Meta's license on HuggingFace.

Monkey-patch AutoTokenizer.from_pretrained during model loading to
redirect Llama tokenizer requests to 'unsloth/Llama-3.2-1B', an
ungated mirror with identical tokenizer files. The patch is scoped
to model loading only and restored immediately after.
2026-03-17 02:22:26 -07:00
James Pine b02ce8e2f3 replace descript-audio-codec with lightweight DAC shim
The real descript-audio-codec package pulls in descript-audiotools,
which transitively requires onnx, tensorboard, protobuf, matplotlib,
pystoi, and other heavy dependencies. onnx fails to build from source
on macOS due to CMake version incompatibility.

TADA only uses Snake1d (a 7-line PyTorch module) from DAC. This commit
adds a shim in backend/utils/dac_shim.py that registers fake dac.*
modules in sys.modules with just the Snake1d class, completely
eliminating the DAC/audiotools dependency chain.
2026-03-17 02:16:33 -07:00
James Pine 4e7772a21d add HumeAI TADA TTS engine (1B English + 3B Multilingual)
Integrates HumeAI's TADA (Text-Acoustic Dual Alignment) speech-language
model as a new TTS engine. TADA uses a novel 1:1 token-audio alignment
that produces coherent speech over long sequences (700s+).

Two model variants:
- tada-1b: English-only, ~4GB, built on Llama 3.2 1B
- tada-3b-ml: 10 languages, ~8GB, built on Llama 3.2 3B

Backend uses the Encoder for voice prompt encoding with caching, and
TadaForCausalLM with flow-matching diffusion for generation. Supports
bf16 inference on CUDA, forces CPU on macOS (MPS compatibility).

Installed with --no-deps due to torch>=2.7 pin conflict; descript-audio-codec
and torchaudio added as explicit sub-dependencies.
2026-03-17 01:55:15 -07:00
James Pine 51fb320b8c readme 2026-03-17 01:24:48 -07:00
James Pine 8ac202aa58 docs for adding new engines 2026-03-17 01:13:30 -07:00
Jamie Pine ac68052945 create stub resource files when actool output is missing
Older Xcode versions don't produce Assets.car from .icon assets.
Fall back to empty stubs for all platforms so the bundler succeeds.
2026-03-17 01:07:54 -07:00
Jamie Pine e601fd2ca4 generate icon assets at build time instead of tracking them
build.rs now generates voicebox.icns via sips + iconutil alongside the
existing actool Assets.car compilation. On non-macOS, empty stub files
are created so Tauri's resource bundler doesn't fail on missing paths.
2026-03-17 00:51:12 -07:00
Jamie Pine 7b25e0ba0b Bump version: 0.2.3 → 0.3.0 2026-03-17 00:25:56 -07:00
Jamie PineandGitHub a6817cd082 Merge pull request #295 from jamiepine/fix/misc-bugs
fix: batch of bug fixes from issue tracker
2026-03-17 00:08:17 -07:00
Jamie Pine df50b8a925 add --force-reinstall --no-deps to torchaudio CUDA install 2026-03-17 00:07:45 -07:00
Jamie Pine a672ac5279 remove voicebox.icns from tracking and add to gitignore 2026-03-17 00:07:19 -07:00
Jamie Pine d35e6f0cc5 fix sample upload blocking the event loop and causing server timeouts
Move audio validation and saving to thread pool so librosa/ffmpeg decoding
doesn't block the async event loop. Combine validate + load into a single
pass to avoid decoding the file twice. Add 50 MB upload limit and chunked
reads to prevent unbounded memory allocation.

Closes #278
2026-03-16 23:29:18 -07:00
Jamie Pine b1069b4521 upgrade CUDA backend build from cu121 to cu126
cu121 only ships kernels up to SM 9.0 (Ada Lovelace). RTX 50-series
(Blackwell, SM 12.0) and RTX 6000 Pro need cu126 which includes SM 12.0
support while remaining backward compatible with older GPUs.

Closes #289
2026-03-16 23:23:55 -07:00
Jamie Pine f9e1aa153d handle client disconnects in SSE and streaming endpoints
Wrap SSE generators with BrokenPipeError/ConnectionResetError handling
so client disconnects during generation status polling, download progress,
or audio streaming don't produce unhandled Errno 32 errors.

Closes #248
2026-03-16 23:17:09 -07:00
Jamie Pine 01800f196f upgrade pip before installing deps in Docker build
Fixes hash mismatch when pip resolves Qwen3-TTS transitive deps.

Closes #286
2026-03-16 23:13:50 -07:00
Jamie Pine 606da1c894 fix generation list not updating on completion
Use refetchQueries instead of invalidateQueries for more reliable history
refresh. Add history refetch to SSE onerror handler so dropped connections
don't leave the list stale. Reset page to 0 in HistoryTable when a pending
generation completes.

Closes #231
2026-03-16 22:54:11 -07:00
Jamie Pine 664178f0cf fix error detail serialization producing [object Object] in error messages
Closes #290
2026-03-16 22:47:01 -07:00
Jamie Pine f1541701fb add model selection and expanded language support to /transcribe endpoint
Closes #233
2026-03-16 22:44:28 -07:00
Jamie PineandGitHub a2adc3b506 Merge pull request #293 from jamiepine/fix/audio-player-freeze
Fix audio player freezing and improve UX
2026-03-16 22:28:39 -07:00
Jamie PineandGitHub 15ba824472 Merge pull request #294 from jamiepine/feat/settings-overhaul
Settings overhaul: routed sub-tabs, server logs, changelog, about page
2026-03-16 13:07:39 -07:00
Jamie Pine 2ad4776a76 fix review feedback: restart race, listener cleanup, stable keys, accessibility 2026-03-16 13:06:57 -07:00
Jamie Pine a8469b39f1 fix about license 2026-03-16 12:31:33 -07:00
Jamie Pine 7dd70a52e4 fix audio player freezing and improve UX
Switch WaveSurfer from MediaElement to WebAudio backend to prevent
WKWebView deadlocks that were freezing the entire Tauri app during
audio playback.

Reuse a single WaveSurfer instance across track changes instead of
destroying and recreating on every URL change, which was exhausting
the browser's AudioContext pool.

Other improvements:
- spacebar play/pause with capture phase to prevent history item activation
- drag-to-seek on waveform with silent scrub to avoid WebAudio popping
- slider always mounted to prevent layout shift during track transitions
- play button fills icon, accent bg when playing, loop button accent bg when active
- fix AudioBars animation getting stuck by keying on mode
- remove focus ring on history items
- sync slider position on pause and during seek
- remove title text from player bar
- thicker cursor (3px)
2026-03-16 12:29:10 -07:00
James Pine 1526f2de26 about page + generation folder open and display 2026-03-16 10:14:47 -07:00
James Pine 2c63dfff25 overhaul settings: split into routed sub-tabs, add server logs, changelog, reusable setting components
- Rename Server tab to Settings with horizontal sub-tab navigation (General, Generation, GPU, Logs, Changelog)
- All sub-tabs are proper routes under /settings/* with /server redirect for backwards compat
- General: connection settings, link cards (docs + discord), API reference card, app updates
- Generation: auto-chunking, crossfade, normalize, autoplay as SettingRow components
- GPU: info card with platform-aware icons (Apple logo for MPS), CUDA management, explainer text
- Logs: real-time server log viewer piped from Tauri sidecar via event system (Tauri-only)
- Changelog: parsed from CHANGELOG.md at build time via Vite virtual module plugin
- New reusable SettingRow/SettingSection components for consistent settings layout
- New Toggle (switch) UI component replacing checkboxes in settings
- Toast viewport now offsets when audio player is open
- Sidebar stays active on settings sub-routes (fuzzy matching)
2026-03-16 09:31:14 -07:00
James Pine e0a798dc0d add release skills, backfill changelog, wire CI to use changelog for GitHub releases
- Backfill CHANGELOG.md from all 17 GitHub releases (was stale at v0.1.0)
- Add draft-release-notes and release-bump agent skills
- Extract release notes from CHANGELOG.md in release CI instead of hardcoded placeholder
- Remove stale PATCH_NOTES.md, mlx-test/, move PROJECT_STATUS to docs/notes
- Reorganize API reference docs from unknown/ to named groups
- Update openapi.json
2026-03-16 06:18:46 -07:00
James Pine 5933cba8e9 add release skills and backfill changelog from GitHub releases
- Backfill CHANGELOG.md from all 17 GitHub releases (was stale at v0.1.0)
- Add draft-release-notes and release-bump agent skills
- Remove stale PATCH_NOTES.md, mlx-test/, move PROJECT_STATUS to docs/notes
- Minor voicebox-server.spec cleanup
2026-03-16 06:15:04 -07:00
James Pine 1b2d492398 add /og preview page and OG image metadata 2026-03-16 05:51:04 -07:00
James Pine faa825290f docs links 2026-03-16 05:33:33 -07:00
James Pine 4a8a9eac14 fix: docker frontend + docs cleanup 2026-03-16 05:28:05 -07:00
Jamie PineandGitHub 3e4d9ff641 Merge pull request #288 from jamiepine/better-docs
Better docs
2026-03-16 05:12:09 -07:00
James Pine 192979a762 docs 2026-03-16 05:11:21 -07:00
James Pine e16cc42d53 enable Edit on GitHub and last updated on all doc pages 2026-03-16 04:45:40 -07:00
James Pine f10e965003 rewrite docs root page, add screenshot 2026-03-16 04:12:11 -07:00
James Pine a8968d4081 rewrite docs introduction based on README content 2026-03-16 04:09:50 -07:00
James Pine 7c4afbe4df expand sidebar groups by default, remove stale plans reference 2026-03-16 04:08:46 -07:00
James Pine a180fcc56f redirect root to /docs 2026-03-16 04:06:38 -07:00
James Pine 1860b8dc92 remove plans/ from docs site 2026-03-16 04:05:14 -07:00
James Pine 1597937535 Merge branch 'main' into better-docs
# Conflicts:
#	backend/main.py
#	docs/content/docs/plans/ADDING_TTS_ENGINES.md
#	docs/content/docs/plans/CUDA_BACKEND_SWAP.md
#	docs/content/docs/plans/CUDA_BACKEND_SWAP_FINAL.md
#	docs/content/docs/plans/EXTERNAL_PROVIDERS.md
#	docs/content/docs/plans/MLX_AUDIO.md
#	docs/content/docs/plans/PROJECT_STATUS.md
2026-03-16 04:01:08 -07:00
Jamie PineandGitHub ac41a89359 Merge pull request #285 from jamiepine/backend-refactor
Backend refactor: modular architecture, style guide, tooling
2026-03-16 03:51:10 -07:00
Jamie Pine 2e6efa00a2 Refactor documentation structure and dependencies for migration to Fumadocs
- Updated `.gitignore` to include new build and generated content directories.
- Removed outdated Mintlify configuration files and documentation.
- Introduced new `MIGRATION.md` to outline the transition from Mintlify to Fumadocs.
- Added `mdx-components.tsx` for MDX component configuration and compatibility.
- Updated `package.json` and `next.config.mjs` for new dependencies and Next.js configuration.
- Created `source.config.ts` for content source configuration.
- Added OpenAPI specification in `openapi.json` for API documentation.
- Removed legacy files and adjusted project structure to align with Fumadocs conventions.
2026-02-02 23:29:35 -08:00
Jamie Pine 788a04f265 Merge branch 'main' into better-docs 2026-02-02 23:18:06 -08:00
Jamie Pine 5cb54ee03c Update API documentation and enhance server configuration
- Added server configurations for local and production environments in `main.py`.
- Removed outdated authentication and generation API documentation files.
- Updated documentation structure to reflect the removal of deprecated API endpoints.
- Adjusted links in the quick start and developer setup documentation to point to the new API reference.
- Enhanced global CSS styles for improved theming support.
2026-01-31 01:45:42 -08:00
Jamie Pine 0922845101 disable cuda for 0.1.12 2026-01-31 01:44:34 -08:00
Jamie Pine 64dd29d35a Add initial setup for Fumadocs documentation migration
- Created new directory structure for documentation under `/docs2`.
- Added `.gitignore` to exclude build artifacts and dependencies.
- Introduced `package.json`, `next.config.mjs`, and `postcss.config.mjs` for project configuration.
- Implemented MDX components in `mdx-components.tsx` for rendering documentation.
- Migrated existing documentation content and created new files for auto-updater and other features.
- Established compatibility layer for Mintlify components in `mintlify-compat.tsx`.
- Set up OpenAPI documentation in `openapi.json`.
- Updated README and migration guide to reflect new structure and usage instructions.
- Ensured all components and pages are ready for development and deployment with Fumadocs.
2026-01-30 23:32:45 -08:00
212 changed files with 10285 additions and 5665 deletions
+120
View File
@@ -0,0 +1,120 @@
---
name: add-tts-engine
description: Use this skill to add a new TTS engine to Voicebox. It walks through dependency research, backend implementation, frontend wiring, PyInstaller bundling, and frozen-build testing. Always start with Phase 0 (dependency audit) before writing any code.
---
# Add TTS Engine
## Goal
Integrate a new text-to-speech engine into Voicebox end-to-end: dependency research, backend protocol implementation, frontend UI wiring, PyInstaller bundling, and frozen-build verification. The user should only need to test the final build locally.
## Reference Doc
The full phased guide lives at `docs/content/docs/developer/tts-engines.mdx`. **Read this file in its entirety before starting.** It contains:
- Phase 0: Dependency research (mandatory before writing code)
- Phase 1: Backend implementation (`TTSBackend` protocol)
- Phase 2: Route and service integration (usually zero changes)
- Phase 3: Frontend integration (5 files)
- Phase 4: Dependencies (`requirements.txt`, justfile, CI, Docker)
- Phase 5: PyInstaller bundling (`build_binary.py` + `server.py`)
- Phase 6: Common upstream workarounds
- Implementation checklist (gate between phases)
## Workflow
### 1. Read the guide
```bash
# Read the full TTS engines doc
cat docs/content/docs/developer/tts-engines.mdx
```
Internalize all phases, especially Phase 0 and Phase 5. The v0.2.3 release was three patch releases because Phase 0 was skipped.
### 2. Dependency research (Phase 0)
Clone the model library into a temporary directory and audit it. Do NOT skip this.
```bash
mkdir /tmp/engine-research && cd /tmp/engine-research
git clone <model-library-url>
```
Run the grep searches from Phase 0.2 in the guide against the cloned source and its transitive dependencies. Produce a written dependency audit covering:
1. PyPI vs non-PyPI packages
2. PyInstaller directives needed (`--collect-all`, `--copy-metadata`, `--hidden-import`)
3. Runtime data files that must be bundled
4. Native library paths that need env var overrides in frozen builds
5. Monkey-patches needed (`torch.load`, float64, MPS, HF token)
6. Sample rate
7. Model download method (`from_pretrained` vs `snapshot_download` + `from_local`)
Test model loading and generation on CPU in the throwaway venv before proceeding.
### 3. Implement (Phases 1–4)
Follow the guide's phases in order. Key files to modify:
**Backend (Phase 1):**
- Create `backend/backends/<engine>_backend.py`
- Register in `backend/backends/__init__.py` (ModelConfig + TTS_ENGINES + factory)
- Update regex in `backend/models.py`
**Frontend (Phase 3):**
- `app/src/lib/api/types.ts` — engine union type
- `app/src/lib/constants/languages.ts` — ENGINE_LANGUAGES
- `app/src/components/Generation/EngineModelSelector.tsx` — ENGINE_OPTIONS, ENGINE_DESCRIPTIONS
- `app/src/lib/hooks/useGenerationForm.ts` — Zod schema, model-name mapping
- `app/src/components/ServerSettings/ModelManagement.tsx` — MODEL_DESCRIPTIONS
**Dependencies (Phase 4):**
- `backend/requirements.txt`
- `justfile` (setup-python, setup-python-release targets)
- `.github/workflows/release.yml`
- `Dockerfile` (if applicable)
### 4. PyInstaller bundling (Phase 5)
Register the engine in `backend/build_binary.py`:
- `--hidden-import` for the backend module and model package
- `--collect-all` for packages using `inspect.getsource`, shipping data files, or native libraries
- `--copy-metadata` for packages using `importlib.metadata`
If the engine has native data paths, add `os.environ.setdefault()` in `backend/server.py` inside the `if getattr(sys, 'frozen', False):` block.
### 5. Verify in dev mode
```bash
just dev
```
Test the full chain: model download → load → generate → voice cloning.
### 6. Use the checklist
Walk through the Implementation Checklist at the bottom of `tts-engines.mdx`. Every item must be checked before handing the build to the user.
## Key Lessons (from v0.2.3)
These are the most common failure modes. Phase 0 research catches all of them:
| Pattern | Symptom in Frozen Build | Fix |
|---------|------------------------|-----|
| `@typechecked` / `inspect.getsource()` | "could not get source code" | `--collect-all <package>` |
| Package ships pretrained model files | `FileNotFoundError` for `.pth.tar`, `.yaml` | `--collect-all <package>` |
| C library with hardcoded system paths | `FileNotFoundError` for `/usr/share/...` | `--collect-all` + env var in `server.py` |
| `importlib.metadata.version()` | "No package metadata found" | `--copy-metadata <package>` |
| `torch.load` without `map_location` | CUDA device not available on CPU build | Monkey-patch `torch.load` |
| `torch.from_numpy` on float64 data | dtype mismatch RuntimeError | Cast to `.float()` |
| `token=True` in HF download calls | Auth failure without stored HF token | Use `snapshot_download(token=None)` + `from_local()` |
## Notes
- The route and service layers have zero per-engine dispatch points. `main.py` requires zero changes.
- The model config registry in `backends/__init__.py` handles all dispatch automatically.
- Use `get_torch_device()` and `model_load_progress()` from `backends/base.py` — don't reimplement device detection or progress tracking.
- Always test with a **clean HuggingFace cache** (no pre-downloaded models from dev).
- Do NOT push or create a release. Hand the build to the user for local testing.
@@ -0,0 +1,94 @@
---
name: draft-release-notes
description: Use this skill to draft or update the [Unreleased] section of CHANGELOG.md from the actual changes since the last tag. Run this at any point during development to keep a working copy of the release narrative. Does NOT bump versions or create tags.
---
# Draft Release Notes
## Goal
Update the `[Unreleased]` section at the top of `CHANGELOG.md` with a narrative release story based on the real changes since the last tag. This is a **non-destructive working copy** — run it as many times as you want during development.
## Workflow
1. **Identify the last release tag and gather changes.**
```bash
LAST_TAG=$(git tag --list "v*" --sort=-v:refname | head -n 1)
echo "Last tag: $LAST_TAG"
```
Then collect raw material from three sources:
a. **Commit log since last tag:**
```bash
git log --oneline "$LAST_TAG"..HEAD
```
b. **GitHub-generated release notes preview** (PR titles, new contributors):
```bash
gh api repos/:owner/:repo/releases/generate-notes \
-f tag_name="vNEXT" \
-f target_commitish="$(git rev-parse HEAD)" \
-f previous_tag_name="$LAST_TAG" \
--jq '.body'
```
c. **Diff stat for theme analysis:**
```bash
git diff --stat "$LAST_TAG"..HEAD
```
2. **Draft the release narrative.**
Write markdown for the `[Unreleased]` section following the format below. Do not include the `## [Unreleased]` heading itself — just the body content.
3. **Update CHANGELOG.md.**
Replace everything between `## [Unreleased]` and the next `## [` heading with the new draft. Preserve the HTML comment header and all existing release sections below.
The `[Unreleased]` section must always exist and always be the first section after the header comments.
4. **Do NOT commit, tag, or bump versions.** Just leave the file modified in the working tree.
## Release Story Format
Structure the `[Unreleased]` section like this:
```markdown
## [Unreleased]
<One strong opening paragraph: what this release is about and why it matters.
Tie it to concrete shipped changes. No vague hype.>
<One paragraph on major technical shifts, if applicable.>
### <Feature/Theme Group>
- Bullet points with specifics
- Reference PRs where available: ([#123](https://github.com/jamiepine/voicebox/pull/123))
### <Another Group>
- ...
### Bug Fixes
- ...
```
### Style Guidelines
- **Factual and specific.** Every claim should trace to a real commit or PR.
- **Narrative over list.** Lead with paragraphs that tell the story, then support with bullets.
- **Group by theme, not by commit.** Cluster related changes under descriptive headings.
- **Reference PRs** where they exist, but don't fabricate them.
- **Skip trivial chores** (typo fixes, CI tweaks) unless they're the bulk of the release.
- **Match the voice of existing releases** — look at the v0.2.1 and v0.2.3 entries in CHANGELOG.md for tone reference.
## When There Are No Changes
If `git log "$LAST_TAG"..HEAD` is empty, leave the `[Unreleased]` section empty (just the heading) and tell the user there's nothing to draft.
## Notes
- This skill only touches the `[Unreleased]` section. It never modifies stamped release sections.
- The agent can be asked to run this skill at any point — mid-feature, before a PR, or right before cutting a release.
- The `release-bump` skill depends on this draft being up to date before it finalizes.
+124
View File
@@ -0,0 +1,124 @@
---
name: release-bump
description: Use this skill to finalize a release. It stamps the [Unreleased] changelog section with a version and date, runs bumpversion to update all version files, and creates the release commit and tag. Only run this when you're ready to ship.
---
# Release Bump
## Goal
Finalize the changelog draft, bump the version across all tracked files, and create a tagged release commit. After this skill runs, the repo has a clean release commit and tag ready to push.
## Prerequisites
- `gh` CLI installed and authenticated (`gh auth status`).
- `bumpversion` installed (`pip install bumpversion` or available in the project venv).
- The `[Unreleased]` section of `CHANGELOG.md` should already contain the release narrative. If it's empty or stale, run the `draft-release-notes` skill first.
## Workflow
1. **Verify the working tree is clean** (except `CHANGELOG.md` which may have the draft).
```bash
git status --porcelain
```
Only `CHANGELOG.md` (and optionally `.agents/` files) should be modified. If there are other uncommitted changes, stop and ask the user to commit or stash them first.
2. **Determine the bump level.**
Ask the user if not specified: `patch`, `minor`, or `major`. Check the current version:
```bash
grep '^current_version' .bumpversion.cfg
```
3. **Stamp the changelog.**
Read the current `[Unreleased]` content from `CHANGELOG.md`. Compute the new version (based on bump level and current version). Then:
a. Replace the `## [Unreleased]` section body with an empty placeholder.
b. Insert a new stamped section immediately after `## [Unreleased]`:
```markdown
## [Unreleased]
## [X.Y.Z] - YYYY-MM-DD
<the content that was in [Unreleased]>
```
c. Update the reference links at the bottom of the file:
- Change the `[Unreleased]` link to compare against the new tag
- Add a new link for the new version
```markdown
[Unreleased]: https://github.com/jamiepine/voicebox/compare/vX.Y.Z...HEAD
[X.Y.Z]: https://github.com/jamiepine/voicebox/compare/vPREVIOUS...vX.Y.Z
```
4. **Stage the changelog.**
```bash
git add CHANGELOG.md
```
5. **Run bumpversion.**
```bash
bumpversion --allow-dirty <patch|minor|major>
```
The `--allow-dirty` flag is needed because `CHANGELOG.md` is already staged. bumpversion will:
- Update version strings in all tracked files (see `.bumpversion.cfg`)
- Create a commit with message `Bump version: X.Y.Z -> A.B.C`
- Create a tag `vA.B.C`
The staged `CHANGELOG.md` will be included in this commit automatically.
6. **Verify results.**
```bash
git show --name-only --stat HEAD
git tag --list "v*" --sort=-v:refname | head -n 5
```
Confirm the commit contains:
- `CHANGELOG.md`
- `.bumpversion.cfg`
- `tauri/src-tauri/tauri.conf.json`
- `tauri/src-tauri/Cargo.toml`
- `package.json`
- `app/package.json`
- `tauri/package.json`
- `landing/package.json`
- `web/package.json`
- `backend/__init__.py`
Confirm the new tag exists.
7. **Do NOT push** unless the user explicitly asks. Report the tag name and suggest:
```
Ready to push. When you're ready:
git push origin main --follow-tags
```
## Version Calculation Reference
Given current version `X.Y.Z`:
- `patch` -> `X.Y.(Z+1)`
- `minor` -> `X.(Y+1).0`
- `major` -> `(X+1).0.0`
## Error Recovery
- If bumpversion fails, the tag won't exist. Fix the issue and re-run — bumpversion is idempotent as long as the tag doesn't already exist.
- If you need to undo a release commit (before pushing): `git tag -d vX.Y.Z && git reset --soft HEAD~1`
- Never amend a release commit that has been pushed.
## Notes
- When the tag is pushed, the release CI (`.github/workflows/release.yml`) automatically extracts the matching version section from `CHANGELOG.md` and uses it as the GitHub Release body. No manual copy-paste needed.
- The release commit message is controlled by `.bumpversion.cfg` (`Bump version: X.Y.Z -> A.B.C`). Do not override it.
- If you need to manually update the GitHub Release body after the fact: `gh release edit vX.Y.Z --notes-file <(sed -n '/## \[X.Y.Z\]/,/## \[/p' CHANGELOG.md | head -n -1)`
+1 -1
View File
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.2.3
current_version = 0.3.1
commit = True
tag = True
tag_name = v{new_version}
+45 -26
View File
@@ -62,6 +62,7 @@ jobs:
pip install pyinstaller
pip install -r backend/requirements.txt
pip install --no-deps chatterbox-tts
pip install --no-deps hume-tada
- name: Install MLX dependencies (Apple Silicon only)
if: matrix.backend == 'mlx'
@@ -123,6 +124,29 @@ jobs:
p12-file-base64: ${{ secrets.APPLE_CERTIFICATE }}
p12-password: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
- name: Extract release notes from CHANGELOG.md
id: changelog
shell: bash
run: |
# Get the version from the tag (strip leading 'v')
VERSION="${GITHUB_REF_NAME#v}"
# Extract the section for this version from CHANGELOG.md
# Matches from "## [X.Y.Z]" until the next "## [" heading
NOTES=$(sed -n "/^## \[${VERSION}\]/,/^## \[/{/^## \[${VERSION}\]/d;/^## \[/d;p;}" CHANGELOG.md)
# Fall back to a placeholder if the version isn't in the changelog
if [ -z "$(echo "$NOTES" | tr -d '[:space:]')" ]; then
NOTES="See the assets below to download and install this version."
fi
# Use multiline output syntax
{
echo "notes<<CHANGELOG_EOF"
echo "$NOTES"
echo "CHANGELOG_EOF"
} >> "$GITHUB_OUTPUT"
- uses: tauri-apps/[email protected]
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -139,17 +163,7 @@ jobs:
projectPath: tauri
tagName: v__VERSION__
releaseName: "voicebox v__VERSION__"
releaseBody: |
## What's Changed
See the assets below to download and install this version.
### Installation
- **macOS (Apple Silicon)**: Download the `aarch64.dmg` file - uses MLX for fast native inference
- **macOS (Intel)**: Download the `x64.dmg` file - uses PyTorch
- **Windows**: Download the `.msi` installer
- **Linux**: Compile from source (see README)
The app includes automatic updates - future updates will be installed automatically.
releaseBody: ${{ steps.changelog.outputs.notes }}
releaseDraft: true
prerelease: false
args: ${{ matrix.args }}
@@ -175,43 +189,48 @@ jobs:
pip install pyinstaller
pip install -r backend/requirements.txt
pip install --no-deps chatterbox-tts
pip install --no-deps hume-tada
- name: Install PyTorch with CUDA 12.1
- name: Install PyTorch with CUDA 12.8
run: |
pip install torch --index-url https://download.pytorch.org/whl/cu121 --force-reinstall --no-deps
pip install torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install torch --index-url https://download.pytorch.org/whl/cu128 --force-reinstall --no-deps
pip install torchaudio --index-url https://download.pytorch.org/whl/cu128 --force-reinstall --no-deps
- name: Verify CUDA support in torch
run: |
python -c "import torch; print(f'CUDA available in build: {torch.cuda.is_available()}'); print(f'CUDA version: {torch.version.cuda}')"
- name: Build CUDA server binary
- name: Build CUDA server binary (onedir)
shell: bash
working-directory: backend
run: python build_binary.py --cuda
- name: Split binary for GitHub Releases
- name: Package into server core + CUDA libs archives
shell: bash
run: |
python scripts/split_binary.py \
backend/dist/voicebox-server-cuda.exe \
--output release-assets/
python scripts/package_cuda.py \
backend/dist/voicebox-server-cuda/ \
--output release-assets/ \
--cuda-libs-version cu128-v1 \
--torch-compat ">=2.7.0,<2.11.0"
- name: Upload split parts to GitHub Release
- name: Upload archives to GitHub Release
if: startsWith(github.ref, 'refs/tags/')
uses: softprops/action-gh-release@v1
uses: softprops/action-gh-release@v2
with:
files: |
release-assets/voicebox-server-cuda.part*.exe
release-assets/voicebox-server-cuda.sha256
release-assets/voicebox-server-cuda.manifest
release-assets/voicebox-server-cuda.tar.gz
release-assets/voicebox-server-cuda.tar.gz.sha256
release-assets/cuda-libs-cu128-v1.tar.gz
release-assets/cuda-libs-cu128-v1.tar.gz.sha256
release-assets/cuda-libs.json
draft: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Upload binary as workflow artifact
- name: Upload onedir as workflow artifact
uses: actions/upload-artifact@v4
with:
name: voicebox-server-cuda-windows
path: backend/dist/voicebox-server-cuda.exe
path: backend/dist/voicebox-server-cuda/
retention-days: 7
+8
View File
@@ -50,6 +50,14 @@ logs/
app/openapi.json
tauri/src-tauri/binaries/*
tauri/src-tauri/gen/Assets.car
tauri/src-tauri/gen/voicebox.icns
tauri/src-tauri/gen/partial.plist
# PyInstaller
*.spec
# Windows artifacts
nul
# Temporary
tmp/
+437 -69
View File
@@ -1,96 +1,464 @@
<!-- This file is compiled automatically during the release workflow. -->
<!-- Do not edit manually — your changes will be overwritten. -->
<!-- To update the draft: ask the agent to use the draft-release-notes skill. -->
<!-- To finalize a release: ask the agent to use the release-bump skill. -->
# Changelog
All notable changes to Voicebox will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Fixed
- **Profile Name Validation** - Added proper validation to prevent duplicate profile names ([#134](https://github.com/jamiepine/voicebox/issues/134))
- Users now receive clear error messages when attempting to create or update profiles with duplicate names
- Improved error handling in create and update profile API endpoints
- Added comprehensive test suite for duplicate name validation
## [0.3.0] - 2026-03-17
## [0.1.0] - 2026-01-25
This release rewrites the backend into a modular architecture, overhauls the settings UI into routed sub-pages, fixes audio player freezing, migrates documentation to Fumadocs, and ships a batch of bug fixes targeting the most-reported issues from the tracker.
### Added
The backend's 3,000-line monolith `main.py` has been decomposed into domain routers, a services layer, and a proper database package. A style guide and ruff configuration now enforce consistency. On the frontend, settings have been split into dedicated routed pages with server logs, a changelog viewer, and an about page. The audio player no longer freezes mid-playback, and model loading status is now visible in the UI. Seven user-reported bugs have been fixed, including server crashes during sample uploads, generation list staleness, cryptic error messages, and CUDA support for RTX 50-series GPUs.
#### Core Features
- **Voice Cloning** - Clone voices from audio samples using Qwen3-TTS (1.7B and 0.6B models)
- **Voice Profile Management** - Create, edit, and organize voice profiles with multiple samples
- **Speech Generation** - Generate high-quality speech from text using cloned voices
- **Generation History** - Track all generations with search and filtering capabilities
- **Audio Transcription** - Automatic transcription powered by Whisper
- **In-App Recording** - Record audio samples directly in the app with waveform visualization
### Settings Overhaul ([#294](https://github.com/jamiepine/voicebox/pull/294))
- Split settings into routed sub-tabs: General, Generation, GPU, Logs, Changelog, About
- Added live server log viewer with auto-scroll
- Added in-app changelog page that parses `CHANGELOG.md` at build time
- Added About page with version info, license, and generation folder quick-open
- Extracted reusable `SettingRow` component for consistent setting layouts
#### Desktop App
- **Tauri Desktop App** - Native desktop application for macOS, Windows, and Linux
- **Local Server Mode** - Embedded Python server runs automatically
- **Remote Server Mode** - Connect to a remote Voicebox server on your network
- **Auto-Updates** - Automatic update notifications and installation
### Audio Player Fix ([#293](https://github.com/jamiepine/voicebox/pull/293))
- Fixed audio player freezing during playback
- Improved playback UX with better state management and listener cleanup
- Fixed restart race condition during regeneration
- Added stable keys for audio element re-rendering
- Improved accessibility across player controls
#### API
- **REST API** - Full REST API for voice synthesis and profile management
- **OpenAPI Documentation** - Interactive API docs at `/docs` endpoint
- **Type-Safe Client** - Auto-generated TypeScript client from OpenAPI schema
### Backend Refactor ([#285](https://github.com/jamiepine/voicebox/pull/285))
- Extracted all routes from `main.py` into 13 domain routers under `backend/routes/` — `main.py` dropped from ~3,100 lines to ~10
- Moved CRUD and service modules into `backend/services/`, platform detection into `backend/utils/`
- Split monolithic `database.py` into a `database/` package with separate `models`, `session`, `migrations`, and `seed` modules
- Added `backend/STYLE_GUIDE.md` and `pyproject.toml` with ruff linting config
- Removed dead code: unused `_get_cuda_dll_excludes`, stale `studio.py`, `example_usage.py`, old `Makefile`
- Deduplicated shared logic across TTS backends into `backends/base.py`
- Improved startup logging with version, platform, data directory, and database stats
- Fixed startup database session leak — sessions now rollback and close in `finally` block
- Isolated shutdown unload calls so one backend failure doesn't block the others
- Handled null duration in `story_items` migration
- Reject model migration when target is a subdirectory of source cache
#### Technical
- **Voice Prompt Caching** - Fast regeneration with cached voice prompts
- **Multi-Sample Support** - Combine multiple audio samples for better voice quality
- **GPU/CPU/MPS Support** - Automatic device detection and optimization
- **Model Management** - Lazy loading and VRAM management
- **SQLite Database** - Local data persistence
### Documentation Rewrite ([#288](https://github.com/jamiepine/voicebox/pull/288))
- Migrated docs site from Mintlify to Fumadocs (Next.js-based)
- Rewrote introduction and root page with content from README
- Added "Edit on GitHub" links and last-updated timestamps on all pages
- Generated OpenAPI spec and auto-generated API reference pages
- Removed stale planning docs (`CUDA_BACKEND_SWAP`, `EXTERNAL_PROVIDERS`, `MLX_AUDIO`, `TTS_PROVIDER_ARCHITECTURE`, etc.)
- Sidebar groups now expand by default; root redirects to `/docs`
- Added OG image metadata and `/og` preview page
### Technical Details
### UI & Frontend
- Added model loading status indicator and effects preset dropdown ([3187344](https://github.com/jamiepine/voicebox/commit/3187344))
- Fixed take-label race condition during regeneration
- Added accessible focus styling to select component
- Softened select focus indicator opacity
- Addressed 4 critical and 12 major issues from CodeRabbit review
- Built with Tauri v2 (Rust + React)
- FastAPI backend with async Python
- TypeScript frontend with React Query and Zustand
- Qwen3-TTS for voice cloning
- Whisper for transcription
### Bug Fixes ([#295](https://github.com/jamiepine/voicebox/pull/295))
- Fixed sample uploads crashing the server — audio decoding now runs in a thread pool instead of blocking the async event loop ([#278](https://github.com/jamiepine/voicebox/issues/278))
- Fixed generation list not updating when a generation completes — switched to `refetchQueries` for reliable cache busting, added SSE error fallback, and page reset on completion ([#231](https://github.com/jamiepine/voicebox/issues/231))
- Fixed error toasts showing `[object Object]` instead of the actual error message ([#290](https://github.com/jamiepine/voicebox/issues/290))
- Added Whisper model selection (`base`, `small`, `medium`, `large`, `turbo`) and expanded language support to the `/transcribe` endpoint ([#233](https://github.com/jamiepine/voicebox/issues/233))
- Upgraded CUDA backend build from cu121 to cu126 for RTX 50-series (Blackwell) GPU support ([#289](https://github.com/jamiepine/voicebox/issues/289))
- Handled client disconnects in SSE and streaming endpoints to suppress `[Errno 32] Broken Pipe` errors ([#248](https://github.com/jamiepine/voicebox/issues/248))
- Fixed Docker build failure from pip hash mismatch on Qwen3-TTS dependencies ([#286](https://github.com/jamiepine/voicebox/issues/286))
- Added 50 MB upload size limit with chunked reads to prevent unbounded memory allocation on sample uploads
- Eliminated redundant double audio decode in sample processing pipeline
### Platform Fixes
- Replaced `netstat` with `TcpStream` + PowerShell for Windows port detection ([#277](https://github.com/jamiepine/voicebox/pull/277))
- Fixed Docker frontend build and cleaned up Docker docs
- Fixed macOS download links to use `.dmg` instead of `.app.tar.gz`
- Added dynamic download redirect routes to landing site
### Release Tooling
- Added `draft-release-notes` and `release-bump` agent skills
- Wired CI release workflow to extract notes from `CHANGELOG.md` for GitHub Releases
- Backfilled changelog with all historical releases
## [0.2.3] - 2026-03-15
The "it works in dev but not in prod" release. This version fixes a series of PyInstaller bundling issues that prevented model downloading, loading, generation, and progress tracking from working in production builds.
### Model Downloads Now Actually Work
The v0.2.1/v0.2.2 builds could not download or load models that weren't already cached from a dev install. This release fixes the entire chain:
- **Chatterbox, Chatterbox Turbo, and LuxTTS** all download, load, and generate correctly in bundled builds
- **Real-time download progress** — byte-level progress bars now work in production. The root cause: `huggingface_hub` silently disables tqdm progress bars based on logger level, which prevented our progress tracker from receiving byte updates. We now force-enable the internal counter regardless.
- **Fixed Python 3.12.0 `code.replace()` bug** — the macOS build was on Python 3.12.0, which has a [known CPython bug](https://github.com/pyinstaller/pyinstaller/issues/7992) that corrupts bytecode when PyInstaller rewrites code objects. This caused `NameError: name 'obj' is not defined` crashes during scipy/torch imports. Upgraded to Python 3.12.13.
### PyInstaller Fixes
- Collect all `inflect` files — `typeguard`'s `@typechecked` decorator calls `inspect.getsource()` at import time, which needs `.py` source files, not just bytecode. Fixes LuxTTS "could not get source code" error.
- Collect all `perth` files — bundles the pretrained watermark model (`hparams.yaml`, `.pth.tar`) needed by Chatterbox at runtime
- Collect all `piper_phonemize` files — bundles `espeak-ng-data/` (phoneme tables, language dicts) needed by LuxTTS for text-to-phoneme conversion
- Set `ESPEAK_DATA_PATH` in frozen builds so the espeak-ng C library finds the bundled data instead of looking at `/usr/share/espeak-ng-data/`
- Collect all `linacodec` files — fixes `inspect.getsource` error in Vocos codec
- Collect all `zipvoice` files — fixes source code lookup in LuxTTS voice cloning
- Copy metadata for `requests`, `transformers`, `huggingface-hub`, `tokenizers`, `safetensors`, `tqdm` — fixes `importlib.metadata` lookups in frozen binary
- Add hidden imports for `chatterbox`, `chatterbox_turbo`, `luxtts`, `zipvoice` backends
- Add `multiprocessing.freeze_support()` to fix resource_tracker subprocess crash in frozen binary
- `--noconsole` now only applied on Windows — macOS/Linux need stdout/stderr for Tauri sidecar log capture
- Hardened `sys.stdout`/`sys.stderr` devnull redirect to test writability, not just `None` check
### Updater
- Fixed updater artifact generation with `v1Compatible` for `tauri-action` signature files
- Updated `tauri-action` to v0.6 to fix updater JSON and `.sig` generation
### Other Fixes
- Full traceback logging on all backend model loading errors (was just `str(e)` before)
## [0.2.2] - 2026-03-15
- Fix Chatterbox model support in bundled builds
- Fix LuxTTS/ZipVoice support in bundled builds
- Auto-update CUDA binary when app version changes
- CUDA download progress bar
- Fix server process staying alive on macOS (SIGHUP handling, watchdog grace period)
- Hide console window when running CUDA binary on Windows
## [0.2.1] - 2026-03-15
Voicebox v0.1.x was a single-engine voice cloning app built around Qwen3-TTS. v0.2.0 is a ground-up rethink: four TTS engines, 23 languages, paralinguistic emotion controls, a post-processing effects pipeline, unlimited generation length, an async generation queue, and support for every major GPU vendor. Plus Docker.
### New TTS Engines
#### Multi-Engine Architecture
Voicebox now runs **four independent TTS engines** behind a thread-safe per-engine backend registry. Switch engines per-generation from a single dropdown — no restart required.
| Engine | Languages | Size | Key Strengths |
| --------------------------- | --------- | ------- | --------------------------------------------- |
| **Qwen3-TTS 1.7B** | 10 | ~3.5 GB | Highest quality, delivery instructions |
| **Qwen3-TTS 0.6B** | 10 | ~1.2 GB | Lighter, faster variant |
| **LuxTTS** | English | ~300 MB | CPU-friendly, 48 kHz output, 150x realtime |
| **Chatterbox Multilingual** | 23 | ~3.2 GB | Broadest language coverage, zero-shot cloning |
| **Chatterbox Turbo** | English | ~1.5 GB | 350M params, low latency, paralinguistic tags |
#### Chatterbox Multilingual — 23 Languages ([#257](https://github.com/jamiepine/voicebox/pull/257))
Zero-shot voice cloning in Arabic, Chinese, Danish, Dutch, English, Finnish, French, German, Greek, Hebrew, Hindi, Italian, Japanese, Korean, Malay, Norwegian, Polish, Portuguese, Russian, Spanish, Swahili, Swedish, and Turkish.
#### LuxTTS — Lightweight English TTS ([#254](https://github.com/jamiepine/voicebox/pull/254))
A fast, CPU-friendly English engine. ~300 MB download, 48 kHz output, runs at 150x realtime on CPU.
#### Chatterbox Turbo — Expressive English ([#258](https://github.com/jamiepine/voicebox/pull/258))
A fast 350M-parameter English model with inline paralinguistic tags.
#### Paralinguistic Tags Autocomplete ([#265](https://github.com/jamiepine/voicebox/pull/265))
Type `/` in the text input with Chatterbox Turbo selected to open an autocomplete for **9 expressive tags**: `[laugh]` `[chuckle]` `[gasp]` `[cough]` `[sigh]` `[groan]` `[sniff]` `[shush]` `[clear throat]`
### Generation
#### Unlimited Generation Length — Auto-Chunking ([#266](https://github.com/jamiepine/voicebox/pull/266))
Long text is now automatically split at sentence boundaries, generated per-chunk, and crossfaded back together. Engine-agnostic.
- Auto-chunking limit slider — 100–5,000 chars (default 800)
- Crossfade slider — 0–200ms (default 50ms)
- Max text length raised to 50,000 characters
- Smart splitting respects abbreviations, CJK punctuation, and `[tags]`
#### Asynchronous Generation Queue ([#269](https://github.com/jamiepine/voicebox/pull/269))
Generation is now fully non-blocking. Serial execution queue prevents GPU contention. Real-time SSE status streaming.
#### Generation Versions
Every generation now supports multiple versions with provenance tracking — original, effects versions, takes, source tracking, version pinning in stories, and favorites.
### Post-Processing Effects ([#271](https://github.com/jamiepine/voicebox/pull/271))
A full audio effects system powered by Spotify's `pedalboard` library: Pitch Shift, Reverb, Delay, Chorus/Flanger, Compressor, Gain, High-Pass Filter, Low-Pass Filter. 4 built-in presets, custom presets, per-profile default effects, and live preview.
### Platform Support
- macOS (Apple Silicon and Intel)
- Windows
- Linux (AppImage)
- **Windows Support** ([#272](https://github.com/jamiepine/voicebox/pull/272)) — Full Windows support with CUDA GPU detection
- **Linux** ([#262](https://github.com/jamiepine/voicebox/pull/262)) — AMD ROCm, NVIDIA GBM fix, WebKitGTK mic access (build from source)
- **NVIDIA CUDA Backend Swap** ([#252](https://github.com/jamiepine/voicebox/pull/252)) — Download and swap in CUDA backend from within the app
- **Intel Arc (XPU) and DirectML** — PyTorch backend supports Intel Arc and DirectML
- **Docker + Web Deployment** ([#161](https://github.com/jamiepine/voicebox/pull/161)) — 3-stage build, non-root runtime, health checks
- **Whisper Turbo** — Added `openai/whisper-large-v3-turbo` as a transcription model option
---
### Model Management ([#268](https://github.com/jamiepine/voicebox/pull/268))
## [Unreleased]
Per-model unload, custom models directory, model folder migration, download cancel/clear UI ([#238](https://github.com/jamiepine/voicebox/pull/238)), restructured settings UI.
### Fixed
- Audio export failing when Tauri save dialog returns object instead of string path
- OpenAPI client generator script now documents the local backend port and avoids an unused loop variable warning
### Security & Reliability
### Added
- **justfile** - Comprehensive development workflow automation with commands for setup, development, building, testing, and code quality checks
- Cross-platform support (macOS, Linux, Windows)
- Python version detection and compatibility warnings
- Self-documenting help system with `just --list`
- CORS hardening ([#88](https://github.com/jamiepine/voicebox/pull/88))
- Network access toggle ([#133](https://github.com/jamiepine/voicebox/pull/133))
- Offline crash fix ([#152](https://github.com/jamiepine/voicebox/pull/152))
- Atomic audio saves ([#263](https://github.com/jamiepine/voicebox/pull/263))
- Filesystem health endpoint
- Chatterbox float64 dtype fix ([#264](https://github.com/jamiepine/voicebox/pull/264))
### Changed
- **README** - Updated Quick Start with justfile-based setup instructions
### Accessibility ([#243](https://github.com/jamiepine/voicebox/pull/243))
### Removed
- **Makefile** - Replaced by justfile (cross-platform, simpler syntax)
Screen reader support, keyboard navigation, state-aware `aria-label` attributes on all interactive controls.
---
### UI Polish
## [Unreleased - Planned]
- Redesigned landing page ([#274](https://github.com/jamiepine/voicebox/pull/274))
- Voices tab overhaul with inline inspector
- Responsive layout improvements
- Duplicate profile name validation ([#175](https://github.com/jamiepine/voicebox/pull/175))
### Planned
- Real-time streaming synthesis
- Conversation mode with multiple speakers
- Voice effects (pitch shift, reverb, M3GAN-style)
- Timeline-based audio editor
- Additional voice models (XTTS, Bark)
- Voice design from text descriptions
- Project system for saving sessions
- Plugin architecture
### Community Contributors
---
[@haosenwang1018](https://github.com/haosenwang1018), [@Balneario-de-Cofrentes](https://github.com/Balneario-de-Cofrentes), [@ageofalgo](https://github.com/ageofalgo), [@mikeswann](https://github.com/mikeswann), [@rayl15](https://github.com/rayl15), [@mpecanha](https://github.com/mpecanha), [@ways2read](https://github.com/ways2read), [@ieguiguren](https://github.com/ieguiguren), [@Vaibhavee89](https://github.com/Vaibhavee89), [@pandego](https://github.com/pandego), [@luminest-llc](https://github.com/luminest-llc)
## [0.1.13] - 2026-02-23
### Stability and reliability
- [#95](https://github.com/jamiepine/voicebox/pull/95) Fix: selecting 0.6B model still downloads and uses 1.7B
- [#93](https://github.com/jamiepine/voicebox/pull/93) fix(mlx): bundle native libs and broaden error handling for Apple Silicon
- [#79](https://github.com/jamiepine/voicebox/pull/79) fix: handle non-ASCII filenames in Content-Disposition headers
- [#78](https://github.com/jamiepine/voicebox/pull/78) fix: guard getUserMedia call against undefined mediaDevices in non-secure contexts
- [#77](https://github.com/jamiepine/voicebox/pull/77) fix: await for confirmation before deleting voices and channels
- [#128](https://github.com/jamiepine/voicebox/pull/128) fix: resolve multiple issues (#96, #119, #111, #108, #121, #125, #127)
- [#40](https://github.com/jamiepine/voicebox/pull/40) Fix: audio export path resolution
### Build and packaging
- [#122](https://github.com/jamiepine/voicebox/pull/122) fix(web): add @tailwindcss/vite plugin to web config
- [#126](https://github.com/jamiepine/voicebox/pull/126) Create requirements.txt
### UX and docs
- [#44](https://github.com/jamiepine/voicebox/pull/44) Enhances floating generate box UX
- [#57](https://github.com/jamiepine/voicebox/pull/57) chore: updates repo URL in README
- [#146](https://github.com/jamiepine/voicebox/pull/146) Add Spacebot banner to landing page
- [#1](https://github.com/jamiepine/voicebox/pull/1) Improvements
## [0.1.12] - 2026-01-31
### Model Download UX Overhaul
- Real-time download progress tracking with accurate percentage and speed info
- No more downloading notifications during generation even when its not downloading
- Better error handling and status reporting throughout the download process
### Other Improvements
- Enhanced health check endpoint with GPU type information
- Improved model caching verification
- More reliable SSE progress updates
- Actual update notifications — no need to manually check in settings anymore
## [0.1.11] - 2026-01-30
- Fixed transcriptions on MLX
- Fixed model download progress (finally)
## [0.1.10] - 2026-01-30
### Faster generation on Apple Silicon
Massive speed gains, from around 20s per generation to 2-3s. Added native MLX backend support for Apple Silicon, providing significantly faster TTS and STT generation on M-series macOS machines.
- **MLX Backend** — New backend implementation optimized for Apple Silicon using MLX framework
- **Dynamic Backend Selection** — Automatically detects platform and selects between MLX (macOS) and PyTorch (other platforms)
- Refactored TTS and STT logic into modular backend implementations
- Updated build process to include MLX-specific dependencies for macOS builds
## [0.1.9] - 2026-01-30
### Improved voice profile creation flow
- Voice create drafts: No longer lose work if you close the modal
- Fixed whisper only transcribing English or Chinese, now has support for all languages
### Improved Stories editor
- Added spacebar for play/pause
- Timeline now auto-scrolls to follow playhead during playback
- Fixed misalignment of the items with mouse when picking up
- Fixed hitbox for selecting an item
- Fixed playhead jumping forward when pressing play
### Generation box improvements
- Instruct mode no longer wipes prompt text
- Improved UI cleanliness
### Misc
- Fixed "Model downloading" toast during generation when model is already downloaded
## [0.1.8] - 2026-01-29
### Model Download Timeout Issues
Fixed critical issue where model downloads would fail with "Failed to fetch" errors on Windows. Refactored download endpoints to return immediately and continue downloads in background.
### Cross-Platform Cache Path Issues
Fixed hardcoded `~/.cache/huggingface/hub` paths that don't work on Windows. All cache paths now use `hf_constants.HF_HUB_CACHE` for proper cross-platform support.
### Windows Process Management
- Added `/shutdown` endpoint for graceful server shutdown on Windows
- Added `gpu_type` field to health check response
## [0.1.7] - 2026-01-29
- Trim and split audio clips in Story Editor
- Auto-activation of stories in Story Editor with visible playhead
- Conditional auto-play support in AudioPlayer for better user control
- Refactored audio loading across HistoryTable, SampleList, and generation forms
- Audio now only auto-plays when explicitly intended, preventing unexpected playback
## [0.1.6] - 2026-01-29
### Introducing Stories
A full voice editor for composing podcasts and generated conversations.
- **Stories Editor** — Create multi-voice narratives, podcasts, or conversations with a timeline-based editor
- Compose tracks with different voices
- Edit and arrange audio segments inline
- Build generated conversations with multiple participants
- **Improved Voice Generation UI** — Auto-resizing input, default voice selection, better layout
- **Track Editor Integration** — Inline track editing within story items
## [0.1.5] - 2026-01-28
Fixed recording length limit at 0:29 to auto stop instead of passing the limit and getting an error, which would cause users to lose their recording.
## [0.1.4] - 2026-01-28
- Audio channel management system
- Native audio playback handling in AudioPlayer component
- Refactored ConnectionForm and Checkbox components
- Improved layout consistency and responsiveness
- Added safe area constants for better responsive design
## [0.1.3] - 2026-01-27
- Improved the generate textbox
- Maybe fixed Windows autoupdate restarting entire computer
## [0.1.2] - 2026-01-27
### Audio Capture & Format Conversion
- Added audio format conversion util
- Enhanced system audio capture on macOS and Windows
- Improved audio recording hooks
- Added audio input entitlement for macOS
- Added audio capture tests
### Update System
- Enhanced auto-updater functionality and update status display
## [0.1.1] - 2026-01-27
### Platform Support
- **macOS Audio Capture** — Native audio capture support for sample creation
- **Windows Audio Capture** — WASAPI implementation with improved thread safety
- **Linux Support** — Temporarily removed builds due to runner disk space constraints
### Audio Features
- Play/pause for audio samples across all components
- Three new sample components: Recording, System capture, Upload with drag-and-drop
- Audio validation, error handling, and consistent cleanup
### Voice Profile Management
- Profile import with file size validation (100MB limit)
- Enhanced profile form with new audio sample components
- Drag-and-drop support for audio file uploads
### Server Management
- Changed default URL from `localhost:8000` to `127.0.0.1:17493`
- Server reuse logic, "keep server running" preference, orphaned process handling
### Build & Release
- Added `.bumpversion.cfg` for automated version management
- Enhanced icon generation script for multi-size Windows icons
### Bug Fixes
- Fixed date formatting for timezone-less date strings
- Fixed getLatestRelease file filtering
- Improved audio duration metadata on Windows
## [0.1.0] - 2026-01-27
The first public release of Voicebox — an open-source voice synthesis studio powered by Qwen3-TTS.
### Voice Cloning with Qwen3-TTS
- Automatic model download from HuggingFace
- Multiple model sizes (1.7B and 0.6B)
- Voice prompt caching for instant regeneration
- English and Chinese support
### Voice Profile Management
- Create profiles from audio files or record directly in the app
- Multiple samples per profile for higher quality cloning
- Import/Export profiles
- Automatic transcription via Whisper
### Speech Generation
- Simple text-to-speech with profile selection
- Seed control for reproducible generations
- Long-form support up to 5,000 characters
### Generation History
- Full history with metadata
- Search by text content
- Inline playback and download
### Flexible Deployment
- Local mode with bundled backend
- Remote mode for GPU servers on your network
- One-click server setup
### Desktop Experience
- Built with Tauri v2 (Rust) — native performance, not Electron
- Cross-platform: macOS and Windows
- No Python installation required
### Tech Stack
Tauri v2, React, TypeScript, Tailwind CSS, FastAPI, Qwen3-TTS, Whisper, SQLite
[Unreleased]: https://github.com/jamiepine/voicebox/compare/v0.2.3...HEAD
[0.2.3]: https://github.com/jamiepine/voicebox/compare/v0.2.2...v0.2.3
[0.2.2]: https://github.com/jamiepine/voicebox/compare/v0.2.1...v0.2.2
[0.2.1]: https://github.com/jamiepine/voicebox/compare/v0.1.13...v0.2.1
[0.1.13]: https://github.com/jamiepine/voicebox/compare/v0.1.12...v0.1.13
[0.1.12]: https://github.com/jamiepine/voicebox/compare/v0.1.11...v0.1.12
[0.1.11]: https://github.com/jamiepine/voicebox/compare/v0.1.10...v0.1.11
[0.1.10]: https://github.com/jamiepine/voicebox/compare/v0.1.9...v0.1.10
[0.1.9]: https://github.com/jamiepine/voicebox/compare/v0.1.8...v0.1.9
[0.1.8]: https://github.com/jamiepine/voicebox/compare/v0.1.7...v0.1.8
[0.1.7]: https://github.com/jamiepine/voicebox/compare/v0.1.6...v0.1.7
[0.1.6]: https://github.com/jamiepine/voicebox/compare/v0.1.5...v0.1.6
[0.1.5]: https://github.com/jamiepine/voicebox/compare/v0.1.4...v0.1.5
[0.1.4]: https://github.com/jamiepine/voicebox/compare/v0.1.3...v0.1.4
[0.1.3]: https://github.com/jamiepine/voicebox/compare/v0.1.2...v0.1.3
[0.1.2]: https://github.com/jamiepine/voicebox/compare/v0.1.1...v0.1.2
[0.1.1]: https://github.com/jamiepine/voicebox/compare/v0.1.0...v0.1.1
[0.1.0]: https://github.com/jamiepine/voicebox/releases/tag/v0.1.0
+4
View File
@@ -31,8 +31,12 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
&& rm -rf /var/lib/apt/lists/*
RUN pip install --no-cache-dir --upgrade pip
COPY backend/requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt
RUN pip install --no-cache-dir --prefix=/install --no-deps chatterbox-tts
RUN pip install --no-cache-dir --prefix=/install --no-deps hume-tada
RUN pip install --no-cache-dir --prefix=/install \
git+https://github.com/QwenLM/Qwen3-TTS.git
-58
View File
@@ -1,58 +0,0 @@
# Voicebox Offline Mode Fix
## Problem
Voicebox crashes when generating speech if HuggingFace is unreachable, even when models are fully cached locally.
**Root Cause:**
- Voicebox downloads `mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16` (MLX optimized version)
- But `mlx_audio.tts.load()` tries to fetch `config.json` from original repo `Qwen/Qwen3-TTS-12Hz-1.7B-Base`
- This network request fails → server crashes with `RemoteDisconnected`
**Related Issues:**
- Issue #150: "Internet connection required, even though models are downloaded?"
- Issue #151: "API Stability Issues: Model Loading Hangs and Server Crashes"
## Solution
Two-part fix:
### 1. Monkey-patch huggingface_hub (`backend/utils/hf_offline_patch.py`)
- Intercepts cache lookup functions
- Forces offline mode early (before mlx_audio imports)
- Adds debug logging for cache hits/misses
### 2. Symlink original repo to MLX version (`ensure_original_qwen_config_cached()`)
- When original `Qwen/Qwen3-TTS-12Hz-1.7B-Base` cache doesn't exist
- But MLX `mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16` does exist
- Creates a symlink so cache lookups succeed
## Files Changed
- `backend/backends/mlx_backend.py` - Added patch imports at top
- `backend/utils/hf_offline_patch.py` - New patch module
## Testing
To test this fix:
1. Build Voicebox from source: `just build`
2. Disconnect from internet
3. Try generating speech
4. Should work without network requests
## Build Instructions
```bash
# Install dependencies
just setup
# Build the app
just build
# Or build just the server
just build-server
```
## Notes
- The patch is applied automatically when `mlx_backend.py` is imported
- Set `VOICEBOX_OFFLINE_PATCH=0` to disable the patch
- The symlink approach works because the config.json is compatible between versions
---
*Patch contributed by community*
+59 -52
View File
@@ -27,10 +27,10 @@
<p align="center">
<a href="https://voicebox.sh">voicebox.sh</a> •
<a href="https://docs.voicebox.sh">Docs</a> •
<a href="#download">Download</a> •
<a href="#features">Features</a> •
<a href="#api">API</a> •
<a href="#roadmap">Roadmap</a>
<a href="#api">API</a>
</p>
<br/>
@@ -59,10 +59,10 @@
## What is Voicebox?
Voicebox is a **local-first voice cloning studio** — a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
Voicebox is a **local-first voice cloning studio** — a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 5 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
- **Complete privacy** — models and voice data stay on your machine
- **4 TTS engines** — Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo
- **5 TTS engines** — Qwen3-TTS, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo, and HumeAI TADA
- **23 languages** — from English to Arabic, Japanese, Hindi, Swahili, and more
- **Post-processing effects** — pitch shift, reverb, delay, chorus, compression, and filters
- **Expressive speech** — paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo
@@ -76,12 +76,12 @@ Voicebox is a **local-first voice cloning studio** — a free and open-source al
## Download
| Platform | Download |
|----------|----------|
| macOS (Apple Silicon) | [Download DMG](https://voicebox.sh/download/mac-arm) |
| macOS (Intel) | [Download DMG](https://voicebox.sh/download/mac-intel) |
| Windows | [Download MSI](https://voicebox.sh/download/windows) |
| Docker | `docker compose up` |
| Platform | Download |
| --------------------- | ------------------------------------------------------ |
| macOS (Apple Silicon) | [Download DMG](https://voicebox.sh/download/mac-arm) |
| macOS (Intel) | [Download DMG](https://voicebox.sh/download/mac-intel) |
| Windows | [Download MSI](https://voicebox.sh/download/windows) |
| Docker | `docker compose up` |
> **[View all binaries →](https://github.com/jamiepine/voicebox/releases/latest)**
@@ -93,14 +93,15 @@ Voicebox is a **local-first voice cloning studio** — a free and open-source al
### Multi-Engine Voice Cloning
Four TTS engines with different strengths, switchable per-generation:
Five TTS engines with different strengths, switchable per-generation:
| Engine | Languages | Strengths |
|--------|-----------|-----------|
| **Qwen3-TTS** (0.6B / 1.7B) | 10 | High-quality multilingual cloning, delivery instructions ("speak slowly", "whisper") |
| **LuxTTS** | English | Lightweight (~1GB VRAM), 48kHz output, 150x realtime on CPU |
| **Chatterbox Multilingual** | 23 | Broadest language coverage — Arabic, Danish, Finnish, Greek, Hebrew, Hindi, Malay, Norwegian, Polish, Swahili, Swedish, Turkish and more |
| **Chatterbox Turbo** | English | Fast 350M model with paralinguistic emotion/sound tags |
| Engine | Languages | Strengths |
| --------------------------- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| **Qwen3-TTS** (0.6B / 1.7B) | 10 | High-quality multilingual cloning, delivery instructions ("speak slowly", "whisper") |
| **LuxTTS** | English | Lightweight (~1GB VRAM), 48kHz output, 150x realtime on CPU |
| **Chatterbox Multilingual** | 23 | Broadest language coverage — Arabic, Danish, Finnish, Greek, Hebrew, Hindi, Malay, Norwegian, Polish, Swahili, Swedish, Turkish and more |
| **Chatterbox Turbo** | English | Fast 350M model with paralinguistic emotion/sound tags |
| **TADA** (1B / 3B) | 10 | HumeAI speech-language model — 700s+ coherent audio, text-acoustic dual alignment |
### Emotions & Paralinguistic Tags
@@ -112,16 +113,16 @@ Type `/` in the text input to insert expressive tags that the model synthesizes
8 audio effects powered by Spotify's `pedalboard` library. Apply after generation, preview in real time, build reusable presets.
| Effect | Description |
|--------|-------------|
| Pitch Shift | Up or down by up to 12 semitones |
| Reverb | Configurable room size, damping, wet/dry mix |
| Delay | Echo with adjustable time, feedback, and mix |
| Effect | Description |
| ---------------- | --------------------------------------------- |
| Pitch Shift | Up or down by up to 12 semitones |
| Reverb | Configurable room size, damping, wet/dry mix |
| Delay | Echo with adjustable time, feedback, and mix |
| Chorus / Flanger | Modulated delay for metallic or lush textures |
| Compressor | Dynamic range compression |
| Gain | Volume adjustment (-40 to +40 dB) |
| High-Pass Filter | Remove low frequencies |
| Low-Pass Filter | Remove high frequencies |
| Compressor | Dynamic range compression |
| Gain | Volume adjustment (-40 to +40 dB) |
| High-Pass Filter | Remove low frequencies |
| Low-Pass Filter | Remove high frequencies |
Ships with 4 built-in presets (Robotic, Radio, Echo Chamber, Deep Voice) and supports custom presets. Effects can be assigned per-profile as defaults.
@@ -186,14 +187,14 @@ Multi-voice timeline editor for conversations, podcasts, and narratives.
### GPU Support
| Platform | Backend | Notes |
|----------|---------|-------|
| macOS (Apple Silicon) | MLX (Metal) | 4-5x faster via Neural Engine |
| Platform | Backend | Notes |
| ------------------------ | -------------- | ---------------------------------------------- |
| macOS (Apple Silicon) | MLX (Metal) | 4-5x faster via Neural Engine |
| Windows / Linux (NVIDIA) | PyTorch (CUDA) | Auto-downloads CUDA binary from within the app |
| Linux (AMD) | PyTorch (ROCm) | Auto-configures HSA_OVERRIDE_GFX_VERSION |
| Windows (any GPU) | DirectML | Universal Windows GPU support |
| Intel Arc | IPEX/XPU | Intel discrete GPU acceleration |
| Any | CPU | Works everywhere, just slower |
| Linux (AMD) | PyTorch (ROCm) | Auto-configures HSA_OVERRIDE_GFX_VERSION |
| Windows (any GPU) | DirectML | Universal Windows GPU support |
| Intel Arc | IPEX/XPU | Intel discrete GPU acceleration |
| Any | CPU | Works everywhere, just slower |
---
@@ -224,30 +225,30 @@ Full API documentation available at `http://localhost:17493/docs`.
## Tech Stack
| Layer | Technology |
|-------|------------|
| Desktop App | Tauri (Rust) |
| Frontend | React, TypeScript, Tailwind CSS |
| State | Zustand, React Query |
| Backend | FastAPI (Python) |
| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo |
| Effects | Pedalboard (Spotify) |
| Transcription | Whisper / Whisper Turbo (PyTorch or MLX) |
| Inference | MLX (Apple Silicon) / PyTorch (CUDA/ROCm/XPU/CPU) |
| Database | SQLite |
| Audio | WaveSurfer.js, librosa |
| Layer | Technology |
| ------------- | ------------------------------------------------- |
| Desktop App | Tauri (Rust) |
| Frontend | React, TypeScript, Tailwind CSS |
| State | Zustand, React Query |
| Backend | FastAPI (Python) |
| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo, TADA |
| Effects | Pedalboard (Spotify) |
| Transcription | Whisper / Whisper Turbo (PyTorch or MLX) |
| Inference | MLX (Apple Silicon) / PyTorch (CUDA/ROCm/XPU/CPU) |
| Database | SQLite |
| Audio | WaveSurfer.js, librosa |
---
## Roadmap
| Feature | Description |
|---------|-------------|
| **Real-time Streaming** | Stream audio as it generates, word by word |
| **Voice Design** | Create new voices from text descriptions |
| **More Models** | XTTS, Bark, and other open-source voice models |
| **Plugin Architecture** | Extend with custom models and effects |
| **Mobile Companion** | Control Voicebox from your phone |
| Feature | Description |
| ----------------------- | ---------------------------------------------- |
| **Real-time Streaming** | Stream audio as it generates, word by word |
| **Voice Design** | Create new voices from text descriptions |
| **More Models** | XTTS, Bark, and other open-source voice models |
| **Plugin Architecture** | Extend with custom models and effects |
| **Mobile Companion** | Control Voicebox from your phone |
---
@@ -276,6 +277,12 @@ just build # Build CPU server binary + Tauri app
just build-local # (Windows) Build CPU + CUDA server binaries + Tauri app
```
### Adding New Voice Models
The multi-engine architecture makes adding new TTS engines straightforward. A [step-by-step guide](docs/content/docs/developer/tts-engines.mdx) covers the full process: dependency research, backend protocol implementation, frontend wiring, and PyInstaller bundling.
The guide is optimized for AI coding agents. An [agent skill](.agents/skills/add-tts-engine/SKILL.md) can pick up a model name and handle the entire integration autonomously — you just test the build locally.
### Project Structure
```
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@voicebox/app",
"version": "0.2.3",
"version": "0.3.1",
"private": true,
"type": "module",
"scripts": {
+23
View File
@@ -0,0 +1,23 @@
import { readFileSync } from 'node:fs';
import path from 'node:path';
import type { Plugin } from 'vite';
/** Vite plugin that exposes CHANGELOG.md as `virtual:changelog`. */
export function changelogPlugin(repoRoot: string): Plugin {
const virtualId = 'virtual:changelog';
const resolvedId = '\0' + virtualId;
const changelogPath = path.resolve(repoRoot, 'CHANGELOG.md');
return {
name: 'changelog',
resolveId(id) {
if (id === virtualId) return resolvedId;
},
load(id) {
if (id === resolvedId) {
const raw = readFileSync(changelogPath, 'utf-8');
return `export default ${JSON.stringify(raw)};`;
}
},
};
}
+9
View File
@@ -8,6 +8,7 @@ import { TOP_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { cn } from '@/lib/utils/cn';
import { usePlatform } from '@/platform/PlatformContext';
import { router } from '@/router';
import { useLogStore } from '@/stores/logStore';
import { useServerStore } from '@/stores/serverStore';
const LOADING_MESSAGES = [
@@ -63,6 +64,14 @@ function App() {
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.lifecycle]);
// Subscribe to server logs
useEffect(() => {
const unsubscribe = platform.lifecycle.subscribeToServerLogs((entry) => {
useLogStore.getState().addEntry(entry);
});
return unsubscribe;
}, [platform.lifecycle]);
// Setup window close handler and auto-start server when running in Tauri (production only)
useEffect(() => {
if (!platform.metadata.isTauri) {
+189 -406
View File
@@ -17,7 +17,6 @@ export function AudioPlayer() {
audioUrl,
audioId,
profileId,
title,
isPlaying,
currentTime,
duration,
@@ -63,7 +62,7 @@ export function AudioPlayer() {
);
return shouldUseNative;
}, [profileChannels, channels, profileId]);
}, [profileChannels, channels, platform.metadata.isTauri]);
const waveformRef = useRef<HTMLDivElement>(null);
const wavesurferRef = useRef<WaveSurfer | null>(null);
@@ -73,31 +72,21 @@ export function AudioPlayer() {
const isUsingNativePlaybackRef = useRef(false);
const [isLoading, setIsLoading] = useState(false);
const [error, setError] = useState<string | null>(null);
const [wsReady, setWsReady] = useState(false);
// Initialize WaveSurfer (only when audioUrl exists and container is ready)
// Create WaveSurfer once when the player becomes visible (audioUrl is set).
// This instance is reused for all subsequent audio loads - never destroyed until unmount.
useEffect(() => {
// Don't initialize if no audioUrl or already initialized
if (!audioUrl) {
return;
}
if (!audioUrl) return;
if (wavesurferRef.current) return; // already created
if (wavesurferRef.current) {
debug.log('WaveSurfer already initialized, skipping');
return;
}
debug.log('Creating NEW WaveSurfer instance');
// Wait for container to be properly rendered
const initWaveSurfer = () => {
const container = waveformRef.current;
if (!container) {
// Container not ready yet, retry
setTimeout(initWaveSurfer, 50);
return;
}
// Check if container has dimensions and is visible
const rect = container.getBoundingClientRect();
const style = window.getComputedStyle(container);
const isVisible =
@@ -107,412 +96,221 @@ export function AudioPlayer() {
style.visibility !== 'hidden';
if (!isVisible) {
// Retry after a short delay
setTimeout(initWaveSurfer, 50);
return;
}
debug.log('Initializing WaveSurfer...', {
container,
debug.log('Creating WaveSurfer instance', {
width: rect.width,
height: rect.height,
});
try {
// Get computed CSS variable values
const root = document.documentElement;
const getCSSVar = (varName: string) => {
const value = getComputedStyle(root).getPropertyValue(varName).trim();
return value ? `hsl(${value})` : '';
};
const waveColor = getCSSVar('--muted');
const progressColor = getCSSVar('--accent');
const cursorColor = getCSSVar('--accent');
const wavesurfer = WaveSurfer.create({
container: container,
waveColor: waveColor,
progressColor: progressColor,
cursorColor: cursorColor,
container,
waveColor: getCSSVar('--muted'),
progressColor: getCSSVar('--accent'),
cursorColor: getCSSVar('--accent'),
cursorWidth: 3,
barWidth: 2,
barRadius: 2,
height: 80,
normalize: true,
// Use MediaElement backend (default). Unlike the WebAudio backend,
// MediaElement uses a standard <audio> element for playback which
// benefits from the browser/webview's built-in audio session recovery.
// This prevents audio loss when another app steals audio output or
// the system audio session is interrupted.
interact: true, // Enable interaction (click to seek)
mediaControls: false, // Don't show native controls
interact: true,
dragToSeek: { debounceTime: 0 },
mediaControls: false,
backend: 'WebAudio',
});
wavesurferRef.current = wavesurfer;
debug.log('WaveSurfer created successfully');
} catch (error) {
debug.error('Failed to create WaveSurfer:', error);
setError(
`Failed to initialize waveform: ${error instanceof Error ? error.message : String(error)}`,
);
return;
}
// Wire up event handlers (these persist for the lifetime of the instance)
wavesurfer.on('timeupdate', (time) => {
const dur = usePlayerStore.getState().duration;
if (dur > 0 && time >= dur) {
setCurrentTime(dur);
const loop = usePlayerStore.getState().isLooping;
if (loop) {
wavesurfer.seekTo(0);
wavesurfer.play().catch((err) => debug.error('Loop play failed:', err));
} else {
wavesurfer.pause();
setIsPlaying(false);
}
return;
}
setCurrentTime(time);
});
const wavesurfer = wavesurferRef.current;
if (!wavesurfer) return;
wavesurfer.on('ready', () => {
const dur = wavesurfer.getDuration();
setDuration(dur);
loadingRef.current = false;
setIsLoading(false);
setError(null);
debug.log('Audio ready, duration:', dur);
// Update store when time changes, stop if past duration
wavesurfer.on('timeupdate', (time) => {
const dur = usePlayerStore.getState().duration;
if (dur > 0 && time >= dur) {
setCurrentTime(dur);
wavesurfer.setVolume(usePlayerStore.getState().volume);
wavesurfer.setMuted(false);
// Auto-play if the flag is set (story mode advance or explicit play)
const shouldAutoPlayNow = usePlayerStore.getState().shouldAutoPlay;
if (shouldAutoPlayNow) {
usePlayerStore.getState().clearAutoPlayFlag();
wavesurfer.play().catch((err) => {
debug.error('Failed to autoplay:', err);
});
} else {
debug.log('Skipping auto-play - shouldAutoPlay is false');
}
});
wavesurfer.on('play', () => setIsPlaying(true));
wavesurfer.on('pause', () => {
setIsPlaying(false);
setCurrentTime(wavesurfer.getCurrentTime());
});
wavesurfer.on('seeking', (time) => setCurrentTime(time));
// Mute audio during drag-to-seek to prevent popping from the WebAudio
// backend's hard stop/start cycle on each seek. Unmute with a short
// fade-in when the drag ends.
const seekMedia = wavesurfer.getMediaElement() as any;
const seekGain: GainNode | null = seekMedia?.getGainNode?.() ?? null;
if (seekGain) {
const ctx = seekGain.context as AudioContext;
wavesurfer.on('dragstart', () => {
seekGain.gain.cancelScheduledValues(ctx.currentTime);
seekGain.gain.setTargetAtTime(0, ctx.currentTime, 0.002);
});
wavesurfer.on('dragend', () => {
seekGain.gain.cancelScheduledValues(ctx.currentTime);
seekGain.gain.setTargetAtTime(1, ctx.currentTime, 0.01);
});
}
wavesurfer.on('finish', () => {
const loop = usePlayerStore.getState().isLooping;
if (loop) {
wavesurfer.seekTo(0);
wavesurfer.play();
wavesurfer.play().catch((err) => debug.error('Loop play failed:', err));
} else {
wavesurfer.pause();
setIsPlaying(false);
const onFinish = usePlayerStore.getState().onFinish;
if (onFinish) onFinish();
}
return;
}
setCurrentTime(time);
});
// Update store when duration is loaded
wavesurfer.on('ready', async () => {
const dur = wavesurfer.getDuration();
setDuration(dur);
loadingRef.current = false;
setIsLoading(false);
setError(null);
debug.log('Audio ready, duration:', dur);
debug.log('Waveform should be visible now');
// Ensure volume is set
const currentVolume = usePlayerStore.getState().volume;
wavesurfer.setVolume(currentVolume);
// Auto-play when ready - check if we should use native playback
// Get current values from the store and queries at runtime (not captured closure values)
const currentAudioUrl = usePlayerStore.getState().audioUrl;
const currentProfileId = usePlayerStore.getState().profileId;
debug.log('Auto-play check - capturing runtime values...');
// Fetch profile channels at runtime (not using captured value)
let runtimeProfileChannels = null;
let runtimeChannels = null;
if (platform.metadata.isTauri && currentProfileId) {
try {
runtimeProfileChannels = await apiClient.getProfileChannels(currentProfileId);
debug.log('Runtime profileChannels:', runtimeProfileChannels);
if (runtimeProfileChannels && runtimeProfileChannels.channel_ids.length > 0) {
runtimeChannels = await apiClient.listChannels();
debug.log('Runtime channels:', runtimeChannels);
}
} catch (error) {
debug.error('Failed to fetch runtime channel data:', error);
}
}
debug.log('Auto-play check:', {
isTauri: platform.metadata.isTauri,
currentAudioUrl,
currentProfileId,
hasProfileChannels: !!runtimeProfileChannels,
hasChannels: !!runtimeChannels,
});
if (
platform.metadata.isTauri &&
currentAudioUrl &&
currentProfileId &&
runtimeProfileChannels &&
runtimeChannels
) {
debug.log('Attempting native audio playback...');
// Stop any existing native playback first
if (isUsingNativePlaybackRef.current) {
try {
platform.audio.stopPlayback();
debug.log('Stopped existing native playback before starting new one');
} catch (error) {
debug.error('Failed to stop existing playback:', error);
}
}
try {
// Collect all device IDs from assigned channels
const assignedChannels = runtimeChannels.filter((ch: any) =>
runtimeProfileChannels.channel_ids.includes(ch.id),
);
debug.log('Assigned channels for playback:', assignedChannels);
// Check if any assigned channel has non-default devices
const shouldUseNative = assignedChannels.some(
(ch: any) => ch.device_ids.length > 0 && !ch.is_default,
);
debug.log('Should use native playback:', shouldUseNative);
if (!shouldUseNative) {
debug.log('No custom devices assigned, using standard playback');
isUsingNativePlaybackRef.current = false;
} else {
const deviceIds = assignedChannels.flatMap((ch: any) => ch.device_ids);
debug.log('Device IDs to play to:', deviceIds);
if (deviceIds.length > 0) {
debug.log('Fetching audio data from:', currentAudioUrl);
// Fetch audio data
const response = await fetch(currentAudioUrl);
const audioData = new Uint8Array(await response.arrayBuffer());
debug.log('Audio data size:', audioData.length);
// Play via native audio
debug.log('Invoking play_audio_to_devices...');
try {
await platform.audio.playToDevices(audioData, deviceIds);
debug.log('play_audio_to_devices completed successfully');
// Mark that we're using native playback
isUsingNativePlaybackRef.current = true;
// Mute WaveSurfer's audio output — native handles the actual sound
// Keep WaveSurfer running for waveform visualization
wavesurfer.setVolume(0);
wavesurfer.setMuted(true);
// Start WaveSurfer playback for visualization (muted)
wavesurfer.play().catch((error) => {
debug.error('Failed to start WaveSurfer visualization:', error);
});
setIsPlaying(true);
debug.log('Auto-playing via native audio routing - SUCCESS');
return;
} catch (invokeError) {
debug.error('play_audio_to_devices invoke failed:', invokeError);
throw invokeError;
}
} else {
debug.log('No device IDs found, falling back to WaveSurfer');
}
}
} catch (error) {
debug.error(
'Native playback failed during auto-play, falling back to WaveSurfer:',
error,
);
isUsingNativePlaybackRef.current = false;
// Fall through to WaveSurfer playback
}
}
// Standard playback path — ensure WaveSurfer is unmuted
if (!isUsingNativePlaybackRef.current) {
wavesurfer.setMuted(false);
wavesurfer.setVolume(usePlayerStore.getState().volume);
}
// Only auto-play if shouldAutoPlay flag is set (user explicitly clicked to play)
const shouldAutoPlayNow = usePlayerStore.getState().shouldAutoPlay;
if (shouldAutoPlayNow) {
// Clear the flag first
usePlayerStore.getState().clearAutoPlayFlag();
// Use a small delay to ensure audio element is fully ready
setTimeout(() => {
wavesurfer.play().catch((error) => {
debug.error('Failed to autoplay:', error);
// Don't show error for autoplay failures (browser restrictions)
});
}, 100);
} else {
debug.log('Skipping auto-play - shouldAutoPlay is false');
}
});
// Handle play/pause
wavesurfer.on('play', () => {
setIsPlaying(true);
});
wavesurfer.on('pause', () => setIsPlaying(false));
wavesurfer.on('finish', () => {
// Check loop state from store
const loop = usePlayerStore.getState().isLooping;
if (loop) {
wavesurfer.seekTo(0);
wavesurfer.play();
} else {
setIsPlaying(false);
// Trigger finish callback if set
const onFinish = usePlayerStore.getState().onFinish;
if (onFinish) {
onFinish();
}
}
});
// Handle errors
wavesurfer.on('error', (error) => {
debug.error('WaveSurfer error:', error);
setIsLoading(false);
setError(`Audio error: ${error instanceof Error ? error.message : String(error)}`);
});
// Handle loading
wavesurfer.on('loading', (percent) => {
setIsLoading(true);
if (percent === 100) {
wavesurfer.on('error', (err) => {
debug.error('WaveSurfer error:', err);
setIsLoading(false);
}
});
setError(`Audio error: ${err instanceof Error ? err.message : String(err)}`);
});
// Load audio immediately if audioUrl is already set
if (audioUrl) {
debug.log('WaveSurfer ready, loading audio:', audioUrl);
loadingRef.current = true;
setIsLoading(true);
// Stop any current playback before loading new audio
if (wavesurfer.isPlaying()) {
wavesurfer.pause();
}
wavesurfer
.load(audioUrl)
.then(() => {
debug.log('Audio loaded into WaveSurfer');
loadingRef.current = false;
})
.catch((error) => {
debug.error('Failed to load audio into WaveSurfer:', error);
loadingRef.current = false;
setIsLoading(false);
setError(
`Failed to load audio: ${error instanceof Error ? error.message : String(error)}`,
);
});
wavesurfer.on('loading', (percent) => {
setIsLoading(true);
if (percent === 100) setIsLoading(false);
});
wavesurferRef.current = wavesurfer;
setWsReady(true);
debug.log('WaveSurfer created successfully');
} catch (err) {
debug.error('Failed to create WaveSurfer:', err);
setError(
`Failed to initialize waveform: ${err instanceof Error ? err.message : String(err)}`,
);
}
};
// Use double requestAnimationFrame to ensure DOM is fully rendered
let rafId1: number;
let rafId2: number;
let timeoutId: number | null = null;
rafId1 = requestAnimationFrame(() => {
rafId2 = requestAnimationFrame(() => {
// Add a small delay to ensure container is fully laid out
timeoutId = setTimeout(() => {
initWaveSurfer();
}, 10);
});
let rafId: number;
rafId = requestAnimationFrame(() => {
initWaveSurfer();
});
return () => {
debug.log('Cleaning up WaveSurfer initialization effect');
if (rafId1) cancelAnimationFrame(rafId1);
if (rafId2) cancelAnimationFrame(rafId2);
if (timeoutId) clearTimeout(timeoutId);
cancelAnimationFrame(rafId);
};
// Only run on mount-like conditions. audioUrl is here so we create the instance
// when the player first appears, but we guard against re-creation above.
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [audioUrl, setIsPlaying, setDuration, setCurrentTime]);
// Destroy WaveSurfer only on unmount
useEffect(() => {
return () => {
if (wavesurferRef.current) {
debug.log('Destroying WaveSurfer instance');
debug.log('Destroying WaveSurfer instance (unmount)');
try {
wavesurferRef.current.destroy();
} catch (error) {
debug.error('Error destroying WaveSurfer:', error);
} catch (err) {
debug.error('Error destroying WaveSurfer:', err);
}
wavesurferRef.current = null;
setWsReady(false);
}
};
}, [audioUrl, setIsPlaying, setCurrentTime, setDuration]);
}, []);
// Load audio when URL changes (only if WaveSurfer is already initialized)
// Load audio when URL changes (reuses the existing WaveSurfer instance)
useEffect(() => {
const wavesurfer = wavesurferRef.current;
if (!wavesurfer || !wsReady) return;
if (!audioUrl || !wavesurfer) {
// Reset state when no audio or WaveSurfer not ready
if (!audioUrl && wavesurfer) {
wavesurfer.pause();
wavesurfer.seekTo(0);
loadingRef.current = false;
setIsLoading(false);
setDuration(0);
setCurrentTime(0);
setError(null);
// Reset native playback flag
isUsingNativePlaybackRef.current = false;
}
if (!audioUrl) {
// No audio - pause and reset
wavesurfer.pause();
wavesurfer.seekTo(0);
loadingRef.current = false;
setIsLoading(false);
setDuration(0);
setCurrentTime(0);
setError(null);
isUsingNativePlaybackRef.current = false;
return;
}
// Stop native playback if it was active
if (isUsingNativePlaybackRef.current && platform.metadata.isTauri) {
try {
platform.audio.stopPlayback();
debug.log('Stopped native audio playback');
} catch (error) {
debug.error('Failed to stop native playback:', error);
}
}
// Reset native playback flag when loading new audio
// Unmute WaveSurfer if it was muted for native playback
if (isUsingNativePlaybackRef.current) {
wavesurfer.setMuted(false);
wavesurfer.setVolume(usePlayerStore.getState().volume);
}
// Reset native playback state
isUsingNativePlaybackRef.current = false;
wavesurfer.setMuted(false);
wavesurfer.setVolume(usePlayerStore.getState().volume);
// CRITICAL: Force stop any current playback and cancel any pending loads
// This must happen BEFORE any early returns
debug.log('Audio URL changed to:', audioUrl);
// COMPLETELY stop and destroy the current audio
// Stop current playback and reset position before loading new audio.
// With the WebAudio backend, pause() accumulates playedDuration internally.
// seekTo(0) resets it so the new track starts from the beginning.
debug.log('Loading new audio URL:', audioUrl);
try {
// First pause if playing
if (wavesurfer.isPlaying()) {
debug.log('Pausing current playback');
wavesurfer.pause();
}
// Use empty() to completely destroy the waveform and reset media
debug.log('Calling wavesurfer.empty() to destroy audio');
wavesurfer.empty();
} catch (error) {
debug.error('Error stopping previous audio:', error);
// Continue anyway to load new audio
wavesurfer.seekTo(0);
} catch (err) {
debug.error('Error resetting before load:', err);
}
// Reset loading state to allow new load (cancel any pending loads)
loadingRef.current = false;
// Now start the new load
loadingRef.current = true;
setIsLoading(true);
setError(null);
setCurrentTime(0);
setDuration(0);
// Load new audio
debug.log('Starting new audio load for:', audioUrl);
wavesurfer
.load(audioUrl)
.then(() => {
debug.log('Audio load promise resolved');
// Don't set loading to false here - wait for 'ready' event
debug.log('Audio loaded into WaveSurfer');
loadingRef.current = false;
})
.catch((error) => {
debug.error('Failed to load audio:', error);
debug.error('Audio URL:', audioUrl);
.catch((err) => {
debug.error('Failed to load audio:', err);
loadingRef.current = false;
setIsLoading(false);
setError(`Failed to load audio: ${error instanceof Error ? error.message : String(error)}`);
setError(`Failed to load audio: ${err instanceof Error ? err.message : String(err)}`);
});
}, [audioUrl, setCurrentTime, setDuration]);
}, [audioUrl, wsReady, setCurrentTime, setDuration]);
// Sync play/pause state (only when user clicks play/pause button, not auto-sync)
// This effect is kept for external state changes but should be minimal
@@ -520,7 +318,6 @@ export function AudioPlayer() {
if (!wavesurferRef.current || duration === 0) return;
if (isPlaying && wavesurferRef.current.isPlaying() === false) {
// Only auto-play if audio is ready
wavesurferRef.current.play().catch((error) => {
debug.error('Failed to play:', error);
setIsPlaying(false);
@@ -534,14 +331,7 @@ export function AudioPlayer() {
// Sync volume
useEffect(() => {
if (wavesurferRef.current) {
// If using native playback, keep WaveSurfer muted regardless of volume setting
if (isUsingNativePlaybackRef.current) {
wavesurferRef.current.setVolume(0);
debug.log('Volume sync: Using native playback, keeping WaveSurfer muted');
} else {
wavesurferRef.current.setVolume(volume);
debug.log('Volume synced:', volume);
}
wavesurferRef.current.setVolume(volume);
}
}, [volume]);
@@ -566,7 +356,6 @@ export function AudioPlayer() {
return;
}
// Reset to beginning and play
debug.log('Restarting current audio from beginning');
wavesurfer.seekTo(0);
wavesurfer.play().catch((error) => {
@@ -575,34 +364,35 @@ export function AudioPlayer() {
setError(`Playback error: ${error instanceof Error ? error.message : String(error)}`);
});
// Clear the restart flag
clearRestartFlag();
}, [shouldRestart, duration, setIsPlaying, clearRestartFlag]);
// Handle shouldAutoPlay flag - for story mode auto-advance
const shouldAutoPlay = usePlayerStore((state) => state.shouldAutoPlay);
const clearAutoPlayFlag = usePlayerStore((state) => state.clearAutoPlayFlag);
// Auto-play is handled exclusively in the WaveSurfer 'ready' event handler.
// A separate effect here would race with the ready event since the WebAudio
// backend needs to fully decode the audio before play() works correctly.
// Spacebar to play/pause (capture phase so it fires before focused elements)
useEffect(() => {
const wavesurfer = wavesurferRef.current;
if (!wavesurfer || !shouldAutoPlay || duration === 0) {
return;
}
// Auto-play the newly loaded audio
debug.log('Auto-playing next track in story mode');
wavesurfer.seekTo(0);
wavesurfer.play().catch((error) => {
debug.error('Failed to auto-play:', error);
setIsPlaying(false);
setError(`Playback error: ${error instanceof Error ? error.message : String(error)}`);
});
// Clear the auto-play flag
clearAutoPlayFlag();
}, [shouldAutoPlay, duration, setIsPlaying, clearAutoPlayFlag]);
// Handle loop - WaveSurfer handles this via the 'finish' event
const onKeyDown = (e: KeyboardEvent) => {
if (e.code !== 'Space') return;
// Ignore if user is typing in an input/textarea
const tag = (e.target as HTMLElement)?.tagName;
if (tag === 'INPUT' || tag === 'TEXTAREA' || (e.target as HTMLElement)?.isContentEditable) {
return;
}
if (audioUrl && duration > 0 && wavesurferRef.current) {
e.preventDefault();
e.stopPropagation();
if (wavesurferRef.current.isPlaying()) {
wavesurferRef.current.pause();
} else {
wavesurferRef.current.play().catch((err) => debug.error('Spacebar play failed:', err));
}
}
};
document.addEventListener('keydown', onKeyDown, true);
return () => document.removeEventListener('keydown', onKeyDown, true);
}, [audioUrl, duration]);
const handlePlayPause = async () => {
// Standard WaveSurfer playback (works for both normal and native playback modes)
@@ -741,32 +531,32 @@ export function AudioPlayer() {
size="icon"
onClick={handlePlayPause}
disabled={isLoading || duration === 0}
className="shrink-0"
className={`shrink-0 -mt-2 ${isPlaying ? 'bg-accent text-accent-foreground' : ''}`}
title={duration === 0 && !isLoading ? 'Audio not loaded' : ''}
aria-label={
duration === 0 && !isLoading ? 'Audio not loaded' : isPlaying ? 'Pause' : 'Play'
}
>
{isPlaying ? <Pause className="h-5 w-5" /> : <Play className="h-5 w-5" />}
{isPlaying ? (
<Pause className="h-5 w-5 fill-current" />
) : (
<Play className="h-5 w-5 fill-current" />
)}
</Button>
{/* Waveform */}
<div className="flex-1 min-w-0 flex flex-col gap-1">
<div ref={waveformRef} className="w-full min-h-[80px]" />
{duration > 0 && (
<Slider
value={duration > 0 ? [(currentTime / duration) * 100] : [0]}
onValueChange={handleSeek}
max={100}
step={0.1}
className="w-full"
aria-label="Playback position"
aria-valuetext={`${formatAudioDuration(currentTime)} of ${formatAudioDuration(duration)}`}
/>
)}
{isLoading && (
<div className="text-xs text-muted-foreground text-center py-2">Loading audio...</div>
)}
<div ref={waveformRef} className="w-full min-h-[80px] select-none" />
<Slider
value={duration > 0 ? [(currentTime / duration) * 100] : [0]}
onValueChange={handleSeek}
max={100}
step={0.1}
className="w-full"
aria-label="Playback position"
aria-valuetext={`${formatAudioDuration(currentTime)} of ${formatAudioDuration(duration)}`}
/>
{error && <div className="text-xs text-destructive text-center py-2">{error}</div>}
</div>
@@ -777,19 +567,12 @@ export function AudioPlayer() {
<span className="font-mono">{formatAudioDuration(duration)}</span>
</div>
{/* Title */}
{title && (
<div className="text-sm font-medium truncate max-w-[200px] shrink-0 hidden lg:block">
{title}
</div>
)}
{/* Loop Button */}
<Button
variant="ghost"
size="icon"
onClick={toggleLoop}
className={isLooping ? 'text-primary' : ''}
className={isLooping ? 'bg-accent text-accent-foreground' : ''}
title="Toggle loop"
aria-label={isLooping ? 'Stop looping' : 'Loop'}
>
@@ -20,6 +20,8 @@ const ENGINE_OPTIONS = [
{ value: 'luxtts', label: 'LuxTTS' },
{ value: 'chatterbox', label: 'Chatterbox' },
{ value: 'chatterbox_turbo', label: 'Chatterbox Turbo' },
{ value: 'tada:1B', label: 'TADA 1B' },
{ value: 'tada:3B', label: 'TADA 3B Multilingual' },
] as const;
const ENGINE_DESCRIPTIONS: Record<string, string> = {
@@ -27,6 +29,7 @@ const ENGINE_DESCRIPTIONS: Record<string, string> = {
luxtts: 'Fast, English-focused',
chatterbox: '23 languages, incl. Hebrew',
chatterbox_turbo: 'English, [laugh] [cough] tags',
tada: 'HumeAI, 700s+ coherent audio',
};
/** Engines that only support English and should force language to 'en' on select. */
@@ -34,6 +37,7 @@ const ENGLISH_ONLY_ENGINES = new Set(['luxtts', 'chatterbox_turbo']);
function getSelectValue(engine: string, modelSize?: string): string {
if (engine === 'qwen') return `qwen:${modelSize || '1.7B'}`;
if (engine === 'tada') return `tada:${modelSize || '1B'}`;
return engine;
}
@@ -48,6 +52,20 @@ function handleEngineChange(form: UseFormReturn<GenerationFormValues>, value: st
if (!available.some((l) => l.value === currentLang)) {
form.setValue('language', available[0]?.value ?? 'en');
}
} else if (value.startsWith('tada:')) {
const [, modelSize] = value.split(':');
form.setValue('engine', 'tada');
form.setValue('modelSize', modelSize as '1B' | '3B');
// TADA 1B is English-only; 3B is multilingual
if (modelSize === '1B') {
form.setValue('language', 'en');
} else {
const currentLang = form.getValues('language');
const available = getLanguageOptionsForEngine('tada');
if (!available.some((l) => l.value === currentLang)) {
form.setValue('language', available[0]?.value ?? 'en');
}
}
} else {
form.setValue('engine', value as GenerationFormValues['engine']);
form.setValue('modelSize', undefined as unknown as '1.7B' | '0.6B');
+50 -11
View File
@@ -15,7 +15,7 @@ import {
Wand2,
} from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import Loader from 'react-loaders';
import { EffectsChainEditor } from '@/components/Effects/EffectsChainEditor';
import { Button } from '@/components/ui/button';
import {
@@ -56,8 +56,35 @@ import { formatDate, formatDuration, formatEngineName } from '@/lib/utils/format
import { useGenerationStore } from '@/stores/generationStore';
import { usePlayerStore } from '@/stores/playerStore';
// OLD TABLE-BASED COMPONENT - REMOVED (can be found in git history)
// This is the new alternate history view with fixed height rows
// ─── Audio Bars ─────────────────────────────────────────────────────────────
function AudioBars({ mode }: { mode: 'idle' | 'generating' | 'playing' }) {
const barColor = mode !== 'idle' ? 'bg-accent' : 'bg-muted-foreground/40';
return (
<div className="flex items-center gap-[2px] h-5">
{[0, 1, 2, 3, 4].map((i) => (
<motion.div
key={`${mode}-${i}`}
className={`w-[3px] rounded-full ${barColor}`}
animate={
mode === 'generating'
? { height: ['6px', '16px', '6px'] }
: mode === 'playing'
? { height: ['8px', '14px', '4px', '12px', '8px'] }
: { height: '8px' }
}
transition={
mode === 'generating'
? { duration: 0.6, repeat: Infinity, delay: i * 0.08, ease: 'easeInOut' }
: mode === 'playing'
? { duration: 1.2, repeat: Infinity, delay: i * 0.15, ease: 'easeInOut' }
: { duration: 0.4, ease: 'easeOut' }
}
/>
))}
</div>
);
}
// NEW ALTERNATE HISTORY VIEW - FIXED HEIGHT ROWS WITH INFINITE SCROLL
export function HistoryTable() {
@@ -126,7 +153,9 @@ export function HistoryTable() {
}
}, [historyData, page]);
// Reset to page 0 when deletions or imports occur
// Reset to page 0 when deletions, imports, or generation completions occur
const pendingCount = useGenerationStore((state) => state.pendingGenerationIds.size);
const prevPendingCountRef = useRef(pendingCount);
useEffect(() => {
if (deleteGeneration.isSuccess || importGeneration.isSuccess) {
setPage(0);
@@ -134,6 +163,19 @@ export function HistoryTable() {
}
}, [deleteGeneration.isSuccess, importGeneration.isSuccess]);
useEffect(() => {
// A generation finished (pending count decreased) — scroll back to show it
if (
prevPendingCountRef.current > 0 &&
pendingCount < prevPendingCountRef.current &&
page !== 0
) {
setPage(0);
setAllHistory([]);
}
prevPendingCountRef.current = pendingCount;
}, [pendingCount, page]);
// Intersection Observer for infinite scroll
useEffect(() => {
const loadMoreEl = loadMoreRef.current;
@@ -413,7 +455,7 @@ export function HistoryTable() {
role={isPlayable ? 'button' : undefined}
tabIndex={isPlayable ? 0 : undefined}
className={cn(
'flex items-stretch gap-4 h-26 p-3',
'flex items-stretch gap-4 h-26 p-3 outline-none',
isPlayable && 'hover:bg-muted/70 cursor-pointer rounded-md',
isVersionsExpanded && 'rounded-b-none',
)}
@@ -446,12 +488,9 @@ export function HistoryTable() {
>
{/* Status icon */}
<div className="flex items-center shrink-0 w-10 justify-center overflow-hidden">
<div className="scale-50">
<Loader
type={isGenerating ? 'line-scale' : 'line-scale-pulse-out-rapid'}
active={isGenerating || isCurrentlyPlaying}
/>
</div>
<AudioBars
mode={isGenerating ? 'generating' : isCurrentlyPlaying ? 'playing' : 'idle'}
/>
</div>
{/* Left side - Meta information */}
@@ -243,7 +243,40 @@ export function GpuAcceleration() {
{/* Native GPU detected - no CUDA download needed */}
{/* CUDA download section - only show when no GPU is active (native or CUDA) */}
{/* Currently running CUDA - show switch back to CPU */}
{isCurrentlyCuda && platform.metadata.isTauri && (
<>
{restartPhase !== 'idle' ? (
<div className="flex items-center gap-2 p-3 rounded-lg bg-primary/5 border">
<Loader2 className="h-4 w-4 animate-spin" />
<span className="text-sm">
{restartPhase === 'stopping' && 'Stopping server...'}
{restartPhase === 'waiting' && 'Restarting server...'}
{restartPhase === 'ready' && 'Server restarted successfully!'}
</span>
</div>
) : (
<div className="space-y-3">
<p className="text-sm text-muted-foreground">
Running with CUDA GPU acceleration. Switch back to CPU if needed (you can
re-download later).
</p>
<Button onClick={handleSwitchToCpu} variant="outline" className="w-full" size="sm">
<RotateCw className="h-4 w-4 mr-2" />
Switch to CPU Backend
</Button>
</div>
)}
{error && (
<div className="flex items-center gap-2 text-sm text-destructive">
<AlertCircle className="h-4 w-4 shrink-0" />
<span>{error}</span>
</div>
)}
</>
)}
{/* CUDA download/manage section - show when no native GPU and not currently running CUDA */}
{!hasNativeGpu && !isCurrentlyCuda && (
<>
{/* Download progress (manual download or auto-update) */}
@@ -315,7 +348,7 @@ export function GpuAcceleration() {
)}
{/* Downloaded but not active - show switch button */}
{cudaAvailable && !isCurrentlyCuda && platform.metadata.isTauri && (
{cudaAvailable && platform.metadata.isTauri && (
<div className="space-y-3">
<p className="text-sm text-muted-foreground">
CUDA backend is downloaded and ready. Restart the server to enable GPU
@@ -328,27 +361,8 @@ export function GpuAcceleration() {
</div>
)}
{/* Currently active - show switch back to CPU */}
{isCurrentlyCuda && platform.metadata.isTauri && (
<div className="space-y-3">
<p className="text-sm text-muted-foreground">
Running with CUDA GPU acceleration. Switch back to CPU if needed (you can
re-download later).
</p>
<Button
onClick={handleSwitchToCpu}
variant="outline"
className="w-full"
size="sm"
>
<RotateCw className="h-4 w-4 mr-2" />
Switch to CPU Backend
</Button>
</div>
)}
{/* Delete option when downloaded (and not active) */}
{cudaAvailable && !isCurrentlyCuda && (
{cudaAvailable && (
<Button
onClick={handleDelete}
variant="ghost"
@@ -62,6 +62,10 @@ const MODEL_DESCRIPTIONS: Record<string, string> = {
'Production-grade open source TTS by Resemble AI. Supports 23 languages with voice cloning and emotion exaggeration control.',
'chatterbox-turbo':
'Streamlined 350M parameter TTS by Resemble AI. High-quality English speech with less compute and VRAM than larger models.',
'tada-1b':
'HumeAI TADA 1B — English speech-language model built on Llama 3.2 1B. Generates 700s+ of coherent audio with synchronized text-acoustic alignment.',
'tada-3b-ml':
'HumeAI TADA 3B Multilingual — built on Llama 3.2 3B. Supports 10 languages with high-fidelity voice cloning via text-acoustic dual alignment.',
'whisper-base':
'Smallest Whisper model (74M parameters). Fast transcription with moderate accuracy.',
'whisper-small':
@@ -391,7 +395,8 @@ export function ModelManagement() {
(m) =>
m.model_name.startsWith('qwen-tts') ||
m.model_name.startsWith('luxtts') ||
m.model_name.startsWith('chatterbox'),
m.model_name.startsWith('chatterbox') ||
m.model_name.startsWith('tada'),
) ?? [];
const whisperModels = modelStatus?.models.filter((m) => m.model_name.startsWith('whisper')) ?? [];
+135
View File
@@ -0,0 +1,135 @@
import { ArrowUpRight } from 'lucide-react';
import type { CSSProperties, ReactNode } from 'react';
import { useEffect, useState } from 'react';
import voiceboxLogo from '@/assets/voicebox-logo.png';
import { usePlatform } from '@/platform/PlatformContext';
function FadeIn({ delay = 0, children }: { delay?: number; children: ReactNode }) {
return (
<div
className="animate-[fadeInUp_0.5s_ease_both]"
style={{ animationDelay: `${delay}ms` } as CSSProperties}
>
{children}
</div>
);
}
export function AboutPage() {
const platform = usePlatform();
const [version, setVersion] = useState('');
useEffect(() => {
platform.metadata
.getVersion()
.then(setVersion)
.catch(() => setVersion(''));
}, [platform]);
return (
<>
<style>{`
@keyframes fadeInUp {
from {
opacity: 0;
transform: translateY(8px);
}
to {
opacity: 1;
transform: translateY(0);
}
}
`}</style>
<div className="max-w-md mx-auto h-full flex items-center">
<div className="flex flex-col items-center text-center space-y-5">
<FadeIn delay={0}>
<img src={voiceboxLogo} alt="Voicebox" className="w-20 h-20 object-contain" />
</FadeIn>
<FadeIn delay={80}>
<div className="space-y-1.5">
<h1 className="text-lg font-semibold">Voicebox</h1>
<p className="text-xs text-muted-foreground/60 h-4">
{version ? `v${version}` : '\u00A0'}
</p>
</div>
</FadeIn>
<FadeIn delay={160}>
<p className="text-sm text-muted-foreground leading-relaxed max-w-sm">
The open-source voice synthesis studio. Clone voices, generate speech, apply effects,
and build voice-powered apps — all running locally on your machine.
</p>
</FadeIn>
<FadeIn delay={240}>
<div className="flex items-center gap-1.5 text-sm text-muted-foreground">
<span>Created by</span>
<a
href="https://github.com/jamiepine"
target="_blank"
rel="noopener noreferrer"
className="text-accent hover:underline"
>
Jamie Pine
</a>
</div>
</FadeIn>
<FadeIn delay={320}>
<div className="flex flex-wrap justify-center gap-3 pt-2">
<a
href="https://buymeacoffee.com/jamiepine"
target="_blank"
rel="noopener noreferrer"
className="group inline-flex items-center gap-2 rounded-lg border border-border/60 px-4 py-2 text-sm transition-colors hover:bg-muted/50"
>
<svg
className="h-4 w-4 text-[#FFDD00]"
viewBox="0 0 24 24"
fill="currentColor"
aria-hidden="true"
>
<path d="m20.216 6.415-.132-.666c-.119-.598-.388-1.163-1.001-1.379-.197-.069-.42-.098-.57-.241-.152-.143-.196-.366-.231-.572-.065-.378-.125-.756-.192-1.133-.057-.325-.102-.69-.25-.987-.195-.4-.597-.634-.996-.788a5.723 5.723 0 0 0-.626-.194c-1-.263-2.05-.36-3.077-.416a25.834 25.834 0 0 0-3.7.062c-.915.083-1.88.184-2.75.5-.318.116-.646.256-.888.501-.297.302-.393.77-.177 1.146.154.267.415.456.692.58.36.162.737.284 1.123.366 1.075.238 2.189.331 3.287.37 1.218.05 2.437.01 3.65-.118.299-.033.598-.073.896-.119.352-.054.578-.513.474-.834-.124-.383-.457-.531-.834-.473-.466.074-.96.108-1.382.146-1.177.08-2.358.082-3.536.006a22.228 22.228 0 0 1-1.157-.107c-.086-.01-.18-.025-.258-.036-.243-.036-.484-.08-.724-.13-.111-.027-.111-.185 0-.212h.005c.277-.06.557-.108.838-.147h.002c.131-.009.263-.032.394-.048a25.076 25.076 0 0 1 3.426-.12c.674.019 1.347.067 2.017.144l.228.031c.267.04.533.088.798.145.392.085.895.113 1.07.542.055.137.08.288.111.431l.319 1.484a.237.237 0 0 1-.199.284h-.003c-.037.006-.075.01-.112.015a36.704 36.704 0 0 1-4.743.295 37.059 37.059 0 0 1-4.699-.304c-.14-.017-.293-.042-.417-.06-.326-.048-.649-.108-.973-.161-.393-.065-.768-.032-1.123.161-.29.16-.527.404-.675.701-.154.316-.199.66-.267 1-.069.34-.176.707-.135 1.056.087.753.613 1.365 1.37 1.502a39.69 39.69 0 0 0 11.343.376.483.483 0 0 1 .535.53l-.071.697-1.018 9.907c-.041.41-.047.832-.125 1.237-.122.637-.553 1.028-1.182 1.171-.577.131-1.165.2-1.756.205-.656.004-1.31-.025-1.966-.022-.699.004-1.556-.06-2.095-.58-.475-.458-.54-1.174-.605-1.793l-.731-7.013-.322-3.094c-.037-.351-.286-.695-.678-.678-.336.015-.718.3-.678.679l.228 2.185.949 9.112c.147 1.344 1.174 2.068 2.446 2.272.742.12 1.503.144 2.257.156.966.016 1.942.053 2.892-.122 1.408-.258 2.465-1.198 2.616-2.657.34-3.332.683-6.663 1.024-9.995l.215-2.087a.484.484 0 0 1 .39-.426c.402-.078.787-.212 1.074-.518.455-.488.546-1.124.385-1.766zm-1.478.772c-.145.137-.363.201-.578.233-2.416.359-4.866.54-7.308.46-1.748-.06-3.477-.254-5.207-.498-.17-.024-.353-.055-.47-.18-.22-.236-.111-.71-.054-.995.052-.26.152-.609.463-.646.484-.057 1.046.148 1.526.22.577.088 1.156.159 1.737.212 2.48.226 5.002.19 7.472-.14.45-.06.899-.13 1.345-.21.399-.072.84-.206 1.08.206.166.281.188.657.162.974a.544.544 0 0 1-.169.364zm-6.159 3.9c-.862.37-1.84.788-3.109.788a5.884 5.884 0 0 1-1.569-.217l.877 9.004c.065.78.717 1.38 1.5 1.38 0 0 1.243.065 1.658.065.447 0 1.786-.065 1.786-.065.783 0 1.434-.6 1.499-1.38l.94-9.95a3.996 3.996 0 0 0-1.322-.238c-.826 0-1.491.284-2.26.613z" />
</svg>
Buy me a coffee
<ArrowUpRight className="h-3.5 w-3.5 text-muted-foreground/40 group-hover:text-muted-foreground transition-colors" />
</a>
<a
href="https://github.com/jamiepine/voicebox"
target="_blank"
rel="noopener noreferrer"
className="group inline-flex items-center gap-2 rounded-lg border border-border/60 px-4 py-2 text-sm transition-colors hover:bg-muted/50"
>
<svg
className="h-4 w-4 text-muted-foreground"
viewBox="0 0 24 24"
fill="currentColor"
aria-hidden="true"
>
<path d="M12 2C6.477 2 2 6.484 2 12.017c0 4.425 2.865 8.18 6.839 9.504.5.092.682-.217.682-.483 0-.237-.008-.868-.013-1.703-2.782.605-3.369-1.343-3.369-1.343-.454-1.158-1.11-1.466-1.11-1.466-.908-.62.069-.608.069-.608 1.003.07 1.531 1.032 1.531 1.032.892 1.53 2.341 1.088 2.91.832.092-.647.35-1.088.636-1.338-2.22-.253-4.555-1.113-4.555-4.951 0-1.093.39-1.988 1.029-2.688-.103-.253-.446-1.272.098-2.65 0 0 .84-.27 2.75 1.026A9.564 9.564 0 0112 6.844c.85.004 1.705.115 2.504.337 1.909-1.296 2.747-1.027 2.747-1.027.546 1.379.202 2.398.1 2.651.64.7 1.028 1.595 1.028 2.688 0 3.848-2.339 4.695-4.566 4.943.359.309.678.92.678 1.855 0 1.338-.012 2.419-.012 2.747 0 .268.18.58.688.482A10.019 10.019 0 0022 12.017C22 6.484 17.522 2 12 2z" />
</svg>
GitHub
<ArrowUpRight className="h-3.5 w-3.5 text-muted-foreground/40 group-hover:text-muted-foreground transition-colors" />
</a>
</div>
</FadeIn>
<FadeIn delay={400}>
<p className="text-xs text-muted-foreground/40 pt-4">
Licensed under{' '}
<a
href="https://github.com/jamiepine/voicebox/blob/main/LICENSE"
target="_blank"
rel="noopener noreferrer"
className="hover:text-muted-foreground/60 transition-colors"
>
MIT
</a>
</p>
</FadeIn>
</div>
</div>
</>
);
}
@@ -0,0 +1,220 @@
import changelogRaw from 'virtual:changelog';
import { useMemo, useState } from 'react';
import { Badge } from '@/components/ui/badge';
import { type ChangelogEntry, parseChangelog } from '@/lib/utils/parseChangelog';
function renderMarkdown(md: string): React.ReactNode[] {
const lines = md.split('\n');
const elements: React.ReactNode[] = [];
let i = 0;
while (i < lines.length) {
const line = lines[i];
// Skip empty lines
if (line.trim() === '') {
i++;
continue;
}
// Tables — collect all lines starting with |
if (line.trim().startsWith('|')) {
const tableLines: string[] = [];
while (i < lines.length && lines[i].trim().startsWith('|')) {
tableLines.push(lines[i]);
i++;
}
elements.push(renderTable(tableLines, elements.length));
continue;
}
// Headings
if (line.startsWith('#### ')) {
elements.push(
<h5 key={elements.length} className="text-sm font-medium mt-5 mb-1">
{inlineMarkdown(line.slice(5))}
</h5>,
);
i++;
continue;
}
if (line.startsWith('### ')) {
elements.push(
<h4 key={elements.length} className="text-sm font-medium mt-6 mb-2">
{inlineMarkdown(line.slice(4))}
</h4>,
);
i++;
continue;
}
// List items — collect consecutive
if (line.startsWith('- ')) {
const items: string[] = [];
while (i < lines.length && lines[i].startsWith('- ')) {
items.push(lines[i].slice(2));
i++;
}
elements.push(
<ul key={elements.length} className="space-y-1 my-2">
{items.map((item, idx) => (
<li key={idx} className="text-sm text-muted-foreground flex gap-2">
<span className="text-muted-foreground/50 select-none shrink-0">&bull;</span>
<span>{inlineMarkdown(item)}</span>
</li>
))}
</ul>,
);
continue;
}
// Paragraph
elements.push(
<p key={elements.length} className="text-sm text-muted-foreground my-2">
{inlineMarkdown(line)}
</p>,
);
i++;
}
return elements;
}
function renderTable(tableLines: string[], keyBase: number): React.ReactNode {
const parseRow = (line: string) =>
line
.split('|')
.slice(1, -1)
.map((c) => c.trim());
const headers = parseRow(tableLines[0]);
// Skip separator line (index 1)
const rows = tableLines.slice(2).map(parseRow);
return (
<div key={keyBase} className="overflow-x-auto my-3">
<table className="text-sm w-full">
<thead>
<tr className="border-b">
{headers.map((h, hIdx) => (
<th
key={hIdx}
className="text-left py-1.5 pr-4 text-muted-foreground font-medium text-xs"
>
{inlineMarkdown(h)}
</th>
))}
</tr>
</thead>
<tbody>
{rows.map((row, rowIdx) => (
<tr key={rowIdx} className="border-b border-border/50">
{row.map((cell, cellIdx) => (
<td key={cellIdx} className="py-1.5 pr-4 text-muted-foreground">
{inlineMarkdown(cell)}
</td>
))}
</tr>
))}
</tbody>
</table>
</div>
);
}
function inlineMarkdown(text: string): React.ReactNode {
// Process inline markdown: bold, code, links
const parts: React.ReactNode[] = [];
// Regex matches: **bold**, `code`, [text](url)
const inlineRe = /\*\*(.+?)\*\*|`([^`]+)`|\[([^\]]+)\]\(([^)]+)\)/g;
let lastIndex = 0;
let match: RegExpExecArray | null = inlineRe.exec(text);
while (match !== null) {
if (match.index > lastIndex) {
parts.push(text.slice(lastIndex, match.index));
}
if (match[1] !== undefined) {
// Bold
parts.push(
<strong key={parts.length} className="font-medium text-foreground">
{match[1]}
</strong>,
);
} else if (match[2] !== undefined) {
// Code
parts.push(
<code key={parts.length} className="px-1 py-0.5 rounded bg-muted text-xs font-mono">
{match[2]}
</code>,
);
} else if (match[3] !== undefined && match[4] !== undefined) {
// Link
parts.push(
<a
key={parts.length}
href={match[4]}
target="_blank"
rel="noopener noreferrer"
className="text-accent hover:underline"
>
{match[3]}
</a>,
);
}
lastIndex = match.index + match[0].length;
match = inlineRe.exec(text);
}
if (lastIndex < text.length) {
parts.push(text.slice(lastIndex));
}
return parts.length === 1 ? parts[0] : parts;
}
function ChangelogEntryCard({ entry }: { entry: ChangelogEntry }) {
const [expanded, setExpanded] = useState(false);
const content = useMemo(() => renderMarkdown(entry.body), [entry.body]);
const isLong = entry.body.split('\n').length > 12;
return (
<div className="border-b border-border/50 pb-6">
<div className="flex items-baseline gap-3 mb-1">
<h3 className="text-sm font-medium">{entry.version}</h3>
{entry.date && <span className="text-xs text-muted-foreground">{entry.date}</span>}
{entry.version === 'Unreleased' && <Badge variant="outline">dev</Badge>}
</div>
<div className={isLong && !expanded ? 'max-h-48 overflow-hidden relative' : ''}>
{content}
{isLong && !expanded && (
<div className="absolute bottom-0 left-0 right-0 h-16 bg-gradient-to-t from-background to-transparent" />
)}
</div>
{isLong && (
<button
onClick={() => setExpanded(!expanded)}
className="text-xs text-accent hover:underline mt-2"
>
{expanded ? 'Show less' : 'Show more'}
</button>
)}
</div>
);
}
export function ChangelogPage() {
const entries = useMemo(() => parseChangelog(changelogRaw), []);
return (
<div className="space-y-6 max-w-2xl">
{entries.map((entry) => (
<ChangelogEntryCard key={entry.version} entry={entry} />
))}
</div>
);
}
@@ -0,0 +1,379 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { AlertCircle, ArrowUpRight, Book, Download, Loader2, RefreshCw } from 'lucide-react';
import { useEffect, useState } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { Button } from '@/components/ui/button';
import { Form, FormControl, FormField, FormItem, FormMessage } from '@/components/ui/form';
import { Input } from '@/components/ui/input';
import { Progress } from '@/components/ui/progress';
import { Toggle } from '@/components/ui/toggle';
import { useToast } from '@/components/ui/use-toast';
import { useAutoUpdater } from '@/hooks/useAutoUpdater';
import { useServerHealth } from '@/lib/hooks/useServer';
import { usePlatform } from '@/platform/PlatformContext';
import { useServerStore } from '@/stores/serverStore';
import { SettingRow, SettingSection } from './SettingRow';
const connectionSchema = z.object({
serverUrl: z.string().url('Please enter a valid URL'),
});
type ConnectionFormValues = z.infer<typeof connectionSchema>;
export function GeneralPage() {
const platform = usePlatform();
const serverUrl = useServerStore((state) => state.serverUrl);
const setServerUrl = useServerStore((state) => state.setServerUrl);
const keepServerRunningOnClose = useServerStore((state) => state.keepServerRunningOnClose);
const setKeepServerRunningOnClose = useServerStore((state) => state.setKeepServerRunningOnClose);
const mode = useServerStore((state) => state.mode);
const setMode = useServerStore((state) => state.setMode);
const { toast } = useToast();
const { data: health, isLoading, error: healthError } = useServerHealth();
const form = useForm<ConnectionFormValues>({
resolver: zodResolver(connectionSchema),
defaultValues: { serverUrl },
});
useEffect(() => {
form.reset({ serverUrl });
}, [serverUrl, form]);
const { isDirty } = form.formState;
function onSubmit(data: ConnectionFormValues) {
setServerUrl(data.serverUrl);
form.reset(data);
toast({
title: 'Server URL updated',
description: `Connected to ${data.serverUrl}`,
});
}
return (
<div className="space-y-8 max-w-2xl">
<div className="grid grid-cols-2 gap-3">
<a
href="https://docs.voicebox.sh"
target="_blank"
rel="noopener noreferrer"
className="group flex items-center gap-3 rounded-lg border border-border/60 p-4 transition-colors hover:bg-muted/50"
>
<Book className="h-5 w-5 shrink-0 text-accent" strokeWidth={2.5} />
<div className="min-w-0 flex-1">
<div className="text-sm font-medium">Read the Docs</div>
<div className="text-xs text-muted-foreground">docs.voicebox.sh</div>
</div>
<ArrowUpRight className="h-4 w-4 text-muted-foreground/40 group-hover:text-muted-foreground transition-colors" />
</a>
<a
href="https://discord.gg/StkzQasqPS"
target="_blank"
rel="noopener noreferrer"
className="group flex items-center gap-3 rounded-lg border border-border/60 p-4 transition-colors hover:bg-muted/50"
>
<svg
className="h-5 w-5 shrink-0 text-accent"
viewBox="0 0 24 24"
fill="currentColor"
aria-hidden="true"
>
<path d="M20.317 4.37a19.791 19.791 0 0 0-4.885-1.515.074.074 0 0 0-.079.037c-.21.375-.444.864-.608 1.25a18.27 18.27 0 0 0-5.487 0 12.64 12.64 0 0 0-.617-1.25.077.077 0 0 0-.079-.037A19.736 19.736 0 0 0 3.677 4.37a.07.07 0 0 0-.032.027C.533 9.046-.32 13.58.099 18.057a.082.082 0 0 0 .031.057 19.9 19.9 0 0 0 5.993 3.03.078.078 0 0 0 .084-.028c.462-.63.874-1.295 1.226-1.994a.076.076 0 0 0-.041-.106 13.107 13.107 0 0 1-1.872-.892.077.077 0 0 1-.008-.128 10.2 10.2 0 0 0 .372-.292.074.074 0 0 1 .077-.01c3.928 1.793 8.18 1.793 12.062 0a.074.074 0 0 1 .078.01c.12.098.246.198.373.292a.077.077 0 0 1-.006.127 12.299 12.299 0 0 1-1.873.892.077.077 0 0 0-.041.107c.36.698.772 1.362 1.225 1.993a.076.076 0 0 0 .084.028 19.839 19.839 0 0 0 6.002-3.03.077.077 0 0 0 .032-.054c.5-5.177-.838-9.674-3.549-13.66a.061.061 0 0 0-.031-.03zM8.02 15.33c-1.183 0-2.157-1.085-2.157-2.419 0-1.333.956-2.419 2.157-2.419 1.21 0 2.176 1.095 2.157 2.42 0 1.333-.956 2.418-2.157 2.418zm7.975 0c-1.183 0-2.157-1.085-2.157-2.419 0-1.333.955-2.419 2.157-2.419 1.21 0 2.176 1.095 2.157 2.42 0 1.333-.946 2.418-2.157 2.418z" />
</svg>
<div className="min-w-0 flex-1">
<div className="text-sm font-medium">Join the Discord</div>
<div className="text-xs text-muted-foreground">Get help & share voices</div>
</div>
<ArrowUpRight className="h-4 w-4 text-muted-foreground/40 group-hover:text-muted-foreground transition-colors" />
</a>
</div>
<SettingSection>
<SettingRow
title="Server URL"
description="The address of your voicebox backend server."
action={
<ConnectionStatus health={health} isLoading={isLoading} healthError={healthError} />
}
>
<Form {...form}>
<form onSubmit={form.handleSubmit(onSubmit)} className="flex gap-2">
<FormField
control={form.control}
name="serverUrl"
render={({ field }) => (
<FormItem className="flex-1">
<FormControl>
<Input placeholder="http://127.0.0.1:17493" {...field} />
</FormControl>
<FormMessage />
</FormItem>
)}
/>
{isDirty && (
<Button type="submit" size="sm">
Save
</Button>
)}
</form>
</Form>
</SettingRow>
<SettingRow
title="Keep server running when app closes"
description="The server will continue running in the background after closing the app."
htmlFor="keepServerRunning"
action={
<Toggle
id="keepServerRunning"
checked={keepServerRunningOnClose}
onCheckedChange={(checked: boolean) => {
setKeepServerRunningOnClose(checked);
platform.lifecycle.setKeepServerRunning(checked).catch((error) => {
console.error('Failed to sync setting to Rust:', error);
setKeepServerRunningOnClose(!checked);
toast({
title: 'Failed to update setting',
description: 'Could not sync setting to backend.',
variant: 'destructive',
});
return;
});
toast({
title: 'Setting updated',
description: checked
? 'Server will continue running when app closes'
: 'Server will stop when app closes',
});
}}
/>
}
/>
{platform.metadata.isTauri && (
<SettingRow
title="Allow network access"
description="Makes the server accessible from other devices on your network. Restart the app after changing."
htmlFor="allowNetworkAccess"
action={
<Toggle
id="allowNetworkAccess"
checked={mode === 'remote'}
onCheckedChange={(checked: boolean) => {
setMode(checked ? 'remote' : 'local');
toast({
title: 'Setting updated',
description: checked
? 'Network access enabled. Restart the app to apply.'
: 'Network access disabled. Restart the app to apply.',
});
}}
/>
}
/>
)}
</SettingSection>
<ApiReferenceCard serverUrl={serverUrl} />
{platform.metadata.isTauri && <UpdatesSection />}
</div>
);
}
function ConnectionStatus({
health,
isLoading,
healthError,
}: {
health: ReturnType<typeof useServerHealth>['data'];
isLoading: boolean;
healthError: ReturnType<typeof useServerHealth>['error'];
}) {
if (isLoading) {
return (
<div className="flex items-center gap-2 rounded-full border border-border/60 px-3 py-1">
<Loader2 className="h-3 w-3 animate-spin text-muted-foreground" />
<span className="text-xs text-muted-foreground">Connecting</span>
</div>
);
}
if (healthError) {
return (
<div className="flex items-center gap-2 rounded-full border border-destructive/30 px-3 py-1">
<span className="relative flex h-2 w-2">
<span className="absolute inline-flex h-full w-full rounded-full bg-destructive/40" />
<span className="relative inline-flex h-2 w-2 rounded-full bg-destructive" />
</span>
<span className="text-xs text-destructive">Offline</span>
</div>
);
}
if (health) {
return (
<div className="flex items-center gap-2 rounded-full border border-accent/30 px-3 py-1">
<span className="relative flex h-2 w-2">
<span className="absolute inline-flex h-full w-full animate-ping rounded-full bg-accent/60" />
<span className="relative inline-flex h-2 w-2 rounded-full bg-accent shadow-[0_0_6px_1px_hsl(var(--accent)/0.5)]" />
</span>
<span className="text-xs text-muted-foreground">Online</span>
</div>
);
}
return null;
}
function UpdatesSection() {
const platform = usePlatform();
const { status, checkForUpdates, downloadAndInstall, restartAndInstall } = useAutoUpdater(false);
const [currentVersion, setCurrentVersion] = useState<string>('');
const isDev = !import.meta.env?.PROD;
useEffect(() => {
platform.metadata
.getVersion()
.then(setCurrentVersion)
.catch(() => setCurrentVersion('Unknown'));
}, [platform]);
return (
<SettingSection title="App Updates" description={`v${currentVersion}${isDev ? ' (dev)' : ''}`}>
{isDev ? (
<SettingRow
title="Development mode"
description="Auto-updates are disabled in development mode."
/>
) : (
<>
<SettingRow
title="Check for updates"
description={
status.available
? `Version ${status.version} available`
: status.checking
? 'Checking...'
: "You're up to date"
}
action={
<Button
onClick={checkForUpdates}
disabled={status.checking || status.downloading || status.readyToInstall}
variant="outline"
size="sm"
>
<RefreshCw
className={`h-3.5 w-3.5 mr-1.5 ${status.checking ? 'animate-spin' : ''}`}
/>
Check
</Button>
}
/>
{status.error && (
<SettingRow title="Update error">
<div className="flex items-center gap-2 text-sm text-destructive">
<AlertCircle className="h-4 w-4" />
{status.error}
</div>
</SettingRow>
)}
{status.available && !status.downloading && !status.readyToInstall && (
<SettingRow
title={`Update to ${status.version}`}
description="Download and install the latest version."
action={
<Button onClick={downloadAndInstall} size="sm">
<Download className="h-3.5 w-3.5 mr-1.5" />
Download
</Button>
}
/>
)}
{status.downloading && (
<SettingRow title="Downloading update...">
<div className="space-y-1.5">
<Progress value={status.downloadProgress} />
<div className="flex items-center justify-between text-xs text-muted-foreground">
{status.downloadedBytes !== undefined &&
status.totalBytes !== undefined &&
status.totalBytes > 0 ? (
<span>
{(status.downloadedBytes / 1024 / 1024).toFixed(1)} MB /{' '}
{(status.totalBytes / 1024 / 1024).toFixed(1)} MB
</span>
) : (
<span />
)}
{status.downloadProgress !== undefined && <span>{status.downloadProgress}%</span>}
</div>
</div>
</SettingRow>
)}
{status.readyToInstall && (
<SettingRow
title="Update ready to install"
description={`Version ${status.version} has been downloaded. Restart to complete.`}
action={
<Button onClick={restartAndInstall} size="sm">
<RefreshCw className="h-3.5 w-3.5 mr-1.5" />
Restart Now
</Button>
}
/>
)}
</>
)}
</SettingSection>
);
}
const API_ENDPOINTS = [
{ method: 'POST', path: '/generate', label: 'Generate speech' },
{ method: 'GET', path: '/health', label: 'Server status' },
{ method: 'GET', path: '/profiles', label: 'List voices' },
{ method: 'GET', path: '/history', label: 'Past generations' },
];
function ApiReferenceCard({ serverUrl }: { serverUrl: string }) {
return (
<div className="rounded-lg border border-border/60 p-4 space-y-3">
<div>
<h3 className="text-sm font-medium">API Access</h3>
<p className="text-sm text-muted-foreground">
Integrate Voicebox into your workflow via the REST API at{' '}
<code className="text-xs bg-muted px-1 py-0.5 rounded font-mono">{serverUrl}</code>
</p>
</div>
<div className="space-y-1">
{API_ENDPOINTS.map((ep) => (
<div key={ep.path} className="flex items-center gap-2.5 py-1">
<span
className={`text-[10px] font-mono font-semibold w-9 text-center rounded px-1 py-px ${
ep.method === 'POST' ? 'bg-accent/10 text-accent' : 'bg-muted text-muted-foreground'
}`}
>
{ep.method}
</span>
<code className="text-xs font-mono text-muted-foreground">{ep.path}</code>
<span className="text-xs text-muted-foreground/50 ml-auto">{ep.label}</span>
</div>
))}
</div>
<p className="text-xs text-muted-foreground">
<a
href={`${serverUrl}/docs`}
target="_blank"
rel="noopener noreferrer"
className="text-accent hover:underline"
>
View the full API reference
</a>
</p>
</div>
);
}
@@ -0,0 +1,138 @@
import { FolderOpen } from 'lucide-react';
import { useCallback, useEffect, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Slider } from '@/components/ui/slider';
import { Toggle } from '@/components/ui/toggle';
import { usePlatform } from '@/platform/PlatformContext';
import { useServerStore } from '@/stores/serverStore';
import { SettingRow, SettingSection } from './SettingRow';
export function GenerationPage() {
const platform = usePlatform();
const serverUrl = useServerStore((state) => state.serverUrl);
const maxChunkChars = useServerStore((state) => state.maxChunkChars);
const setMaxChunkChars = useServerStore((state) => state.setMaxChunkChars);
const crossfadeMs = useServerStore((state) => state.crossfadeMs);
const setCrossfadeMs = useServerStore((state) => state.setCrossfadeMs);
const normalizeAudio = useServerStore((state) => state.normalizeAudio);
const setNormalizeAudio = useServerStore((state) => state.setNormalizeAudio);
const autoplayOnGenerate = useServerStore((state) => state.autoplayOnGenerate);
const setAutoplayOnGenerate = useServerStore((state) => state.setAutoplayOnGenerate);
const [opening, setOpening] = useState(false);
const [generationsPath, setGenerationsPath] = useState<string | null>(null);
useEffect(() => {
fetch(`${serverUrl}/health/filesystem`)
.then((res) => res.json())
.then((data) => {
const genDir = data.directories?.find((d: { path: string }) =>
d.path.includes('generations'),
);
if (genDir?.path) setGenerationsPath(genDir.path);
})
.catch(() => {});
}, [serverUrl]);
const openGenerationsFolder = useCallback(async () => {
if (!generationsPath) return;
setOpening(true);
try {
await platform.filesystem.openPath(generationsPath);
} catch (e) {
console.error('Failed to open generations folder:', e);
} finally {
setOpening(false);
}
}, [platform, generationsPath]);
return (
<div className="space-y-8 max-w-2xl">
<SettingSection
title="Generation"
description="Controls for long text generation. These settings apply to all engines."
>
<SettingRow
title="Auto-chunking limit"
description="Long text is split into chunks at sentence boundaries. Lower values can improve quality for long outputs."
action={
<span className="text-sm tabular-nums text-muted-foreground">
{maxChunkChars} chars
</span>
}
>
<Slider
id="maxChunkChars"
value={[maxChunkChars]}
onValueChange={([value]) => setMaxChunkChars(value)}
min={100}
max={5000}
step={50}
aria-label="Auto-chunking character limit"
/>
</SettingRow>
<SettingRow
title="Chunk crossfade"
description="Blends audio between chunks to smooth transitions. Set to 0 for a hard cut."
action={
<span className="text-sm tabular-nums text-muted-foreground">
{crossfadeMs === 0 ? 'Cut' : `${crossfadeMs}ms`}
</span>
}
>
<Slider
id="crossfadeMs"
value={[crossfadeMs]}
onValueChange={([value]) => setCrossfadeMs(value)}
min={0}
max={200}
step={10}
aria-label="Chunk crossfade duration"
/>
</SettingRow>
<SettingRow
title="Normalize audio"
description="Adjusts output volume to a consistent level across generations."
htmlFor="normalizeAudio"
action={
<Toggle
id="normalizeAudio"
checked={normalizeAudio}
onCheckedChange={setNormalizeAudio}
/>
}
/>
<SettingRow
title="Autoplay on generate"
description="Automatically play audio when a generation completes."
htmlFor="autoplayOnGenerate"
action={
<Toggle
id="autoplayOnGenerate"
checked={autoplayOnGenerate}
onCheckedChange={setAutoplayOnGenerate}
/>
}
/>
<SettingRow
title="Generations folder"
description={generationsPath ?? 'Where generated audio files are stored on disk.'}
action={
<Button
variant="outline"
size="sm"
onClick={openGenerationsFolder}
disabled={opening || !generationsPath}
>
<FolderOpen className="h-3.5 w-3.5 mr-1.5" />
Open
</Button>
}
/>
</SettingSection>
</div>
);
}
+405
View File
@@ -0,0 +1,405 @@
import { useQuery, useQueryClient } from '@tanstack/react-query';
import { AlertCircle, Cpu, Download, Loader2, RotateCw, Trash2 } from 'lucide-react';
import { useCallback, useEffect, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Progress } from '@/components/ui/progress';
import { apiClient } from '@/lib/api/client';
import type { CudaDownloadProgress, HealthResponse } from '@/lib/api/types';
import { useServerHealth } from '@/lib/hooks/useServer';
import { usePlatform } from '@/platform/PlatformContext';
import { useServerStore } from '@/stores/serverStore';
import { SettingRow, SettingSection } from './SettingRow';
type RestartPhase = 'idle' | 'stopping' | 'waiting' | 'ready';
function AppleLogo({ className }: { className?: string }) {
return (
<svg className={className} viewBox="0 0 24 24" fill="currentColor" aria-hidden="true">
<path d="M18.71 19.5c-.83 1.24-1.71 2.45-3.05 2.47-1.34.03-1.77-.79-3.29-.79-1.53 0-2 .77-3.27.82-1.31.05-2.3-1.32-3.14-2.53C4.25 17 2.94 12.45 4.7 9.39c.87-1.52 2.43-2.48 4.12-2.51 1.28-.02 2.5.87 3.29.87.78 0 2.26-1.07 3.8-.91.65.03 2.47.26 3.64 1.98-.09.06-2.17 1.28-2.15 3.81.03 3.02 2.65 4.03 2.68 4.04-.03.07-.42 1.44-1.38 2.83M13 3.5c.73-.83 1.94-1.46 2.94-1.5.13 1.17-.34 2.35-1.04 3.19-.69.85-1.83 1.51-2.95 1.42-.15-1.15.41-2.35 1.05-3.11z" />
</svg>
);
}
function GpuIcon({ className }: { className?: string }) {
return (
<svg
className={className}
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth="1.5"
strokeLinecap="round"
strokeLinejoin="round"
aria-hidden="true"
>
<rect x="4" y="6" width="16" height="12" rx="2" />
<path d="M2 10h2M2 14h2M20 10h2M20 14h2" />
<path d="M9 10h6M9 14h4" />
</svg>
);
}
function GpuInfoCard({ health }: { health: HealthResponse }) {
const hasGpu = health.gpu_available && health.gpu_type;
// Parse GPU name from type string like "CUDA (NVIDIA RTX 4090)" or "MPS (Apple M2 Pro)"
const gpuName = hasGpu
? health.gpu_type!.replace(/^(CUDA|ROCm|MPS|Metal|XPU|DirectML)\s*\((.+)\)$/, '$2') ||
health.gpu_type!
: null;
const gpuBackend = hasGpu ? health.gpu_type!.replace(/\s*\(.+\)$/, '') : null;
const isApple = gpuBackend === 'MPS' || gpuBackend === 'Metal';
const showBackendVariant = health.backend_variant && health.backend_variant !== 'cpu';
return (
<div className="rounded-lg border border-border/60 p-4">
<div className="flex items-center gap-3">
{hasGpu ? (
isApple ? (
<AppleLogo className="h-5 w-5 shrink-0 text-muted-foreground" />
) : (
<GpuIcon className="h-5 w-5 shrink-0 text-accent" />
)
) : (
<Cpu className="h-5 w-5 shrink-0 text-muted-foreground" />
)}
<div className="flex-1 min-w-0 space-y-0.5">
<div className="text-sm font-medium">{hasGpu ? gpuName : 'CPU Only'}</div>
<div className="flex flex-wrap items-center gap-x-3 gap-y-1 text-xs text-muted-foreground">
{hasGpu ? (
<>
<span>{gpuBackend}</span>
{showBackendVariant && (
<>
<span className="text-border">|</span>
<span className="uppercase">{health.backend_variant}</span>
</>
)}
{health.vram_used_mb != null && health.vram_used_mb > 0 && (
<>
<span className="text-border">|</span>
<span>{health.vram_used_mb.toFixed(0)} MB VRAM</span>
</>
)}
</>
) : (
<span>No GPU acceleration detected</span>
)}
</div>
</div>
{hasGpu && (
<div className="flex items-center gap-2 rounded-full border border-accent/30 px-2.5 py-0.5">
<span className="relative flex h-1.5 w-1.5">
<span className="absolute inline-flex h-full w-full animate-ping rounded-full bg-accent/60" />
<span className="relative inline-flex h-1.5 w-1.5 rounded-full bg-accent shadow-[0_0_4px_1px_hsl(var(--accent)/0.4)]" />
</span>
<span className="text-[10px] font-medium text-muted-foreground">Active</span>
</div>
)}
</div>
</div>
);
}
export function GpuPage() {
const platform = usePlatform();
const queryClient = useQueryClient();
const serverUrl = useServerStore((state) => state.serverUrl);
const { data: health } = useServerHealth();
const [restartPhase, setRestartPhase] = useState<RestartPhase>('idle');
const [error, setError] = useState<string | null>(null);
const [downloadProgress, setDownloadProgress] = useState<CudaDownloadProgress | null>(null);
const healthPollRef = useRef<ReturnType<typeof setInterval> | null>(null);
const {
data: cudaStatus,
isLoading: _cudaStatusLoading,
refetch: refetchCudaStatus,
} = useQuery({
queryKey: ['cuda-status', serverUrl],
queryFn: () => apiClient.getCudaStatus(),
refetchInterval: (query) => (query.state.status === 'pending' ? false : 10000),
retry: 1,
enabled: !!health,
});
const isCurrentlyCuda = health?.backend_variant === 'cuda';
const cudaAvailable = cudaStatus?.available ?? false;
const cudaDownloading = cudaStatus?.downloading ?? false;
useEffect(() => {
return () => {
if (healthPollRef.current) {
clearInterval(healthPollRef.current);
healthPollRef.current = null;
}
};
}, []);
useEffect(() => {
if (!cudaDownloading || !serverUrl) return;
const eventSource = new EventSource(`${serverUrl}/backend/cuda-progress`);
eventSource.onmessage = (event) => {
try {
const data = JSON.parse(event.data) as CudaDownloadProgress;
setDownloadProgress(data);
if (data.status === 'complete') {
eventSource.close();
setDownloadProgress(null);
refetchCudaStatus();
} else if (data.status === 'error') {
eventSource.close();
setError(data.error || 'Download failed');
setDownloadProgress(null);
refetchCudaStatus();
}
} catch (e) {
console.error('Error parsing CUDA progress event:', e);
}
};
eventSource.onerror = () => {
eventSource.close();
};
return () => {
eventSource.close();
};
}, [cudaDownloading, serverUrl, refetchCudaStatus]);
const clearHealthPolling = useCallback(() => {
if (healthPollRef.current) {
clearInterval(healthPollRef.current);
healthPollRef.current = null;
}
}, []);
const startHealthPolling = useCallback(() => {
clearHealthPolling();
healthPollRef.current = setInterval(async () => {
try {
const result = await apiClient.getHealth();
if (result.status === 'healthy') {
clearHealthPolling();
setRestartPhase('ready');
queryClient.invalidateQueries();
setTimeout(() => setRestartPhase('idle'), 2000);
}
} catch {
// Server still down, keep polling
}
}, 1000);
}, [queryClient, clearHealthPolling]);
const restartServerWithPolling = useCallback(
async (errorMessage: string) => {
setRestartPhase('stopping');
try {
await platform.lifecycle.restartServer();
setRestartPhase('waiting');
startHealthPolling();
} catch (e: unknown) {
clearHealthPolling();
setRestartPhase('idle');
throw new Error(e instanceof Error ? e.message : errorMessage);
}
},
[platform, startHealthPolling, clearHealthPolling],
);
const handleDownload = async () => {
setError(null);
try {
await apiClient.downloadCudaBackend();
refetchCudaStatus();
} catch (e: unknown) {
const msg = e instanceof Error ? e.message : 'Failed to start download';
if (msg.includes('already downloaded')) {
refetchCudaStatus();
} else {
setError(msg);
}
}
};
const handleRestart = async () => {
setError(null);
try {
await restartServerWithPolling('Restart failed');
} catch (e: unknown) {
setError(e instanceof Error ? e.message : 'Restart failed');
}
};
const handleSwitchToCpu = async () => {
setError(null);
setRestartPhase('stopping');
try {
await apiClient.deleteCudaBackend();
await restartServerWithPolling('Failed to switch to CPU');
} catch (e: unknown) {
setError(e instanceof Error ? e.message : 'Failed to switch to CPU');
refetchCudaStatus();
}
};
const handleDelete = async () => {
setError(null);
try {
await apiClient.deleteCudaBackend();
refetchCudaStatus();
} catch (e: unknown) {
setError(e instanceof Error ? e.message : 'Failed to delete CUDA backend');
}
};
const formatBytes = (bytes: number): string => {
if (bytes === 0) return '0 B';
const k = 1024;
const sizes = ['B', 'KB', 'MB', 'GB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return `${(bytes / k ** i).toFixed(1)} ${sizes[i]}`;
};
if (!health) return null;
const hasNativeGpu =
health.gpu_available &&
!isCurrentlyCuda &&
health.gpu_type &&
!health.gpu_type.includes('CUDA');
return (
<div className="space-y-8 max-w-2xl">
<GpuInfoCard health={health} />
{/* CUDA section — only when no native GPU and not already on CUDA */}
{!hasNativeGpu && !isCurrentlyCuda && (
<SettingSection
title="CUDA Backend"
description="NVIDIA GPU acceleration via a downloadable CUDA backend."
>
{/* Download progress */}
{cudaDownloading && downloadProgress && (
<SettingRow title="Downloading CUDA backend...">
<div className="space-y-1.5">
<Progress value={downloadProgress.progress} className="h-2" />
<div className="flex items-center justify-between text-xs text-muted-foreground">
<span>
{downloadProgress.filename ||
(cudaAvailable ? 'Updating...' : 'Downloading...')}
</span>
<span>
{downloadProgress.total > 0
? `${formatBytes(downloadProgress.current)} / ${formatBytes(downloadProgress.total)}`
: `${downloadProgress.progress.toFixed(1)}%`}
</span>
</div>
</div>
</SettingRow>
)}
{/* Restart in progress */}
{restartPhase !== 'idle' && (
<SettingRow
title={
restartPhase === 'ready'
? 'Server restarted successfully'
: restartPhase === 'waiting'
? 'Restarting server...'
: 'Stopping server...'
}
action={<Loader2 className="h-4 w-4 animate-spin text-muted-foreground" />}
/>
)}
{/* Error */}
{error && (
<SettingRow title="Error">
<div className="flex items-center gap-2 text-sm text-destructive">
<AlertCircle className="h-4 w-4 shrink-0" />
<span>{error}</span>
</div>
</SettingRow>
)}
{/* Actions */}
{restartPhase === 'idle' && !cudaDownloading && (
<>
{!cudaAvailable && !isCurrentlyCuda && (
<SettingRow
title="Download CUDA backend"
description="~2.4 GB download. Requires an NVIDIA GPU with CUDA support."
action={
<Button onClick={handleDownload} size="sm">
<Download className="h-3.5 w-3.5 mr-1.5" />
Download
</Button>
}
/>
)}
{cudaAvailable && !isCurrentlyCuda && platform.metadata.isTauri && (
<SettingRow
title="Switch to CUDA backend"
description="CUDA backend is downloaded and ready. Restart to enable."
action={
<Button onClick={handleRestart} size="sm">
<RotateCw className="h-3.5 w-3.5 mr-1.5" />
Restart
</Button>
}
/>
)}
{isCurrentlyCuda && platform.metadata.isTauri && (
<SettingRow
title="Switch to CPU backend"
description="Disable GPU acceleration. You can re-download CUDA later."
action={
<Button onClick={handleSwitchToCpu} variant="outline" size="sm">
<RotateCw className="h-3.5 w-3.5 mr-1.5" />
Switch
</Button>
}
/>
)}
{cudaAvailable && !isCurrentlyCuda && (
<SettingRow
title="Remove CUDA backend"
description="Delete the downloaded CUDA binary to free disk space."
action={
<Button
onClick={handleDelete}
variant="ghost"
size="sm"
className="text-muted-foreground hover:text-destructive"
>
<Trash2 className="h-3.5 w-3.5 mr-1.5" />
Remove
</Button>
}
/>
)}
</>
)}
</SettingSection>
)}
<p className="text-xs text-muted-foreground/60 leading-relaxed">
Voicebox automatically detects and uses the best available GPU on your system. On Apple
Silicon Macs, the MLX backend runs natively on the Neural Engine and GPU via Metal
Performance Shaders (MPS), with no additional setup required. On Windows and Linux with
NVIDIA GPUs, you can download an optional CUDA backend for hardware-accelerated inference.
AMD ROCm, Intel XPU, and DirectML are also supported where available through PyTorch. When
no GPU is detected, Voicebox falls back to CPU — all engines still work, just slower.
</p>
</div>
);
}
+104
View File
@@ -0,0 +1,104 @@
import { useEffect, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { cn } from '@/lib/utils/cn';
import { type LogEntry, useLogStore } from '@/stores/logStore';
function formatTime(timestamp: number): string {
const d = new Date(timestamp);
return d.toLocaleTimeString(undefined, {
hour: '2-digit',
minute: '2-digit',
second: '2-digit',
hour12: false,
});
}
function LogLine({ entry }: { entry: LogEntry }) {
return (
<div className="flex gap-3 font-mono text-xs leading-5 hover:bg-muted/30">
<span className="text-muted-foreground/50 select-none shrink-0">
{formatTime(entry.timestamp)}
</span>
<span
className={cn(
'whitespace-pre-wrap break-all',
entry.stream === 'stderr' ? 'text-orange-400/80' : 'text-muted-foreground',
)}
>
{entry.line}
</span>
</div>
);
}
export function LogsPage() {
const entries = useLogStore((s) => s.entries);
const clear = useLogStore((s) => s.clear);
const containerRef = useRef<HTMLDivElement>(null);
const [autoScroll, setAutoScroll] = useState(true);
// Auto-scroll to bottom when new entries arrive
useEffect(() => {
if (autoScroll && containerRef.current) {
containerRef.current.scrollTop = containerRef.current.scrollHeight;
}
}, [entries.length, autoScroll]);
// Detect manual scroll to disable auto-scroll
const handleScroll = () => {
const el = containerRef.current;
if (!el) return;
const atBottom = el.scrollHeight - el.scrollTop - el.clientHeight < 40;
setAutoScroll(atBottom);
};
return (
<div className="flex flex-col h-full min-h-0">
<div className="flex items-center justify-between mb-3">
<div>
<h3 className="text-sm font-medium">Server Logs</h3>
<p className="text-sm text-muted-foreground">
{entries.length} {entries.length === 1 ? 'line' : 'lines'}
</p>
</div>
<div className="flex items-center gap-2">
{!autoScroll && (
<Button
variant="outline"
size="sm"
onClick={() => {
setAutoScroll(true);
containerRef.current?.scrollTo({ top: containerRef.current.scrollHeight });
}}
>
Scroll to bottom
</Button>
)}
<Button variant="outline" size="sm" onClick={clear}>
Clear
</Button>
</div>
</div>
<div
ref={containerRef}
onScroll={handleScroll}
className="flex-1 min-h-0 overflow-y-auto rounded-md border bg-black/20 p-3"
>
{entries.length === 0 ? (
<div className="text-sm text-muted-foreground/50 font-mono space-y-1">
<p>No log output yet.</p>
{!import.meta.env?.PROD && (
<p>
Server logs are only captured when the app manages the server process (production
builds).
</p>
)}
</div>
) : (
entries.map((entry) => <LogLine key={entry.id} entry={entry} />)
)}
</div>
</div>
);
}
+59 -24
View File
@@ -1,35 +1,70 @@
import { ConnectionForm } from '@/components/ServerSettings/ConnectionForm';
import { GenerationSettings } from '@/components/ServerSettings/GenerationSettings';
import { GpuAcceleration } from '@/components/ServerSettings/GpuAcceleration';
import { UpdateStatus } from '@/components/ServerSettings/UpdateStatus';
import { Link, Outlet, useMatchRoute } from '@tanstack/react-router';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { cn } from '@/lib/utils/cn';
import { usePlatform } from '@/platform/PlatformContext';
import { usePlayerStore } from '@/stores/playerStore';
export function ServerTab() {
interface SettingsTab {
label: string;
path:
| '/settings'
| '/settings/generation'
| '/settings/gpu'
| '/settings/logs'
| '/settings/changelog'
| '/settings/about';
tauriOnly?: boolean;
}
const tabs: SettingsTab[] = [
{ label: 'General', path: '/settings' },
{ label: 'Generation', path: '/settings/generation' },
{ label: 'GPU', path: '/settings/gpu', tauriOnly: true },
{ label: 'Logs', path: '/settings/logs', tauriOnly: true },
{ label: 'Changelog', path: '/settings/changelog' },
{ label: 'About', path: '/settings/about' },
];
export function SettingsLayout() {
const platform = usePlatform();
const isPlayerVisible = !!usePlayerStore((state) => state.audioUrl);
const matchRoute = useMatchRoute();
return (
<div
className={cn('overflow-y-auto flex flex-col', isPlayerVisible && BOTTOM_SAFE_AREA_PADDING)}
>
<div className="grid gap-4 md:grid-cols-2">
<ConnectionForm />
<GenerationSettings />
{platform.metadata.isTauri && <GpuAcceleration />}
{platform.metadata.isTauri && <UpdateStatus />}
</div>
<div className="py-8 text-center text-sm text-muted-foreground">
Created by{' '}
<a
href="https://github.com/jamiepine"
target="_blank"
rel="noopener noreferrer"
className="text-accent hover:underline"
>
Jamie Pine
</a>
<div className="flex flex-col h-full min-h-0">
<nav className="flex gap-1 border-b shrink-0">
{tabs.map((tab) => {
if (tab.tauriOnly && !platform.metadata.isTauri) return null;
const isActive =
tab.path === '/settings'
? matchRoute({ to: tab.path, fuzzy: false })
: matchRoute({ to: tab.path });
return (
<Link
key={tab.path}
to={tab.path}
className={cn(
'px-4 py-2 text-sm font-medium transition-colors border-b-2 -mb-px',
isActive
? 'border-accent text-foreground'
: 'border-transparent text-muted-foreground hover:text-foreground hover:border-muted-foreground/30',
)}
>
{tab.label}
</Link>
);
})}
</nav>
<div
className={cn(
'flex-1 overflow-y-auto pt-6 pb-6 px-2 -mx-2',
isPlayerVisible && BOTTOM_SAFE_AREA_PADDING,
)}
>
<Outlet />
</div>
</div>
);
@@ -0,0 +1,62 @@
import type { ReactNode } from 'react';
/**
* A section header with title and optional description, separated by a border.
*/
export function SettingSection({
title,
description,
children,
}: {
title?: string;
description?: string;
children: ReactNode;
}) {
return (
<div className="space-y-1">
{title && <h3 className="text-sm font-medium">{title}</h3>}
{description && <p className="text-sm text-muted-foreground">{description}</p>}
<div className={`${title || description ? 'pt-3' : ''} space-y-0 divide-y divide-border/60`}>
{children}
</div>
</div>
);
}
/**
* A single settings row: label+description on the left, action on the right.
* Use for toggles, inputs, buttons, badges — any control type.
*/
export function SettingRow({
title,
description,
htmlFor,
action,
children,
}: {
title: string;
description?: string;
htmlFor?: string;
/** Right-aligned control (checkbox, button, badge, etc.) */
action?: ReactNode;
/** Full-width content rendered below the label row (for sliders, inputs, etc.) */
children?: ReactNode;
}) {
return (
<div className="py-3">
<div className="flex items-center justify-between gap-8">
<div className="min-w-0">
<label
htmlFor={htmlFor}
className={`text-sm font-medium leading-none select-none ${htmlFor ? 'cursor-pointer' : ''}`}
>
{title}
</label>
{description && <p className="text-sm text-muted-foreground mt-0.5">{description}</p>}
</div>
{action && <div className="shrink-0">{action}</div>}
</div>
{children && <div className="mt-3">{children}</div>}
</div>
);
}
+6 -5
View File
@@ -1,5 +1,5 @@
import { Link, useMatchRoute } from '@tanstack/react-router';
import { AudioLines, Box, Mic, Server, Speaker, Volume2, Wand2 } from 'lucide-react';
import { AudioLines, Box, Mic, Settings, Speaker, Volume2, Wand2 } from 'lucide-react';
import { useEffect, useState } from 'react';
import voiceboxLogo from '@/assets/voicebox-logo.png';
import { cn } from '@/lib/utils/cn';
@@ -19,7 +19,7 @@ const tabs = [
{ id: 'effects', path: '/effects', icon: Wand2, label: 'Effects' },
{ id: 'audio', path: '/audio', icon: Speaker, label: 'Audio' },
{ id: 'models', path: '/models', icon: Box, label: 'Models' },
{ id: 'server', path: '/server', icon: Server, label: 'Server' },
{ id: 'settings', path: '/settings', icon: Settings, label: 'Settings' },
];
export function Sidebar({ isMacOS }: SidebarProps) {
@@ -54,9 +54,10 @@ export function Sidebar({ isMacOS }: SidebarProps) {
<div className="flex flex-col gap-3">
{tabs.map((tab, index) => {
const Icon = tab.icon;
// For index route, use exact match; for others, use default matching
const isActive =
tab.path === '/' ? matchRoute({ to: '/', exact: true }) : matchRoute({ to: tab.path });
tab.path === '/'
? matchRoute({ to: '/', fuzzy: false })
: matchRoute({ to: tab.path, fuzzy: true });
// Accent fades as buttons get further from the logo
const accentOpacity = Math.max(0.08, 0.5 - index * 0.07);
@@ -98,7 +99,7 @@ export function Sidebar({ isMacOS }: SidebarProps) {
<span className="text-[10px] text-muted-foreground/50">v{version}</span>
{updateStatus.available && (
<Link
to="/server"
to="/settings"
className="text-[9px] font-semibold tracking-wide uppercase px-2 py-0.5 rounded-full bg-accent/15 text-accent hover:bg-accent/25 transition-colors"
>
Update
+2 -2
View File
@@ -1,5 +1,5 @@
import * as React from 'react';
import * as SliderPrimitive from '@radix-ui/react-slider';
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
const Slider = React.forwardRef<
@@ -14,7 +14,7 @@ const Slider = React.forwardRef<
<SliderPrimitive.Track className="relative h-2 w-full grow overflow-hidden rounded-full bg-secondary">
<SliderPrimitive.Range className="absolute h-full bg-primary" />
</SliderPrimitive.Track>
<SliderPrimitive.Thumb className="block h-5 w-5 rounded-full border-2 border-primary bg-background ring-offset-background transition-colors focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:pointer-events-none disabled:opacity-50 translate-x-0.5" />
<SliderPrimitive.Thumb className="block h-0 w-0 outline-none disabled:pointer-events-none disabled:opacity-50 after:block after:h-5 after:w-5 after:rounded-full after:border-2 after:border-primary after:bg-background after:ring-offset-background after:transition-colors after:absolute after:top-1/2 after:left-1/2 after:-translate-x-1/2 after:-translate-y-1/2 focus-visible:after:ring-2 focus-visible:after:ring-ring focus-visible:after:ring-offset-2" />
</SliderPrimitive.Root>
));
Slider.displayName = SliderPrimitive.Root.displayName;
+3 -1
View File
@@ -1,3 +1,4 @@
import { usePlayerStore } from '@/stores/playerStore';
import {
Toast,
ToastClose,
@@ -10,6 +11,7 @@ import { useToast } from './use-toast';
export function Toaster() {
const { toasts } = useToast();
const isPlayerOpen = !!usePlayerStore((s) => s.audioUrl);
return (
<ToastProvider>
@@ -23,7 +25,7 @@ export function Toaster() {
<ToastClose />
</Toast>
))}
<ToastViewport />
<ToastViewport className={isPlayerOpen ? 'sm:bottom-44' : ''} />
</ToastProvider>
);
}
+48
View File
@@ -0,0 +1,48 @@
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
export interface ToggleProps {
checked?: boolean;
onCheckedChange?: (checked: boolean) => void;
disabled?: boolean;
className?: string;
id?: string;
}
const Toggle = React.forwardRef<HTMLButtonElement, ToggleProps>(
({ checked = false, onCheckedChange, disabled = false, className, id, ...props }, ref) => {
return (
<button
type="button"
ref={ref}
id={id}
role="switch"
aria-checked={checked}
disabled={disabled}
onClick={() => {
if (!disabled && onCheckedChange) {
onCheckedChange(!checked);
}
}}
className={cn(
'relative inline-flex h-5 w-9 shrink-0 items-center rounded-full transition-colors',
'focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2',
checked ? 'bg-accent' : 'bg-muted-foreground/25',
disabled ? 'opacity-50 cursor-not-allowed' : 'cursor-pointer',
className,
)}
{...props}
>
<span
className={cn(
'pointer-events-none block h-4 w-4 rounded-full bg-white shadow-sm transition-transform',
checked ? 'translate-x-[18px]' : 'translate-x-[2px]',
)}
/>
</button>
);
},
);
Toggle.displayName = 'Toggle';
export { Toggle };
+5
View File
@@ -1,3 +1,8 @@
interface Window {
__voiceboxServerStartedByApp?: boolean;
}
declare module 'virtual:changelog' {
const raw: string;
export default raw;
}
+35 -12
View File
@@ -32,8 +32,24 @@ import type {
TranscriptionResponse,
VoiceProfileCreate,
VoiceProfileResponse,
WhisperModelSize,
} from './types';
function formatErrorDetail(detail: unknown, fallback: string): string {
if (typeof detail === 'string') return detail;
if (Array.isArray(detail)) {
return detail
.map((e: Record<string, unknown>) => e.msg || e.message || JSON.stringify(e))
.join('; ');
}
if (detail && typeof detail === 'object') {
const obj = detail as Record<string, unknown>;
if (typeof obj.message === 'string') return obj.message;
return JSON.stringify(detail);
}
return fallback;
}
class ApiClient {
private getBaseUrl(): string {
const serverUrl = useServerStore.getState().serverUrl;
@@ -54,7 +70,7 @@ class ApiClient {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
}
return response.json();
@@ -113,7 +129,7 @@ class ApiClient {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
}
return response.json();
@@ -147,7 +163,7 @@ class ApiClient {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
}
return response.blob();
@@ -167,7 +183,7 @@ class ApiClient {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
}
return response.json();
@@ -187,7 +203,7 @@ class ApiClient {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
}
return response.json();
@@ -257,7 +273,7 @@ class ApiClient {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
}
return response.blob();
@@ -271,7 +287,7 @@ class ApiClient {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
}
return response.blob();
@@ -297,7 +313,7 @@ class ApiClient {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
}
return response.json();
@@ -318,12 +334,19 @@ class ApiClient {
}
// Transcription
async transcribeAudio(file: File, language?: LanguageCode): Promise<TranscriptionResponse> {
async transcribeAudio(
file: File,
language?: LanguageCode,
model?: WhisperModelSize,
): Promise<TranscriptionResponse> {
const formData = new FormData();
formData.append('file', file);
if (language) {
formData.append('language', language);
}
if (model) {
formData.append('model', model);
}
const url = `${this.getBaseUrl()}/transcribe`;
const response = await fetch(url, {
@@ -335,7 +358,7 @@ class ApiClient {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
}
return response.json();
@@ -608,7 +631,7 @@ class ApiClient {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
}
return response.blob();
@@ -705,7 +728,7 @@ class ApiClient {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
}
return response.blob();
+5 -2
View File
@@ -42,8 +42,8 @@ export interface GenerationRequest {
text: string;
language: LanguageCode;
seed?: number;
model_size?: '1.7B' | '0.6B';
engine?: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo';
model_size?: '1.7B' | '0.6B' | '1B' | '3B';
engine?: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo' | 'tada';
instruct?: string;
max_chunk_chars?: number;
crossfade_ms?: number;
@@ -99,8 +99,11 @@ export interface HistoryListResponse {
total: number;
}
export type WhisperModelSize = 'base' | 'small' | 'medium' | 'large' | 'turbo';
export interface TranscriptionRequest {
language?: LanguageCode;
model?: WhisperModelSize;
}
export interface TranscriptionResponse {
+1
View File
@@ -66,6 +66,7 @@ export const ENGINE_LANGUAGES: Record<string, readonly LanguageCode[]> = {
'zh',
],
chatterbox_turbo: ['en'],
tada: ['en', 'ar', 'zh', 'de', 'es', 'fr', 'it', 'ja', 'pl', 'pt'],
} as const;
/** Helper: get language options for a given engine. */
+17 -9
View File
@@ -15,9 +15,9 @@ const generationSchema = z.object({
text: z.string().min(1, '').max(50000),
language: z.enum(LANGUAGE_CODES as [LanguageCode, ...LanguageCode[]]),
seed: z.number().int().optional(),
modelSize: z.enum(['1.7B', '0.6B']).optional(),
modelSize: z.enum(['1.7B', '0.6B', '1B', '3B']).optional(),
instruct: z.string().max(500).optional(),
engine: z.enum(['qwen', 'luxtts', 'chatterbox', 'chatterbox_turbo']).optional(),
engine: z.enum(['qwen', 'luxtts', 'chatterbox', 'chatterbox_turbo', 'tada']).optional(),
});
export type GenerationFormValues = z.infer<typeof generationSchema>;
@@ -79,7 +79,11 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
? 'chatterbox-tts'
: engine === 'chatterbox_turbo'
? 'chatterbox-turbo'
: `qwen-tts-${data.modelSize}`;
: engine === 'tada'
? data.modelSize === '3B'
? 'tada-3b-ml'
: 'tada-1b'
: `qwen-tts-${data.modelSize}`;
const displayName =
engine === 'luxtts'
? 'LuxTTS'
@@ -87,9 +91,13 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
? 'Chatterbox TTS'
: engine === 'chatterbox_turbo'
? 'Chatterbox Turbo'
: data.modelSize === '1.7B'
? 'Qwen TTS 1.7B'
: 'Qwen TTS 0.6B';
: engine === 'tada'
? data.modelSize === '3B'
? 'TADA 3B Multilingual'
: 'TADA 1B'
: data.modelSize === '1.7B'
? 'Qwen TTS 1.7B'
: 'Qwen TTS 0.6B';
// Check if model needs downloading
try {
@@ -104,7 +112,7 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
console.error('Failed to check model status:', error);
}
const isQwen = engine === 'qwen';
const hasModelSizes = engine === 'qwen' || engine === 'tada';
const effectsChain = options.getEffectsChain?.();
// This now returns immediately with status="generating"
const result = await generation.mutateAsync({
@@ -112,9 +120,9 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
text: data.text,
language: data.language,
seed: data.seed,
model_size: isQwen ? data.modelSize : undefined,
model_size: hasModelSizes ? data.modelSize : undefined,
engine,
instruct: isQwen ? data.instruct || undefined : undefined,
instruct: engine === 'qwen' ? data.instruct || undefined : undefined,
max_chunk_chars: maxChunkChars,
crossfade_ms: crossfadeMs,
normalize: normalizeAudio,
+6 -5
View File
@@ -75,8 +75,8 @@ export function useGenerationProgress() {
currentSources.delete(id);
removePendingGeneration(id);
// Refresh history to pick up the completed generation
queryClient.invalidateQueries({ queryKey: ['history'] });
// Refetch history to pick up the completed generation
queryClient.refetchQueries({ queryKey: ['history'] });
// If this generation was queued for a story, add it now
const storyId = removePendingStoryAdd(id);
@@ -120,7 +120,7 @@ export function useGenerationProgress() {
removePendingGeneration(id);
removePendingStoryAdd(id);
queryClient.invalidateQueries({ queryKey: ['history'] });
queryClient.refetchQueries({ queryKey: ['history'] });
toast({
title: data.status === 'not_found' ? 'Generation not found' : 'Generation failed',
@@ -134,11 +134,12 @@ export function useGenerationProgress() {
};
source.onerror = () => {
// EventSource auto-reconnects, but if we get repeated errors
// just clean up
// SSE connection dropped — clean up and refresh history so any
// completed/failed generation still appears in the list
source.close();
currentSources.delete(id);
removePendingGeneration(id);
queryClient.refetchQueries({ queryKey: ['history'] });
};
currentSources.set(id, source);
+4 -6
View File
@@ -1,4 +1,4 @@
import { useState, useRef, useCallback, useEffect } from 'react';
import { useCallback, useEffect, useRef, useState } from 'react';
import { usePlatform } from '@/platform/PlatformContext';
interface UseSystemAudioCaptureOptions {
@@ -94,15 +94,13 @@ export function useSystemAudioCapture({
const blob = await platform.audio.stopSystemAudioCapture();
// Pass the actual recorded duration
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
onRecordingComplete?.(blob, recordedDuration);
} catch (err) {
const errorMessage =
err instanceof Error
? err.message
: 'Failed to stop system audio capture.';
err instanceof Error ? err.message : 'Failed to stop system audio capture.';
setError(errorMessage);
}
}, [isRecording, onRecordingComplete, platform]);
+10 -2
View File
@@ -1,10 +1,18 @@
import { useMutation } from '@tanstack/react-query';
import { apiClient } from '@/lib/api/client';
import type { WhisperModelSize } from '@/lib/api/types';
import type { LanguageCode } from '@/lib/constants/languages';
export function useTranscription() {
return useMutation({
mutationFn: ({ file, language }: { file: File; language?: LanguageCode }) =>
apiClient.transcribeAudio(file, language),
mutationFn: ({
file,
language,
model,
}: {
file: File;
language?: LanguageCode;
model?: WhisperModelSize;
}) => apiClient.transcribeAudio(file, language, model),
});
}
+37
View File
@@ -0,0 +1,37 @@
export interface ChangelogEntry {
version: string;
date: string | null;
body: string;
}
/**
* Parses a Keep-a-Changelog style markdown string into structured entries.
*
* Splits on `## [version]` headings and extracts the version + date from each.
* The body is the raw markdown between headings (trimmed), with the leading
* `# Changelog` title and trailing link references stripped.
*/
export function parseChangelog(raw: string): ChangelogEntry[] {
const entries: ChangelogEntry[] = [];
// Strip trailing link reference definitions (e.g. [0.1.0]: https://...)
const cleaned = raw.replace(/^\[[\w.]+\]:.*$/gm, '').trimEnd();
// Match `## [version]` or `## [version] - date`
const headingRe = /^## \[(.+?)\](?:\s*-\s*(.+))?$/gm;
const matches = [...cleaned.matchAll(headingRe)];
for (let i = 0; i < matches.length; i++) {
const match = matches[i];
const version = match[1];
const date = match[2]?.trim() || null;
const start = match.index! + match[0].length;
const end = i + 1 < matches.length ? matches[i + 1].index! : cleaned.length;
const body = cleaned.slice(start, end).trim();
entries.push({ version, date, body });
}
return entries;
}
+6
View File
@@ -50,12 +50,18 @@ export interface PlatformAudio {
stopPlayback(): void;
}
export interface ServerLogEntry {
stream: 'stdout' | 'stderr';
line: string;
}
export interface PlatformLifecycle {
startServer(remote?: boolean, modelsDir?: string | null): Promise<string>;
stopServer(): Promise<void>;
restartServer(modelsDir?: string | null): Promise<string>;
setKeepServerRunning(keep: boolean): Promise<void>;
setupWindowCloseHandler(): Promise<void>;
subscribeToServerLogs(callback: (entry: ServerLogEntry) => void): () => void;
onServerReady?: () => void;
}
+72 -6
View File
@@ -1,10 +1,22 @@
import { createRootRoute, createRoute, createRouter, Outlet } from '@tanstack/react-router';
import {
createRootRoute,
createRoute,
createRouter,
Outlet,
redirect,
} from '@tanstack/react-router';
import { AppFrame } from '@/components/AppFrame/AppFrame';
import { AudioTab } from '@/components/AudioTab/AudioTab';
import { EffectsTab } from '@/components/EffectsTab/EffectsTab';
import { MainEditor } from '@/components/MainEditor/MainEditor';
import { ModelsTab } from '@/components/ModelsTab/ModelsTab';
import { ServerTab } from '@/components/ServerTab/ServerTab';
import { AboutPage } from '@/components/ServerTab/AboutPage';
import { ChangelogPage } from '@/components/ServerTab/ChangelogPage';
import { GeneralPage } from '@/components/ServerTab/GeneralPage';
import { GenerationPage } from '@/components/ServerTab/GenerationPage';
import { GpuPage } from '@/components/ServerTab/GpuPage';
import { LogsPage } from '@/components/ServerTab/LogsPage';
import { SettingsLayout } from '@/components/ServerTab/ServerTab';
import { Sidebar } from '@/components/Sidebar';
import { StoriesTab } from '@/components/StoriesTab/StoriesTab';
import { Toaster } from '@/components/ui/toaster';
@@ -120,11 +132,57 @@ const modelsRoute = createRoute({
component: ModelsTab,
});
// Server route
const serverRoute = createRoute({
// Settings layout route (parent for sub-tabs)
const settingsRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/settings',
component: SettingsLayout,
});
// Settings sub-routes
const settingsGeneralRoute = createRoute({
getParentRoute: () => settingsRoute,
path: '/',
component: GeneralPage,
});
const settingsGenerationRoute = createRoute({
getParentRoute: () => settingsRoute,
path: '/generation',
component: GenerationPage,
});
const settingsGpuRoute = createRoute({
getParentRoute: () => settingsRoute,
path: '/gpu',
component: GpuPage,
});
const settingsChangelogRoute = createRoute({
getParentRoute: () => settingsRoute,
path: '/changelog',
component: ChangelogPage,
});
const settingsLogsRoute = createRoute({
getParentRoute: () => settingsRoute,
path: '/logs',
component: LogsPage,
});
const settingsAboutRoute = createRoute({
getParentRoute: () => settingsRoute,
path: '/about',
component: AboutPage,
});
// Redirect old /server path to /settings
const serverRedirectRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/server',
component: ServerTab,
beforeLoad: () => {
throw redirect({ to: '/settings' });
},
});
// Route tree
@@ -135,7 +193,15 @@ const routeTree = rootRoute.addChildren([
audioRoute,
effectsRoute,
modelsRoute,
serverRoute,
settingsRoute.addChildren([
settingsGeneralRoute,
settingsGenerationRoute,
settingsGpuRoute,
settingsLogsRoute,
settingsChangelogRoute,
settingsAboutRoute,
]),
serverRedirectRoute,
]);
// Create router
+31
View File
@@ -0,0 +1,31 @@
import { create } from 'zustand';
import type { ServerLogEntry } from '@/platform/types';
const MAX_LOG_ENTRIES = 2000;
let nextLogEntryId = 0;
export interface LogEntry extends ServerLogEntry {
id: number;
timestamp: number;
}
interface LogStore {
entries: LogEntry[];
addEntry: (entry: ServerLogEntry) => void;
clear: () => void;
}
export const useLogStore = create<LogStore>((set) => ({
entries: [],
addEntry: (entry) =>
set((state) => {
const newEntry: LogEntry = { ...entry, id: nextLogEntryId++, timestamp: Date.now() };
const entries = [...state.entries, newEntry];
if (entries.length > MAX_LOG_ENTRIES) {
return { entries: entries.slice(entries.length - MAX_LOG_ENTRIES) };
}
return { entries };
}),
clear: () => set({ entries: [] }),
}));
+1 -1
View File
@@ -6,5 +6,5 @@
"moduleResolution": "bundler",
"allowSyntheticDefaultImports": true
},
"include": ["vite.config.ts"]
"include": ["vite.config.ts", "plugins/**/*.ts"]
}
+2 -1
View File
@@ -2,9 +2,10 @@ import path from 'node:path';
import tailwindcss from '@tailwindcss/vite';
import react from '@vitejs/plugin-react';
import { defineConfig } from 'vite';
import { changelogPlugin } from './plugins/changelog';
export default defineConfig({
plugins: [tailwindcss(), react()],
plugins: [tailwindcss(), react(), changelogPlugin(path.resolve(__dirname, '..'))],
resolve: {
alias: {
'@': path.resolve(__dirname, './src'),
+1 -1
View File
@@ -1,3 +1,3 @@
# Backend package
__version__ = "0.2.3"
__version__ = "0.3.1"
+52
View File
@@ -77,6 +77,7 @@ def create_app() -> FastAPI:
_configure_cors(application)
register_routers(application)
_register_lifecycle(application)
_mount_frontend(application)
return application
@@ -104,6 +105,43 @@ def _configure_cors(application: FastAPI) -> None:
)
def _mount_frontend(application: FastAPI) -> None:
"""Serve the built web frontend when present (Docker / web deployment).
The Dockerfile copies the Vite build output to ``/app/frontend/``. When
that directory exists we mount static assets and add a catch-all route so
the React SPA handles client-side routing. In dev or API-only mode the
directory is absent and this function is a no-op.
"""
frontend_dir = Path(__file__).resolve().parent.parent / "frontend"
if not frontend_dir.is_dir():
return
from fastapi.staticfiles import StaticFiles
from fastapi.responses import FileResponse
# Mount hashed assets (JS, CSS, images) that Vite places under /assets
assets_dir = frontend_dir / "assets"
if assets_dir.is_dir():
application.mount(
"/assets",
StaticFiles(directory=str(assets_dir)),
name="frontend-assets",
)
# SPA catch-all: serve files if they exist, otherwise index.html for
# client-side routes like /voices, /stories, /models, etc.
@application.get("/{full_path:path}")
async def serve_spa(full_path: str):
file_path = (frontend_dir / full_path).resolve()
# Guard against path traversal — only serve files inside frontend_dir
if full_path and file_path.is_file() and str(file_path).startswith(str(frontend_dir)):
return FileResponse(file_path)
return FileResponse(frontend_dir / "index.html", media_type="text/html")
logger.info("Frontend: serving SPA from %s", frontend_dir)
def _get_gpu_status() -> str:
"""Return a human-readable string describing GPU availability."""
backend_type = get_backend_type()
@@ -117,6 +155,20 @@ def _get_gpu_status() -> str:
return "MPS (Apple Silicon)"
elif backend_type == "mlx":
return "Metal (Apple Silicon via MLX)"
# Intel XPU (Arc / Data Center) via IPEX
try:
import intel_extension_for_pytorch # noqa: F401
if hasattr(torch, "xpu") and torch.xpu.is_available():
try:
xpu_name = torch.xpu.get_device_name(0)
except Exception:
xpu_name = "Intel GPU"
return f"XPU ({xpu_name})"
except ImportError:
pass
return "None (CPU only)"
+28 -2
View File
@@ -134,6 +134,7 @@ class STTBackend(Protocol):
self,
audio_path: str,
language: Optional[str] = None,
model_size: Optional[str] = None,
) -> str:
"""
Transcribe audio to text.
@@ -165,6 +166,7 @@ TTS_ENGINES = {
"luxtts": "LuxTTS",
"chatterbox": "Chatterbox TTS",
"chatterbox_turbo": "Chatterbox Turbo",
"tada": "TADA",
}
@@ -258,6 +260,24 @@ def _get_non_qwen_tts_configs() -> list[ModelConfig]:
needs_trim=True,
languages=["en"],
),
ModelConfig(
model_name="tada-1b",
display_name="TADA 1B (English)",
engine="tada",
hf_repo_id="HumeAI/tada-1b",
model_size="1B",
size_mb=4000,
languages=["en"],
),
ModelConfig(
model_name="tada-3b-ml",
display_name="TADA 3B Multilingual",
engine="tada",
hf_repo_id="HumeAI/tada-3b-ml",
model_size="3B",
size_mb=8000,
languages=["en", "ar", "zh", "de", "es", "fr", "it", "ja", "pl", "pt"],
),
]
@@ -338,10 +358,12 @@ def engine_has_model_sizes(engine: str) -> bool:
async def load_engine_model(engine: str, model_size: str = "default") -> None:
"""Load a model for the given engine, handling the Qwen model_size special case."""
"""Load a model for the given engine, handling engines with multiple model sizes."""
backend = get_tts_backend_for_engine(engine)
if engine == "qwen":
await backend.load_model_async(model_size)
elif engine == "tada":
await backend.load_model(model_size)
else:
await backend.load_model()
@@ -357,7 +379,7 @@ async def ensure_model_cached_or_raise(engine: str, model_size: str = "default")
cfg = c
break
if engine == "qwen":
if engine in ("qwen", "tada"):
if not backend._is_model_cached(model_size):
raise HTTPException(
status_code=400,
@@ -489,6 +511,10 @@ def get_tts_backend_for_engine(engine: str) -> TTSBackend:
from .chatterbox_turbo_backend import ChatterboxTurboTTSBackend
backend = ChatterboxTurboTTSBackend()
elif engine == "tada":
from .hume_backend import HumeTadaBackend
backend = HumeTadaBackend()
else:
raise ValueError(f"Unknown TTS engine: {engine}. Supported: {list(TTS_ENGINES.keys())}")
+31
View File
@@ -126,6 +126,37 @@ def get_torch_device(
return "cpu"
def empty_device_cache(device: str) -> None:
"""
Free cached memory on the given device (CUDA or XPU).
Backends should call this after unloading models so VRAM is returned
to the OS.
"""
import torch
if device == "cuda" and torch.cuda.is_available():
torch.cuda.empty_cache()
elif device == "xpu" and hasattr(torch, "xpu"):
torch.xpu.empty_cache()
def manual_seed(seed: int, device: str) -> None:
"""
Set the random seed on both CPU and the active accelerator.
Covers CUDA and Intel XPU so that generation is reproducible
regardless of which GPU backend is in use.
"""
import torch
torch.manual_seed(seed)
if device == "cuda" and torch.cuda.is_available():
torch.cuda.manual_seed(seed)
elif device == "xpu" and hasattr(torch, "xpu"):
torch.xpu.manual_seed(seed)
async def combine_voice_prompts(
audio_paths: List[str],
reference_texts: List[str],
+6 -10
View File
@@ -18,6 +18,8 @@ from . import TTSBackend
from .base import (
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
patch_chatterbox_f32,
@@ -48,7 +50,7 @@ class ChatterboxTTSBackend:
self._model_load_lock = asyncio.Lock()
def _get_device(self) -> str:
return get_torch_device(force_cpu_on_mac=True)
return get_torch_device(force_cpu_on_mac=True, allow_xpu=True)
def is_loaded(self) -> bool:
return self.model is not None
@@ -117,10 +119,7 @@ class ChatterboxTTSBackend:
del self.model
self.model = None
self._device = None
if device == "cuda":
import torch
torch.cuda.empty_cache()
empty_device_cache(device)
logger.info("Chatterbox unloaded")
async def create_voice_prompt(
@@ -200,7 +199,7 @@ class ChatterboxTTSBackend:
import torch
if seed is not None:
torch.manual_seed(seed)
manual_seed(seed, self._device)
logger.info(f"[Chatterbox] Generating: lang={language}")
@@ -220,10 +219,7 @@ class ChatterboxTTSBackend:
else:
audio = np.asarray(wav, dtype=np.float32)
sample_rate = (
getattr(self.model, "sr", None)
or getattr(self.model, "sample_rate", 24000)
)
sample_rate = getattr(self.model, "sr", None) or getattr(self.model, "sample_rate", 24000)
return audio, sample_rate
+6 -10
View File
@@ -18,6 +18,8 @@ from . import TTSBackend
from .base import (
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
patch_chatterbox_f32,
@@ -48,7 +50,7 @@ class ChatterboxTurboTTSBackend:
self._model_load_lock = asyncio.Lock()
def _get_device(self) -> str:
return get_torch_device(force_cpu_on_mac=True)
return get_torch_device(force_cpu_on_mac=True, allow_xpu=True)
def is_loaded(self) -> bool:
return self.model is not None
@@ -116,10 +118,7 @@ class ChatterboxTurboTTSBackend:
del self.model
self.model = None
self._device = None
if device == "cuda":
import torch
torch.cuda.empty_cache()
empty_device_cache(device)
logger.info("Chatterbox Turbo unloaded")
async def create_voice_prompt(
@@ -181,7 +180,7 @@ class ChatterboxTurboTTSBackend:
import torch
if seed is not None:
torch.manual_seed(seed)
manual_seed(seed, self._device)
logger.info("[Chatterbox Turbo] Generating (English)")
@@ -200,10 +199,7 @@ class ChatterboxTurboTTSBackend:
else:
audio = np.asarray(wav, dtype=np.float32)
sample_rate = (
getattr(self.model, "sr", None)
or getattr(self.model, "sample_rate", 24000)
)
sample_rate = getattr(self.model, "sr", None) or getattr(self.model, "sample_rate", 24000)
return audio, sample_rate
+346
View File
@@ -0,0 +1,346 @@
"""
HumeAI TADA TTS backend implementation.
Wraps HumeAI's TADA (Text-Acoustic Dual Alignment) model for
high-quality voice cloning. Two model variants:
- tada-1b: English-only, ~2B params (Llama 3.2 1B base)
- tada-3b-ml: Multilingual, ~4B params (Llama 3.2 3B base)
Both use a shared encoder/codec (HumeAI/tada-codec). The encoder
produces 1:1 aligned token embeddings from reference audio, and the
causal LM generates speech via flow-matching diffusion.
24kHz output, bf16 inference on CUDA, fp32 on CPU.
"""
import asyncio
import logging
import threading
from typing import ClassVar, List, Optional, Tuple
import numpy as np
from . import TTSBackend
from .base import (
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
logger = logging.getLogger(__name__)
# HuggingFace repos
TADA_CODEC_REPO = "HumeAI/tada-codec"
TADA_1B_REPO = "HumeAI/tada-1b"
TADA_3B_ML_REPO = "HumeAI/tada-3b-ml"
TADA_MODEL_REPOS = {
"1B": TADA_1B_REPO,
"3B": TADA_3B_ML_REPO,
}
# Key weight files for cache detection
_TADA_MODEL_WEIGHT_FILES = [
"model.safetensors",
]
_TADA_CODEC_WEIGHT_FILES = [
"encoder/model.safetensors",
]
class HumeTadaBackend:
"""HumeAI TADA TTS backend for high-quality voice cloning."""
_load_lock: ClassVar[threading.Lock] = threading.Lock()
def __init__(self):
self.model = None
self.encoder = None
self.model_size = "1B" # default to 1B
self._device = None
self._model_load_lock = asyncio.Lock()
def _get_device(self) -> str:
# Force CPU on macOS — MPS has issues with flow matching
# and large vocab lm_head (>65536 output channels)
return get_torch_device(force_cpu_on_mac=True, allow_xpu=True)
def is_loaded(self) -> bool:
return self.model is not None
def _get_model_path(self, model_size: str = "1B") -> str:
return TADA_MODEL_REPOS.get(model_size, TADA_1B_REPO)
def _is_model_cached(self, model_size: str = "1B") -> bool:
repo = TADA_MODEL_REPOS.get(model_size, TADA_1B_REPO)
model_cached = is_model_cached(repo, required_files=_TADA_MODEL_WEIGHT_FILES)
codec_cached = is_model_cached(TADA_CODEC_REPO, required_files=_TADA_CODEC_WEIGHT_FILES)
return model_cached and codec_cached
async def load_model(self, model_size: str = "1B") -> None:
"""Load the TADA model and encoder."""
if self.model is not None and self.model_size == model_size:
return
async with self._model_load_lock:
if self.model is not None and self.model_size == model_size:
return
# Unload existing model if switching sizes
if self.model is not None:
self.unload_model()
self.model_size = model_size
await asyncio.to_thread(self._load_model_sync, model_size)
def _load_model_sync(self, model_size: str = "1B"):
"""Synchronous model loading with progress tracking."""
model_name = f"tada-{model_size.lower()}"
is_cached = self._is_model_cached(model_size)
repo = TADA_MODEL_REPOS.get(model_size, TADA_1B_REPO)
with model_load_progress(model_name, is_cached):
# Install DAC shim before importing tada — tada's encoder/decoder
# import dac.nn.layers.Snake1d which requires the descript-audio-codec
# package. The real package pulls in onnx/tensorboard/matplotlib via
# descript-audiotools, so we use a lightweight shim instead.
from ..utils.dac_shim import install_dac_shim
install_dac_shim()
import torch
from huggingface_hub import snapshot_download
device = self._get_device()
self._device = device
logger.info(f"Loading HumeAI TADA {model_size} on {device}...")
# Download codec (encoder + decoder) if not cached
logger.info("Downloading TADA codec...")
snapshot_download(
repo_id=TADA_CODEC_REPO,
token=None,
allow_patterns=["*.safetensors", "*.json", "*.txt", "*.bin"],
)
# Download model weights if not cached
logger.info(f"Downloading TADA {model_size} model...")
snapshot_download(
repo_id=repo,
token=None,
allow_patterns=["*.safetensors", "*.json", "*.txt", "*.bin", "*.model"],
)
# TADA hardcodes "meta-llama/Llama-3.2-1B" as the tokenizer
# source in its Aligner and TadaForCausalLM.from_pretrained().
# That repo is gated (requires Meta license acceptance).
# Download the tokenizer from an ungated mirror and get its
# local cache path so we can point TADA at it directly.
logger.info("Downloading Llama tokenizer (ungated mirror)...")
tokenizer_path = snapshot_download(
repo_id="unsloth/Llama-3.2-1B",
token=None,
allow_patterns=["tokenizer*", "special_tokens*"],
)
# Determine dtype — use bf16 on CUDA/XPU for ~50% memory savings
if device == "cuda" and torch.cuda.is_bf16_supported():
model_dtype = torch.bfloat16
elif device == "xpu":
# Intel Arc (Alchemist+) supports bf16 natively
model_dtype = torch.bfloat16
else:
model_dtype = torch.float32
# Patch the Aligner config class to use the local tokenizer
# path instead of the gated "meta-llama/Llama-3.2-1B" default.
# This avoids monkey-patching AutoTokenizer.from_pretrained
# which corrupts the classmethod descriptor for other engines.
from tada.modules.aligner import AlignerConfig
AlignerConfig.tokenizer_name = tokenizer_path
# Load encoder (only needed for voice prompt encoding)
from tada.modules.encoder import Encoder
logger.info("Loading TADA encoder...")
self.encoder = Encoder.from_pretrained(TADA_CODEC_REPO, subfolder="encoder").to(device)
self.encoder.eval()
# Load the causal LM (includes decoder for wav generation).
# TadaForCausalLM.from_pretrained() calls
# getattr(config, "tokenizer_name", "meta-llama/Llama-3.2-1B")
# which hits the gated repo. Pre-load the config from HF,
# inject the local tokenizer path, then pass it in.
from tada.modules.tada import TadaForCausalLM, TadaConfig
logger.info(f"Loading TADA {model_size} model...")
config = TadaConfig.from_pretrained(repo)
config.tokenizer_name = tokenizer_path
self.model = TadaForCausalLM.from_pretrained(repo, config=config, torch_dtype=model_dtype).to(device)
self.model.eval()
logger.info(f"HumeAI TADA {model_size} loaded successfully on {device}")
def unload_model(self) -> None:
"""Unload model and encoder to free memory."""
if self.model is not None:
del self.model
self.model = None
if self.encoder is not None:
del self.encoder
self.encoder = None
device = self._device
self._device = None
if device:
empty_device_cache(device)
logger.info("HumeAI TADA unloaded")
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio using TADA's encoder.
TADA's encoder performs forced alignment between audio and text tokens,
producing an EncoderOutput with 1:1 token-audio alignment. If no
reference_text is provided, the encoder uses built-in ASR (English only).
We serialize the EncoderOutput to a dict for caching.
"""
await self.load_model(self.model_size)
cache_key = ("tada_" + get_cache_key(audio_path, reference_text)) if use_cache else None
if cache_key:
cached = get_cached_voice_prompt(cache_key)
if cached is not None and isinstance(cached, dict):
return cached, True
def _encode_sync():
import torch
import soundfile as sf
device = self._device
# Load audio with soundfile (torchaudio 2.10+ requires torchcodec)
audio_np, sr = sf.read(str(audio_path), dtype="float32")
audio = torch.from_numpy(audio_np).float()
if audio.ndim == 1:
audio = audio.unsqueeze(0) # (samples,) -> (1, samples)
else:
audio = audio.T # (samples, channels) -> (channels, samples)
audio = audio.to(device)
# Encode with forced alignment
text_arg = [reference_text] if reference_text else None
prompt = self.encoder(audio, text=text_arg, sample_rate=sr)
# Serialize EncoderOutput to a dict of CPU tensors for caching
prompt_dict = {}
for field_name in prompt.__dataclass_fields__:
val = getattr(prompt, field_name)
if isinstance(val, torch.Tensor):
prompt_dict[field_name] = val.detach().cpu()
elif isinstance(val, list):
prompt_dict[field_name] = val
elif isinstance(val, (int, float)):
prompt_dict[field_name] = val
else:
prompt_dict[field_name] = val
return prompt_dict
encoded = await asyncio.to_thread(_encode_sync)
if cache_key:
cache_voice_prompt(cache_key, encoded)
return encoded, False
async def combine_voice_prompts(
self,
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
return await _combine_voice_prompts(audio_paths, reference_texts, sample_rate=24000)
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text using HumeAI TADA.
Args:
text: Text to synthesize
voice_prompt: Serialized EncoderOutput dict from create_voice_prompt()
language: Language code (en, ar, de, es, fr, it, ja, pl, pt, zh)
seed: Random seed for reproducibility
instruct: Not supported by TADA (ignored)
Returns:
Tuple of (audio_array, sample_rate=24000)
"""
await self.load_model(self.model_size)
def _generate_sync():
import torch
from tada.modules.encoder import EncoderOutput
if seed is not None:
manual_seed(seed, self._device)
device = self._device
# Reconstruct EncoderOutput from the cached dict
restored = {}
for k, v in voice_prompt.items():
if isinstance(v, torch.Tensor):
# Move to device and match model dtype for float tensors
if v.is_floating_point():
model_dtype = next(self.model.parameters()).dtype
restored[k] = v.to(device=device, dtype=model_dtype)
else:
restored[k] = v.to(device=device)
else:
restored[k] = v
prompt = EncoderOutput(**restored)
# For non-English with the 3B-ML model, we could reload the
# encoder with the language-specific aligner. However, the
# generation itself is language-agnostic — only the encoder's
# aligner changes. Since we encode at create_voice_prompt time,
# the language is already baked in. For simplicity, we don't
# reload the encoder here.
logger.info(f"[TADA] Generating ({language}), text length: {len(text)}")
output = self.model.generate(
prompt=prompt,
text=text,
)
# output.audio is a list of tensors (one per batch item)
if output.audio and output.audio[0] is not None:
audio_tensor = output.audio[0]
audio = audio_tensor.detach().cpu().numpy().squeeze().astype(np.float32)
else:
logger.warning("[TADA] Generation produced no audio")
audio = np.zeros(24000, dtype=np.float32)
return audio, 24000
return await asyncio.to_thread(_generate_sync)
+17 -11
View File
@@ -12,7 +12,14 @@ from typing import Optional, Tuple
import numpy as np
from . import TTSBackend
from .base import is_model_cached, get_torch_device, combine_voice_prompts as _combine_voice_prompts, model_load_progress
from .base import (
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
logger = logging.getLogger(__name__)
@@ -30,7 +37,7 @@ class LuxTTSBackend:
self._device = None
def _get_device(self) -> str:
return get_torch_device(allow_mps=True)
return get_torch_device(allow_mps=True, allow_xpu=True)
def is_loaded(self) -> bool:
return self.model is not None
@@ -69,9 +76,12 @@ class LuxTTSBackend:
if device == "cpu":
import os
threads = os.cpu_count() or 4
self.model = LuxTTS(
model_path=LUXTTS_HF_REPO, device="cpu", threads=min(threads, 8),
model_path=LUXTTS_HF_REPO,
device="cpu",
threads=min(threads, 8),
)
else:
self.model = LuxTTS(model_path=LUXTTS_HF_REPO, device=device)
@@ -81,12 +91,12 @@ class LuxTTSBackend:
def unload_model(self) -> None:
"""Unload model to free memory."""
if self.model is not None:
device = self.device
del self.model
self.model = None
self._device = None
import torch
if torch.cuda.is_available():
torch.cuda.empty_cache()
empty_device_cache(device)
logger.info("LuxTTS unloaded")
@@ -154,12 +164,8 @@ class LuxTTSBackend:
await self.load_model()
def _generate_sync():
import torch
if seed is not None:
torch.manual_seed(seed)
if torch.cuda.is_available():
torch.cuda.manual_seed(seed)
manual_seed(seed, self.device)
wav = self.model.generate_speech(
text=text,
+4 -2
View File
@@ -345,18 +345,20 @@ class MLXSTTBackend:
self,
audio_path: str,
language: Optional[str] = None,
model_size: Optional[str] = None,
) -> str:
"""
Transcribe audio to text.
Args:
audio_path: Path to audio file
language: Optional language hint (en or zh)
language: Optional language hint
model_size: Optional model size override
Returns:
Transcribed text
"""
await self.load_model_async(None)
await self.load_model_async(model_size)
def _transcribe_sync():
"""Run synchronous transcription in thread pool."""
+9 -9
View File
@@ -14,6 +14,8 @@ from . import TTSBackend, STTBackend, LANGUAGE_CODE_TO_NAME, WHISPER_HF_REPOS
from .base import (
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
@@ -120,8 +122,7 @@ class PyTorchTTSBackend:
self.model = None
self._current_model_size = None
if torch.cuda.is_available():
torch.cuda.empty_cache()
empty_device_cache(self.device)
logger.info("TTS model unloaded")
@@ -213,9 +214,7 @@ class PyTorchTTSBackend:
"""Run synchronous generation in thread pool."""
# Set seed if provided
if seed is not None:
torch.manual_seed(seed)
if torch.cuda.is_available():
torch.cuda.manual_seed(seed)
manual_seed(seed, self.device)
# Generate audio - this is the blocking operation
wavs, sample_rate = self.model.generate_voice_clone(
@@ -297,8 +296,7 @@ class PyTorchSTTBackend:
self.model = None
self.processor = None
if torch.cuda.is_available():
torch.cuda.empty_cache()
empty_device_cache(self.device)
logger.info("Whisper model unloaded")
@@ -306,18 +304,20 @@ class PyTorchSTTBackend:
self,
audio_path: str,
language: Optional[str] = None,
model_size: Optional[str] = None,
) -> str:
"""
Transcribe audio to text.
Args:
audio_path: Path to audio file
language: Optional language hint (en or zh)
language: Optional language hint
model_size: Optional model size override
Returns:
Transcribed text
"""
await self.load_model_async(None)
await self.load_model_async(model_size)
def _transcribe_sync():
"""Run synchronous transcription in thread pool."""
+54 -6
View File
@@ -34,9 +34,15 @@ def build_server(cuda=False):
binary_name = "voicebox-server-cuda" if cuda else "voicebox-server"
# PyInstaller arguments
# CUDA builds use --onedir so we can split the output into two archives:
# 1. Server core (~200-400MB) — versioned with the app
# 2. CUDA libs (~2GB) — versioned independently (only redownloaded on
# CUDA toolkit / torch major version changes)
# CPU builds remain --onefile for simplicity.
pack_mode = "--onedir" if cuda else "--onefile"
args = [
"server.py", # Use server.py as entry point instead of main.py
"--onefile",
pack_mode,
"--name",
binary_name,
]
@@ -127,7 +133,13 @@ def build_server(cuda=False):
"uvicorn",
"--hidden-import",
"sqlalchemy",
"--hidden-import",
# librosa uses lazy_loader which generates .pyi stub files at
# install time and reads them at runtime to discover submodules.
# --hidden-import alone doesn't bundle the stubs, causing
# "Cannot load imports from non-existent stub" at runtime.
"--collect-all",
"lazy_loader",
"--collect-all",
"librosa",
"--hidden-import",
"soundfile",
@@ -159,9 +171,9 @@ def build_server(cuda=False):
"tqdm",
"--hidden-import",
"requests",
"--collect-submodules",
"qwen_tts",
"--collect-data",
# qwen_tts uses inspect.getsource() at runtime to locate
# modeling_qwen3_tts.py — needs physical .py source files bundled
"--collect-all",
"qwen_tts",
# Fix for pkg_resources and jaraco namespace packages
"--hidden-import",
@@ -180,6 +192,42 @@ def build_server(cuda=False):
# needed by LuxTTS for text-to-phoneme conversion
"--collect-all",
"piper_phonemize",
# HumeAI TADA — speech-language model using Llama + flow matching
"--hidden-import",
"backend.backends.hume_backend",
"--hidden-import",
"tada",
"--hidden-import",
"tada.modules",
"--hidden-import",
"tada.modules.tada",
"--hidden-import",
"tada.modules.encoder",
"--hidden-import",
"tada.modules.decoder",
"--hidden-import",
"tada.modules.aligner",
"--hidden-import",
"tada.modules.acoustic_spkr_verf",
"--hidden-import",
"tada.nn",
"--hidden-import",
"tada.nn.vibevoice",
"--hidden-import",
"tada.utils",
"--hidden-import",
"tada.utils.gray_code",
"--hidden-import",
"tada.utils.text",
# DAC shim — provides dac.nn.layers.Snake1d without the real
# descript-audio-codec package (which pulls onnx/tensorboard via
# descript-audiotools). The shim is in backend/utils/dac_shim.py.
"--hidden-import",
"backend.utils.dac_shim",
"--hidden-import",
"torchaudio",
"--collect-submodules",
"tada",
]
)
@@ -322,7 +370,7 @@ def build_server(cuda=False):
"torchvision",
"torchaudio",
"--index-url",
"https://download.pytorch.org/whl/cu126",
"https://download.pytorch.org/whl/cu128",
"--force-reinstall",
"-q",
],
+4 -3
View File
@@ -66,9 +66,9 @@ class GenerationRequest(BaseModel):
text: str = Field(..., min_length=1, max_length=50000)
language: str = Field(default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it|he|ar|da|el|fi|hi|ms|nl|no|pl|sv|sw|tr)$")
seed: Optional[int] = Field(None, ge=0)
model_size: Optional[str] = Field(default="1.7B", pattern="^(1\\.7B|0\\.6B)$")
model_size: Optional[str] = Field(default="1.7B", pattern="^(1\\.7B|0\\.6B|1B|3B)$")
instruct: Optional[str] = Field(None, max_length=500)
engine: Optional[str] = Field(default="qwen", pattern="^(qwen|luxtts|chatterbox|chatterbox_turbo)$")
engine: Optional[str] = Field(default="qwen", pattern="^(qwen|luxtts|chatterbox|chatterbox_turbo|tada)$")
max_chunk_chars: int = Field(
default=800, ge=100, le=5000, description="Max characters per chunk for long text splitting"
)
@@ -149,7 +149,8 @@ class HistoryListResponse(BaseModel):
class TranscriptionRequest(BaseModel):
"""Request model for audio transcription."""
language: Optional[str] = Field(None, pattern="^(en|zh)$")
language: Optional[str] = Field(None, pattern="^(en|zh|ja|ko|de|fr|ru|pt|es|it)$")
model: Optional[str] = Field(None, pattern="^(base|small|medium|large|turbo)$")
class TranscriptionResponse(BaseModel):
+8 -1
View File
@@ -8,7 +8,7 @@ sqlalchemy>=2.0.0
alembic>=1.13.0
# ML models
torch>=2.1.0
torch>=2.7.0
transformers>=4.36.0,<=4.57.6
accelerate>=0.26.0
huggingface_hub>=0.20.0
@@ -33,6 +33,13 @@ s3tokenizer
spacy-pkuseg
pyloudnorm
# HumeAI TADA sub-dependencies (hume-tada itself is installed
# --no-deps in the setup script because it pins torch>=2.7,<2.8.
# descript-audio-codec is NOT installed — it pulls onnx/tensorboard
# via descript-audiotools. A lightweight shim in utils/dac_shim.py
# provides the only class TADA uses: Snake1d.)
torchaudio
# Audio processing
librosa>=0.10.0
soundfile>=0.12.0
+28 -19
View File
@@ -1,12 +1,15 @@
"""TTS generation endpoints."""
import asyncio
import logging
import uuid
from fastapi import APIRouter, Depends, HTTPException
from fastapi.responses import StreamingResponse
from sqlalchemy.orm import Session
logger = logging.getLogger(__name__)
from .. import models
from ..services import history, profiles, tts
from ..database import Generation as DBGeneration, VoiceProfile as DBVoiceProfile, get_db
@@ -181,25 +184,28 @@ async def get_generation_status(generation_id: str, db: Session = Depends(get_db
import json
async def event_stream():
while True:
db.expire_all()
gen = db.query(DBGeneration).filter_by(id=generation_id).first()
if not gen:
yield f"data: {json.dumps({'status': 'not_found', 'id': generation_id})}\n\n"
return
try:
while True:
db.expire_all()
gen = db.query(DBGeneration).filter_by(id=generation_id).first()
if not gen:
yield f"data: {json.dumps({'status': 'not_found', 'id': generation_id})}\n\n"
return
payload = {
"id": gen.id,
"status": gen.status or "completed",
"duration": gen.duration,
"error": gen.error,
}
yield f"data: {json.dumps(payload)}\n\n"
payload = {
"id": gen.id,
"status": gen.status or "completed",
"duration": gen.duration,
"error": gen.error,
}
yield f"data: {json.dumps(payload)}\n\n"
if (gen.status or "completed") in ("completed", "failed"):
return
if (gen.status or "completed") in ("completed", "failed"):
return
await asyncio.sleep(1)
await asyncio.sleep(1)
except (BrokenPipeError, ConnectionResetError, asyncio.CancelledError):
logger.debug("SSE client disconnected for generation %s", generation_id)
return StreamingResponse(
event_stream(),
@@ -265,9 +271,12 @@ async def stream_speech(
wav_bytes = tts.audio_to_wav_bytes(audio, sample_rate)
async def _wav_stream():
chunk_size = 64 * 1024
for i in range(0, len(wav_bytes), chunk_size):
yield wav_bytes[i : i + chunk_size]
try:
chunk_size = 64 * 1024
for i in range(0, len(wav_bytes), chunk_size):
yield wav_bytes[i : i + chunk_size]
except (BrokenPipeError, ConnectionResetError, asyncio.CancelledError):
logger.debug("Client disconnected during audio stream")
return StreamingResponse(
_wav_stream(),
+19 -3
View File
@@ -3,9 +3,11 @@
import asyncio
import os
import signal
from pathlib import Path
import torch
from fastapi import APIRouter, Depends
from fastapi.responses import FileResponse
from sqlalchemy.orm import Session
from .. import config, models
@@ -15,12 +17,18 @@ from ..utils.platform_detect import get_backend_type
router = APIRouter()
# Frontend build directory — present in Docker, absent in dev/API-only mode
_frontend_dir = Path(__file__).resolve().parent.parent.parent / "frontend"
@router.get("/")
async def root():
"""Root endpoint."""
"""Root endpoint — serves SPA index.html in Docker, JSON otherwise."""
from .. import __version__
index = _frontend_dir / "index.html"
if index.is_file():
return FileResponse(index, media_type="text/html")
return {"message": "voicebox API", "version": __version__}
@@ -102,6 +110,11 @@ async def health():
vram_used = None
if has_cuda:
vram_used = torch.cuda.memory_allocated() / 1024 / 1024
elif has_xpu:
try:
vram_used = torch.xpu.memory_allocated() / 1024 / 1024
except Exception:
pass # memory_allocated() may not be available on all IPEX versions
model_loaded = False
model_size = None
@@ -154,7 +167,10 @@ async def health():
gpu_type=gpu_type,
vram_used_mb=vram_used,
backend_type=backend_type,
backend_variant=os.environ.get("VOICEBOX_BACKEND_VARIANT", "cuda" if torch.cuda.is_available() else "cpu"),
backend_variant=os.environ.get(
"VOICEBOX_BACKEND_VARIANT",
"cuda" if torch.cuda.is_available() else ("xpu" if has_xpu else "cpu"),
),
)
@@ -199,7 +215,7 @@ async def filesystem_health():
checks.append(
models.DirectoryCheck(
path=str(dir_path),
path=str(dir_path.resolve()),
exists=exists,
writable=writable,
error=error,
+14 -2
View File
@@ -102,6 +102,10 @@ async def delete_profile(
return {"message": "Profile deleted successfully"}
SAMPLE_MAX_FILE_SIZE = 50 * 1024 * 1024 # 50 MB
SAMPLE_UPLOAD_CHUNK_SIZE = 1024 * 1024 # 1 MB
@router.post("/profiles/{profile_id}/samples", response_model=models.ProfileSampleResponse)
async def add_profile_sample(
profile_id: str,
@@ -115,8 +119,16 @@ async def add_profile_sample(
file_suffix = _uploaded_ext if _uploaded_ext in _allowed_audio_exts else ".wav"
with tempfile.NamedTemporaryFile(suffix=file_suffix, delete=False) as tmp:
content = await file.read()
tmp.write(content)
total_size = 0
while chunk := await file.read(SAMPLE_UPLOAD_CHUNK_SIZE):
total_size += len(chunk)
if total_size > SAMPLE_MAX_FILE_SIZE:
Path(tmp.name).unlink(missing_ok=True)
raise HTTPException(
status_code=413,
detail=f"File too large (max {SAMPLE_MAX_FILE_SIZE // (1024 * 1024)} MB)",
)
tmp.write(chunk)
tmp_path = tmp.name
try:
+13 -3
View File
@@ -20,6 +20,7 @@ UPLOAD_CHUNK_SIZE = 1024 * 1024 # 1MB
async def transcribe_audio(
file: UploadFile = File(...),
language: str | None = Form(None),
model: str | None = Form(None),
):
"""Transcribe audio file to text."""
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
@@ -29,14 +30,23 @@ async def transcribe_audio(
try:
from ..utils.audio import load_audio
from ..backends import WHISPER_HF_REPOS
audio, sr = await asyncio.to_thread(load_audio, tmp_path)
duration = len(audio) / sr
whisper_model = transcribe.get_whisper_model()
model_size = whisper_model.model_size
model_size = model if model else whisper_model.model_size
if not whisper_model.is_loaded() and not whisper_model._is_model_cached(model_size):
valid_sizes = list(WHISPER_HF_REPOS.keys())
if model_size not in valid_sizes:
raise HTTPException(
status_code=400,
detail=f"Invalid model size '{model_size}'. Must be one of: {', '.join(valid_sizes)}",
)
already_loaded = whisper_model.is_loaded() and whisper_model.model_size == model_size
if not already_loaded and not whisper_model._is_model_cached(model_size):
progress_model_name = f"whisper-{model_size}"
task_manager = get_task_manager()
@@ -59,7 +69,7 @@ async def transcribe_audio(
},
)
text = await whisper_model.transcribe(tmp_path, language)
text = await whisper_model.transcribe(tmp_path, language, model_size)
return models.TranscriptionResponse(
text=text,
+266 -119
View File
@@ -1,16 +1,22 @@
"""
CUDA backend binary download, assembly, and verification.
CUDA backend download, assembly, and verification.
Downloads split parts of the CUDA-enabled voicebox-server binary from
GitHub Releases, reassembles them, verifies integrity via SHA-256,
and places the binary in the app's data directory for use on next
backend restart.
Downloads two archives from GitHub Releases:
1. Server core (voicebox-server-cuda.tar.gz) — the exe + non-NVIDIA deps,
versioned with the app.
2. CUDA libs (cuda-libs-{version}.tar.gz) — NVIDIA runtime libraries,
versioned independently (only redownloaded on CUDA toolkit bump).
Both archives are extracted into {data_dir}/backends/cuda/ which forms the
complete PyInstaller --onedir directory structure that torch expects.
"""
import hashlib
import json
import logging
import os
import sys
import tarfile
from pathlib import Path
from typing import Optional
@@ -24,6 +30,10 @@ GITHUB_RELEASES_URL = "https://github.com/jamiepine/voicebox/releases/download"
PROGRESS_KEY = "cuda-backend"
# The current expected CUDA libs version. Bump this when we change the
# CUDA toolkit version or torch's CUDA dependency changes (e.g. cu126 -> cu128).
CUDA_LIBS_VERSION = "cu128-v1"
def get_backends_dir() -> Path:
"""Directory where downloaded backend binaries are stored."""
@@ -32,21 +42,46 @@ def get_backends_dir() -> Path:
return d
def get_cuda_binary_name() -> str:
"""Platform-specific CUDA binary filename."""
def get_cuda_dir() -> Path:
"""Directory where the CUDA backend (onedir) is extracted."""
d = get_backends_dir() / "cuda"
d.mkdir(parents=True, exist_ok=True)
return d
def get_cuda_exe_name() -> str:
"""Platform-specific CUDA executable filename."""
if sys.platform == "win32":
return "voicebox-server-cuda.exe"
return "voicebox-server-cuda"
def get_cuda_binary_path() -> Optional[Path]:
"""Return path to CUDA binary if it exists."""
p = get_backends_dir() / get_cuda_binary_name()
"""Return path to the CUDA executable if it exists inside the onedir."""
p = get_cuda_dir() / get_cuda_exe_name()
if p.exists():
return p
return None
def get_cuda_libs_manifest_path() -> Path:
"""Path to the cuda-libs.json manifest inside the CUDA dir."""
return get_cuda_dir() / "cuda-libs.json"
def get_installed_cuda_libs_version() -> Optional[str]:
"""Read the installed CUDA libs version from cuda-libs.json, or None."""
manifest_path = get_cuda_libs_manifest_path()
if not manifest_path.exists():
return None
try:
data = json.loads(manifest_path.read_text())
return data.get("version")
except Exception as e:
logger.warning(f"Could not read cuda-libs.json: {e}")
return None
def is_cuda_active() -> bool:
"""Check if the current process is the CUDA binary.
@@ -60,25 +95,151 @@ def get_cuda_status() -> dict:
progress_manager = get_progress_manager()
cuda_path = get_cuda_binary_path()
progress = progress_manager.get_progress(PROGRESS_KEY)
cuda_libs_version = get_installed_cuda_libs_version()
return {
"available": cuda_path is not None,
"active": is_cuda_active(),
"binary_path": str(cuda_path) if cuda_path else None,
"cuda_libs_version": cuda_libs_version,
"downloading": progress is not None and progress.get("status") == "downloading",
"download_progress": progress,
}
async def download_cuda_binary(version: Optional[str] = None):
"""Download the CUDA backend binary from GitHub Releases.
def _needs_server_download(version: Optional[str] = None) -> bool:
"""Check if the server core archive needs to be (re)downloaded."""
cuda_path = get_cuda_binary_path()
if not cuda_path:
return True
# Check if the binary version matches the expected app version
installed = get_cuda_binary_version()
expected = version or __version__
if expected.startswith("v"):
expected = expected[1:]
return installed != expected
Downloads split parts listed in a manifest file, concatenates them,
and verifies the SHA-256 checksum for integrity. Atomic write
(temp file -> rename).
def _needs_cuda_libs_download() -> bool:
"""Check if the CUDA libs archive needs to be (re)downloaded."""
installed = get_installed_cuda_libs_version()
if installed is None:
return True
return installed != CUDA_LIBS_VERSION
async def _download_and_extract_archive(
client,
url: str,
sha256_url: Optional[str],
dest_dir: Path,
label: str,
progress_offset: int,
total_size: int,
):
"""Download a .tar.gz archive and extract it into dest_dir.
Args:
version: Version tag (e.g. "v0.2.0"). Defaults to current app version.
client: httpx.AsyncClient
url: URL of the .tar.gz archive
sha256_url: URL of the .sha256 checksum file (optional)
dest_dir: Directory to extract into
label: Human-readable label for progress updates
progress_offset: Byte offset for progress reporting (when downloading
multiple archives sequentially)
total_size: Total bytes across all downloads (for progress bar)
"""
progress = get_progress_manager()
temp_path = dest_dir / f".download-{label.replace(' ', '-')}.tmp"
# Clean up leftover partial download
if temp_path.exists():
temp_path.unlink()
# Fetch expected checksum (fail-fast: never extract an unverified archive)
expected_sha = None
if sha256_url:
try:
sha_resp = await client.get(sha256_url)
sha_resp.raise_for_status()
expected_sha = sha_resp.text.strip().split()[0]
logger.info(f"{label}: expected SHA-256: {expected_sha[:16]}...")
except Exception as e:
raise RuntimeError(f"{label}: failed to fetch checksum from {sha256_url}") from e
# Stream download, verify, and extract — always clean up temp file
downloaded = 0
try:
async with client.stream("GET", url) as response:
response.raise_for_status()
with open(temp_path, "wb") as f:
async for chunk in response.aiter_bytes(chunk_size=1024 * 1024):
f.write(chunk)
downloaded += len(chunk)
progress.update_progress(
PROGRESS_KEY,
current=progress_offset + downloaded,
total=total_size,
filename=f"Downloading {label}",
status="downloading",
)
# Verify integrity
if expected_sha:
progress.update_progress(
PROGRESS_KEY,
current=progress_offset + downloaded,
total=total_size,
filename=f"Verifying {label}...",
status="downloading",
)
sha256 = hashlib.sha256()
with open(temp_path, "rb") as f:
while True:
data = f.read(1024 * 1024)
if not data:
break
sha256.update(data)
actual = sha256.hexdigest()
if actual != expected_sha:
raise ValueError(
f"{label} integrity check failed: expected {expected_sha[:16]}..., got {actual[:16]}..."
)
logger.info(f"{label}: integrity verified")
# Extract (use data filter for path traversal protection on Python 3.12+)
progress.update_progress(
PROGRESS_KEY,
current=progress_offset + downloaded,
total=total_size,
filename=f"Extracting {label}...",
status="downloading",
)
with tarfile.open(temp_path, "r:gz") as tar:
if sys.version_info >= (3, 12):
tar.extractall(path=dest_dir, filter="data")
else:
tar.extractall(path=dest_dir)
logger.info(f"{label}: extracted to {dest_dir}")
finally:
if temp_path.exists():
temp_path.unlink()
return downloaded
async def download_cuda_binary(version: Optional[str] = None):
"""Download the CUDA backend (server core + CUDA libs if needed).
Downloads both archives from GitHub Releases, extracts them into
{data_dir}/backends/cuda/, and writes the cuda-libs.json manifest.
Only downloads what's needed:
- Server core: always redownloaded (versioned with app)
- CUDA libs: only if missing or version mismatch
Args:
version: Version tag (e.g. "v0.3.0"). Defaults to current app version.
"""
import httpx
@@ -86,114 +247,91 @@ async def download_cuda_binary(version: Optional[str] = None):
version = f"v{__version__}"
progress = get_progress_manager()
binary_name = get_cuda_binary_name()
dest_dir = get_backends_dir()
final_path = dest_dir / binary_name
temp_path = dest_dir / f"{binary_name}.download"
cuda_dir = get_cuda_dir()
# Clean up any leftover partial download
if temp_path.exists():
temp_path.unlink()
need_server = _needs_server_download(version)
need_libs = _needs_cuda_libs_download()
logger.info(f"Starting CUDA backend download for {version}")
if not need_server and not need_libs:
logger.info("CUDA backend is up to date, nothing to download")
return
logger.info(
f"Starting CUDA backend download for {version} "
f"(server={'yes' if need_server else 'cached'}, "
f"libs={'yes' if need_libs else 'cached'})"
)
progress.update_progress(
PROGRESS_KEY, current=0, total=0,
filename="Fetching manifest...", status="downloading",
PROGRESS_KEY,
current=0,
total=0,
filename="Preparing download...",
status="downloading",
)
base_url = f"{GITHUB_RELEASES_URL}/{version}"
stem = Path(binary_name).stem # voicebox-server-cuda
server_archive = "voicebox-server-cuda.tar.gz"
libs_archive = f"cuda-libs-{CUDA_LIBS_VERSION}.tar.gz"
try:
async with httpx.AsyncClient(follow_redirects=True, timeout=30.0) as client:
# Fetch the manifest (list of split part filenames)
manifest_url = f"{base_url}/{stem}.manifest"
manifest_resp = await client.get(manifest_url)
manifest_resp.raise_for_status()
parts = [p.strip() for p in manifest_resp.text.strip().splitlines() if p.strip()]
if not parts:
raise ValueError("Empty manifest — no split parts found")
logger.info(f"Found {len(parts)} split parts to download")
# Fetch expected checksum (optional — for integrity verification)
expected_sha = None
try:
sha_url = f"{base_url}/{stem}.sha256"
sha_resp = await client.get(sha_url)
if sha_resp.status_code == 200:
# Format: "sha256hex filename\n"
expected_sha = sha_resp.text.strip().split()[0]
logger.info(f"Expected SHA-256: {expected_sha[:16]}...")
except Exception as e:
logger.warning(f"Could not fetch checksum file — skipping verification: {e}")
# Get total size across all parts by issuing HEAD requests
# Estimate total download size
total_size = 0
for part_name in parts:
if need_server:
try:
head_resp = await client.head(f"{base_url}/{part_name}")
content_length = int(head_resp.headers.get("content-length", 0))
total_size += content_length
head = await client.head(f"{base_url}/{server_archive}")
total_size += int(head.headers.get("content-length", 0))
except Exception:
pass
if need_libs:
try:
head = await client.head(f"{base_url}/{libs_archive}")
total_size += int(head.headers.get("content-length", 0))
except Exception:
pass
logger.info(f"Total download size: {total_size / 1024 / 1024:.1f} MB")
# Download and concatenate parts
total_downloaded = 0
with open(temp_path, "wb") as f:
for i, part_name in enumerate(parts):
part_url = f"{base_url}/{part_name}"
logger.info(f"Downloading part {i + 1}/{len(parts)}: {part_name}")
offset = 0
async with client.stream("GET", part_url) as response:
response.raise_for_status()
async for chunk in response.aiter_bytes(chunk_size=1024 * 1024):
f.write(chunk)
total_downloaded += len(chunk)
progress.update_progress(
PROGRESS_KEY, current=total_downloaded, total=total_size,
filename=f"Downloading CUDA backend ({i + 1}/{len(parts)})",
status="downloading",
)
# Verify integrity if checksum was available
if expected_sha:
progress.update_progress(
PROGRESS_KEY, current=total_downloaded, total=total_downloaded,
filename="Verifying integrity...", status="downloading",
)
sha256 = hashlib.sha256()
with open(temp_path, "rb") as f:
while True:
chunk = f.read(1024 * 1024)
if not chunk:
break
sha256.update(chunk)
actual = sha256.hexdigest()
if actual != expected_sha:
raise ValueError(
f"Integrity check failed: expected {expected_sha[:16]}..., "
f"got {actual[:16]}..."
# Download server core
if need_server:
server_downloaded = await _download_and_extract_archive(
client,
url=f"{base_url}/{server_archive}",
sha256_url=f"{base_url}/{server_archive}.sha256",
dest_dir=cuda_dir,
label="CUDA server",
progress_offset=offset,
total_size=total_size,
)
logger.info(f"Integrity verified: {actual[:16]}...")
offset += server_downloaded
# Atomic move into place (replace handles existing target on all platforms)
temp_path.replace(final_path)
# Make executable on Unix
exe_path = cuda_dir / get_cuda_exe_name()
if sys.platform != "win32" and exe_path.exists():
exe_path.chmod(0o755)
# Make executable on Unix
if sys.platform != "win32":
final_path.chmod(0o755)
# Download CUDA libs
if need_libs:
await _download_and_extract_archive(
client,
url=f"{base_url}/{libs_archive}",
sha256_url=f"{base_url}/{libs_archive}.sha256",
dest_dir=cuda_dir,
label="CUDA libraries",
progress_offset=offset,
total_size=total_size,
)
logger.info(f"CUDA backend downloaded to {final_path}")
# Write local cuda-libs.json manifest
manifest = {"version": CUDA_LIBS_VERSION}
get_cuda_libs_manifest_path().write_text(json.dumps(manifest, indent=2) + "\n")
logger.info(f"CUDA backend ready at {cuda_dir}")
progress.mark_complete(PROGRESS_KEY)
except Exception as e:
# Clean up on failure
if temp_path.exists():
temp_path.unlink()
logger.error(f"CUDA backend download failed: {e}")
progress.mark_error(PROGRESS_KEY, str(e))
raise
@@ -202,15 +340,19 @@ async def download_cuda_binary(version: Optional[str] = None):
def get_cuda_binary_version() -> Optional[str]:
"""Get the version of the installed CUDA binary, or None if not installed."""
import subprocess
cuda_path = get_cuda_binary_path()
if not cuda_path:
return None
try:
result = subprocess.run(
[str(cuda_path), "--version"],
capture_output=True, text=True, timeout=30,
capture_output=True,
text=True,
timeout=30,
cwd=str(cuda_path.parent), # Run from the onedir directory
)
# Output format: "voicebox-server 0.2.0"
# Output format: "voicebox-server 0.3.0"
for line in result.stdout.strip().splitlines():
if "voicebox-server" in line:
return line.split()[-1]
@@ -222,26 +364,29 @@ def get_cuda_binary_version() -> Optional[str]:
async def check_and_update_cuda_binary():
"""Check if the CUDA binary is outdated and auto-download if so.
Called on server startup. If a CUDA binary exists but its version
doesn't match the current app version, triggers a background download
of the updated CUDA binary. The download progress is visible to the
frontend via the existing SSE progress endpoint.
Called on server startup. Checks both server version and CUDA libs
version. Downloads only what's needed.
"""
cuda_path = get_cuda_binary_path()
if not cuda_path:
return # No CUDA binary installed, nothing to update
cuda_version = get_cuda_binary_version()
current_version = __version__
need_server = _needs_server_download()
need_libs = _needs_cuda_libs_download()
if cuda_version == current_version:
logger.info(f"CUDA binary is up to date (v{current_version})")
if not need_server and not need_libs:
logger.info(f"CUDA binary is up to date (server=v{__version__}, libs={get_installed_cuda_libs_version()})")
return
logger.info(
f"CUDA binary version mismatch: binary=v{cuda_version}, app=v{current_version}. "
f"Auto-downloading updated CUDA backend..."
)
reasons = []
if need_server:
cuda_version = get_cuda_binary_version()
reasons.append(f"server v{cuda_version} != v{__version__}")
if need_libs:
installed_libs = get_installed_cuda_libs_version()
reasons.append(f"libs {installed_libs} != {CUDA_LIBS_VERSION}")
logger.info(f"CUDA backend needs update ({', '.join(reasons)}). Auto-downloading...")
try:
await download_cuda_binary()
@@ -250,10 +395,12 @@ async def check_and_update_cuda_binary():
async def delete_cuda_binary() -> bool:
"""Delete the downloaded CUDA binary. Returns True if deleted."""
path = get_cuda_binary_path()
if path and path.exists():
path.unlink()
logger.info(f"Deleted CUDA binary: {path}")
"""Delete the downloaded CUDA backend directory. Returns True if deleted."""
import shutil
cuda_dir = get_cuda_dir()
if cuda_dir.exists() and any(cuda_dir.iterdir()):
shutil.rmtree(cuda_dir)
logger.info(f"Deleted CUDA backend directory: {cuda_dir}")
return True
return False
+8 -4
View File
@@ -22,7 +22,7 @@ from ..database import (
Generation as DBGeneration,
)
from ..models import EffectConfig
from ..utils.audio import validate_reference_audio, load_audio, save_audio
from ..utils.audio import validate_reference_audio, validate_and_load_reference_audio, load_audio, save_audio
from ..utils.images import validate_image, process_avatar
from ..utils.cache import _get_cache_dir, clear_profile_cache
from .tts import get_tts_model
@@ -117,11 +117,16 @@ async def add_profile_sample(
Returns:
Created sample
"""
import asyncio
profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
if not profile:
raise ValueError(f"Profile {profile_id} not found")
is_valid, error_msg = validate_reference_audio(audio_path)
# Validate and load audio in a single pass, off the event loop
is_valid, error_msg, audio, sr = await asyncio.to_thread(
validate_and_load_reference_audio, audio_path
)
if not is_valid:
raise ValueError(f"Invalid reference audio: {error_msg}")
@@ -130,8 +135,7 @@ async def add_profile_sample(
profile_dir.mkdir(parents=True, exist_ok=True)
dest_path = profile_dir / f"{sample_id}.wav"
audio, sr = load_audio(audio_path)
save_audio(audio, str(dest_path), sr)
await asyncio.to_thread(save_audio, audio, str(dest_path), sr)
db_sample = DBProfileSample(
id=sample_id,
+24 -6
View File
@@ -217,22 +217,40 @@ def validate_reference_audio(
Returns:
Tuple of (is_valid, error_message)
"""
result = validate_and_load_reference_audio(
audio_path, min_duration, max_duration, min_rms
)
return (result[0], result[1])
def validate_and_load_reference_audio(
audio_path: str,
min_duration: float = 2.0,
max_duration: float = 30.0,
min_rms: float = 0.01,
) -> Tuple[bool, Optional[str], Optional[np.ndarray], Optional[int]]:
"""
Validate and load reference audio in a single pass.
Returns:
Tuple of (is_valid, error_message, audio_array, sample_rate)
"""
try:
audio, sr = load_audio(audio_path)
duration = len(audio) / sr
if duration < min_duration:
return False, f"Audio too short (minimum {min_duration} seconds)"
return False, f"Audio too short (minimum {min_duration} seconds)", None, None
if duration > max_duration:
return False, f"Audio too long (maximum {max_duration} seconds)"
return False, f"Audio too long (maximum {max_duration} seconds)", None, None
rms = np.sqrt(np.mean(audio**2))
if rms < min_rms:
return False, "Audio is too quiet or silent"
return False, "Audio is too quiet or silent", None, None
if np.abs(audio).max() > 0.99:
return False, "Audio is clipping (reduce input gain)"
return False, "Audio is clipping (reduce input gain)", None, None
return True, None
return True, None, audio, sr
except Exception as e:
return False, f"Error validating audio: {str(e)}"
return False, f"Error validating audio: {str(e)}", None, None
+95
View File
@@ -0,0 +1,95 @@
"""
Minimal shim for descript-audio-codec (DAC).
TADA only imports Snake1d from dac.nn.layers and dac.model.dac.
The real DAC package pulls in descript-audiotools which depends on
onnx, tensorboard, protobuf, matplotlib, pystoi, etc. — none of
which are needed for TADA's runtime use of Snake1d.
This shim provides the exact Snake1d implementation (MIT-licensed,
from https://github.com/descriptinc/descript-audio-codec) so we can
avoid the entire audiotools dependency chain.
If the real DAC package is installed, this module is never used —
Python's import system will find the site-packages version first.
Install this shim only when descript-audio-codec is NOT installed.
"""
import sys
import types
import torch
import torch.nn as nn
# ── Snake activation (from dac/nn/layers.py) ────────────────────────
# NOTE: The original DAC code uses @torch.jit.script here for a 1.4x
# speedup. We omit it because TorchScript calls inspect.getsource()
# which fails inside a PyInstaller frozen binary (no .py source files).
def snake(x: torch.Tensor, alpha: torch.Tensor) -> torch.Tensor:
shape = x.shape
x = x.reshape(shape[0], shape[1], -1)
x = x + (alpha + 1e-9).reciprocal() * torch.sin(alpha * x).pow(2)
x = x.reshape(shape)
return x
class Snake1d(nn.Module):
def __init__(self, channels: int):
super().__init__()
self.alpha = nn.Parameter(torch.ones(1, channels, 1))
def forward(self, x: torch.Tensor) -> torch.Tensor:
return snake(x, self.alpha)
# ── Register as dac.nn.layers and dac.model.dac ─────────────────────
def install_dac_shim() -> None:
"""Register fake dac package modules in sys.modules.
Only installs the shim if 'dac' is not already importable
(i.e. the real descript-audio-codec is not installed).
"""
try:
import dac # noqa: F401 — real package exists, do nothing
return
except ImportError:
pass
# Create the module tree: dac -> dac.nn -> dac.nn.layers
# -> dac.model -> dac.model.dac
dac_pkg = types.ModuleType("dac")
dac_pkg.__path__ = [] # make it a package
dac_pkg.__package__ = "dac"
dac_nn = types.ModuleType("dac.nn")
dac_nn.__path__ = []
dac_nn.__package__ = "dac.nn"
dac_nn_layers = types.ModuleType("dac.nn.layers")
dac_nn_layers.__package__ = "dac.nn"
dac_nn_layers.Snake1d = Snake1d
dac_nn_layers.snake = snake
dac_model = types.ModuleType("dac.model")
dac_model.__path__ = []
dac_model.__package__ = "dac.model"
dac_model_dac = types.ModuleType("dac.model.dac")
dac_model_dac.__package__ = "dac.model"
dac_model_dac.Snake1d = Snake1d
# Wire up submodules
dac_pkg.nn = dac_nn
dac_pkg.model = dac_model
dac_nn.layers = dac_nn_layers
dac_model.dac = dac_model_dac
# Register in sys.modules
sys.modules["dac"] = dac_pkg
sys.modules["dac.nn"] = dac_nn
sys.modules["dac.nn.layers"] = dac_nn_layers
sys.modules["dac.model"] = dac_model
sys.modules["dac.model.dac"] = dac_model_dac
+2
View File
@@ -246,6 +246,8 @@ class ProgressManager:
# Send heartbeat
yield ": heartbeat\n\n"
continue
except (BrokenPipeError, ConnectionResetError, asyncio.CancelledError):
logger.debug(f"SSE client disconnected from {model_name}")
finally:
# Remove from listeners
if model_name in self._listeners:
+11 -1
View File
@@ -6,7 +6,7 @@ from PyInstaller.utils.hooks import copy_metadata
datas = []
binaries = []
hiddenimports = ['backend', 'backend.main', 'backend.config', 'backend.database', 'backend.models', 'backend.profiles', 'backend.history', 'backend.tts', 'backend.transcribe', 'backend.platform_detect', 'backend.backends', 'backend.backends.pytorch_backend', 'backend.utils.audio', 'backend.utils.cache', 'backend.utils.progress', 'backend.utils.hf_progress', 'backend.utils.validation', 'backend.cuda_download', 'backend.effects', 'backend.utils.effects', 'backend.versions', 'pedalboard', 'chatterbox', 'chatterbox.tts_turbo', 'chatterbox.mtl_tts', 'backend.backends.chatterbox_backend', 'backend.backends.chatterbox_turbo_backend', 'backend.backends.luxtts_backend', 'zipvoice', 'zipvoice.luxvoice', 'torch', 'transformers', 'fastapi', 'uvicorn', 'sqlalchemy', 'librosa', 'soundfile', 'qwen_tts', 'qwen_tts.inference', 'qwen_tts.inference.qwen3_tts_model', 'qwen_tts.inference.qwen3_tts_tokenizer', 'qwen_tts.core', 'qwen_tts.cli', 'requests', 'pkg_resources.extern', 'backend.backends.mlx_backend', 'mlx', 'mlx.core', 'mlx.nn', 'mlx_audio', 'mlx_audio.tts', 'mlx_audio.stt']
hiddenimports = ['backend', 'backend.main', 'backend.config', 'backend.database', 'backend.models', 'backend.services.profiles', 'backend.services.history', 'backend.services.tts', 'backend.services.transcribe', 'backend.utils.platform_detect', 'backend.backends', 'backend.backends.pytorch_backend', 'backend.utils.audio', 'backend.utils.cache', 'backend.utils.progress', 'backend.utils.hf_progress', 'backend.services.cuda', 'backend.services.effects', 'backend.utils.effects', 'backend.services.versions', 'pedalboard', 'chatterbox', 'chatterbox.tts_turbo', 'chatterbox.mtl_tts', 'backend.backends.chatterbox_backend', 'backend.backends.chatterbox_turbo_backend', 'backend.backends.luxtts_backend', 'zipvoice', 'zipvoice.luxvoice', 'torch', 'transformers', 'fastapi', 'uvicorn', 'sqlalchemy', 'soundfile', 'qwen_tts', 'qwen_tts.inference', 'qwen_tts.inference.qwen3_tts_model', 'qwen_tts.inference.qwen3_tts_tokenizer', 'qwen_tts.core', 'qwen_tts.cli', 'requests', 'pkg_resources.extern', 'backend.backends.mlx_backend', 'mlx', 'mlx.core', 'mlx.nn', 'mlx_audio', 'mlx_audio.tts', 'mlx_audio.stt']
datas += collect_data_files('qwen_tts')
datas += copy_metadata('qwen-tts')
datas += copy_metadata('requests')
@@ -23,6 +23,16 @@ tmp_ret = collect_all('zipvoice')
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
tmp_ret = collect_all('linacodec')
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
tmp_ret = collect_all('lazy_loader')
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
tmp_ret = collect_all('librosa')
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
tmp_ret = collect_all('inflect')
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
tmp_ret = collect_all('perth')
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
tmp_ret = collect_all('piper_phonemize')
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
tmp_ret = collect_all('mlx')
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
tmp_ret = collect_all('mlx_audio')
+25 -2
View File
@@ -1,3 +1,26 @@
node_modules
.mintlify
# deps
/node_modules
# generated content
.source
# test & build
/coverage
/.next/
/out/
/build
*.tsbuildinfo
# misc
.DS_Store
*.pem
/.pnp
.pnp.js
npm-debug.log*
yarn-debug.log*
yarn-error.log*
# others
.env*.local
.vercel
next-env.d.ts
-192
View File
@@ -1,192 +0,0 @@
# Auto-Updater Documentation
Voicebox includes automatic updates powered by Tauri's updater plugin. This document explains how it works for both users and developers.
## 1. Generate Signing Keys
Run this command to generate your signing keypair:
```bash
cd tauri && bun tauri signer generate -w ~/.tauri/voicebox.key
```
This creates:
- **Private key**: `~/.tauri/voicebox.key` (keep this secret!)
- **Public key**: `~/.tauri/voicebox.key.pub`
## 2. Update Configuration
Copy the content from `~/.tauri/voicebox.key.pub` and replace the placeholder in `tauri/src-tauri/tauri.conf.json`:
```json
{
"plugins": {
"updater": {
"pubkey": "PASTE_PUBLIC_KEY_CONTENT_HERE",
"endpoints": [
"https://github.com/YOUR_USERNAME/voicebox/releases/latest/download/latest.json"
]
}
}
}
```
Update the endpoint URL with your actual GitHub username/organization.
## 3. Building with Signatures
When building releases, set these environment variables:
**macOS/Linux:**
```bash
export TAURI_SIGNING_PRIVATE_KEY="$(cat ~/.tauri/voicebox.key)"
export TAURI_SIGNING_PRIVATE_KEY_PASSWORD=""
bun run build
```
**Windows PowerShell:**
```powershell
$env:TAURI_SIGNING_PRIVATE_KEY = Get-Content ~/.tauri/voicebox.key -Raw
$env:TAURI_SIGNING_PRIVATE_KEY_PASSWORD = ""
bun run build
```
## 4. GitHub Release Setup
When you create a GitHub release, the build process will generate:
- Installers for each platform
- `.sig` signature files
- `latest.json` update manifest
### Manual Release Process
1. Build the app with signing keys set
2. Create a new GitHub release
3. Upload all files from `tauri/src-tauri/target/release/bundle/`
4. Create `latest.json` in your release assets:
```json
{
"version": "0.2.0",
"notes": "Bug fixes and improvements",
"pub_date": "2026-01-25T12:00:00Z",
"platforms": {
"darwin-aarch64": {
"signature": "CONTENT_FROM_.app.tar.gz.sig",
"url": "https://github.com/YOUR_USERNAME/voicebox/releases/download/v0.2.0/voicebox_0.2.0_aarch64.dmg"
},
"darwin-x86_64": {
"signature": "CONTENT_FROM_.app.tar.gz.sig",
"url": "https://github.com/YOUR_USERNAME/voicebox/releases/download/v0.2.0/voicebox_0.2.0_x64.dmg"
},
"linux-x86_64": {
"signature": "CONTENT_FROM_.AppImage.sig",
"url": "https://github.com/YOUR_USERNAME/voicebox/releases/download/v0.2.0/voicebox_0.2.0_amd64.AppImage"
},
"windows-x86_64": {
"signature": "CONTENT_FROM_.msi.sig",
"url": "https://github.com/YOUR_USERNAME/voicebox/releases/download/v0.2.0/voicebox_0.2.0_x64_en-US.msi"
}
}
}
```
### Automated GitHub Actions (Recommended)
Create `.github/workflows/release.yml`:
```yaml
name: Release
on:
push:
tags:
- 'v*'
jobs:
release:
strategy:
matrix:
platform: [macos-latest, ubuntu-22.04, windows-latest]
runs-on: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v1
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Install dependencies (Ubuntu)
if: matrix.platform == 'ubuntu-22.04'
run: |
sudo apt-get update
sudo apt-get install -y libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf
- name: Install dependencies
run: bun install
- name: Build
env:
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
run: bun run build
- name: Upload Release
uses: softprops/action-gh-release@v1
with:
files: tauri/src-tauri/target/release/bundle/**/*
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
```
Add your private key to GitHub secrets:
- Go to Settings → Secrets and variables → Actions
- Add `TAURI_SIGNING_PRIVATE_KEY` with the content of `~/.tauri/voicebox.key`
- Add `TAURI_SIGNING_PRIVATE_KEY_PASSWORD` (empty string if no password)
## Frontend Integration
The frontend integration is complete with automatic update notifications and manual update checks:
- **Update Notification Banner** - Appears automatically when updates are available
- **Settings Panel** - Manual "Check for Updates" button in Settings tab
- **Update Hook** - React hook handles all update operations
See `docs/AUTOUPDATER_QUICKSTART.md` for a quick setup guide.
## Security Notes
- Never commit your private key to version control
- Store private keys securely (use GitHub secrets for CI/CD)
- The public key in `tauri.conf.json` is safe to commit
- Updates are cryptographically verified before installation
- HTTP endpoints are blocked by default (HTTPS only)
## Testing Updates
1. Build version 0.1.0 and install it
2. Update version in `tauri.conf.json` to 0.2.0
3. Build version 0.2.0 with signatures
4. Create a local server or GitHub release with `latest.json`
5. Run version 0.1.0 and trigger update check
6. Verify update downloads and installs correctly
## Troubleshooting
**"Invalid signature" error:**
- Verify public key matches the private key used to sign
- Ensure signature files (.sig) are uploaded correctly
**"No update available" when one exists:**
- Check endpoint URL is correct
- Verify `latest.json` format matches specification
- Ensure version in latest.json is higher than current version
**Build fails with signing:**
- Confirm environment variables are set correctly
- Check private key file exists and is readable
- Verify private key format (should start with `dW50cnVzdGVkIGNvbW1lbnQ6`)
-116
View File
@@ -1,116 +0,0 @@
# Autoupdater Quick Start
The Tauri v2 autoupdater has been fully configured and integrated. Follow these steps to activate it.
## What's Already Done
✅ Rust plugin installed and initialized
✅ Tauri configuration set up with updater settings
✅ Permissions granted for update operations
✅ GitHub Actions workflow updated with signing support
✅ Frontend components created and integrated
✅ Update notifications on app startup
✅ Manual update check in Settings tab
## Required Steps (5 minutes)
### 1. Generate Signing Keys
```bash
bun run generate:keys
```
This creates:
- Private key: `~/.tauri/voicebox.key` (keep secret!)
- Public key: `~/.tauri/voicebox.key.pub` (safe to share)
### 2. Update Tauri Config
Open `tauri/src-tauri/tauri.conf.json` and:
1. Replace `"REPLACE_WITH_YOUR_PUBLIC_KEY"` with the content from `~/.tauri/voicebox.key.pub`
2. Update the endpoint URL with your GitHub username:
```json
"endpoints": [
"https://github.com/YOUR_USERNAME/voicebox/releases/latest/download/latest.json"
]
```
### 3. Add GitHub Secrets
Go to your repo Settings → Secrets and variables → Actions:
1. Add `TAURI_SIGNING_PRIVATE_KEY`:
```bash
cat ~/.tauri/voicebox.key
```
Copy the entire output and paste as the secret value
2. Add `TAURI_SIGNING_PRIVATE_KEY_PASSWORD`:
Leave empty (or add your password if you set one)
### 4. Test the Setup
To test locally before creating a release:
```bash
bun run build:release
```
This will verify your keys are set up correctly.
## How It Works
### For Users
1. App checks for updates on startup (only in Tauri builds)
2. If an update is available, a banner appears at the top
3. Users can click "Install Now" to download and install
4. App restarts automatically after installation
### For Developers
1. Create a new git tag: `git tag v0.2.0 && git push --tags`
2. GitHub Actions builds signed releases for all platforms
3. Uploads installers and generates `latest.json` manifest
4. Users running older versions will be notified automatically
## UI Components
### Update Notification Banner
- Shows at top of app when update is available
- Appears automatically on startup
- Displays download/install progress
### Settings Panel
- Located in Settings tab
- Shows current version
- Manual "Check for Updates" button
- Update status and progress
## Troubleshooting
**"Public key not configured"**
- Make sure you copied the entire content from `voicebox.key.pub`
- The key should start with `dW50cnVzdGVkIGNvbW1lbnQ6`
**"Failed to check for updates"**
- Endpoint URL might be incorrect
- No releases published yet (expected for first setup)
**Build fails with signing error**
- Check that GitHub secrets are set correctly
- Verify private key file exists at `~/.tauri/voicebox.key`
## Next Release Workflow
1. Update version in `tauri/src-tauri/tauri.conf.json`
2. Commit changes
3. Create and push tag: `git tag v0.2.0 && git push --tags`
4. GitHub Actions will automatically build and create a draft release
5. Review the release and publish it
6. Users will be notified of the update
## See Also
- Full documentation: `docs/AUTOUPDATER.md`
- Build script: `scripts/prepare-release.sh`
- GitHub workflow: `.github/workflows/release.yml`
+24 -47
View File
@@ -1,64 +1,41 @@
# Voicebox Documentation
# fumadocs-ui-template
This directory contains the documentation for Voicebox, built with [Mintlify](https://mintlify.com).
This is a Next.js application generated with
[Create Fumadocs](https://github.com/fuma-nama/fumadocs).
## Development
### Prerequisites
Install Mintlify globally using bun:
```bash
bun add -g mintlify
```
Or use the helper script:
```bash
bun run install:mintlify
```
### Running Locally
Run development server:
```bash
bun run dev
```
This will start the Mintlify dev server.
Open http://localhost:3000 with your browser to see the result.
The docs will be available at `http://localhost:3000`
## Explore
### Structure
In the project, you can see:
```
docs/
├── mint.json # Mintlify configuration
├── custom.css # Custom styles
├── overview/ # Getting started & feature docs
├── guides/ # User guides
├── api/ # API reference
├── development/ # Developer documentation
├── logo/ # Logo assets
└── public/ # Static assets
```
- `lib/source.ts`: Code for content source adapter, [`loader()`](https://fumadocs.dev/docs/headless/source-api) provides the interface to access your content.
- `lib/layout.shared.tsx`: Shared options for layouts, optional but preferred to keep.
### Writing Docs
| Route | Description |
| ------------------------- | ------------------------------------------------------ |
| `app/(home)` | The route group for your landing page and other pages. |
| `app/docs` | The documentation layout and pages. |
| `app/api/search/route.ts` | The Route Handler for search. |
- Use `.mdx` files for all documentation pages
- Follow the existing structure in `mint.json` for navigation
- Use Mintlify components for enhanced formatting (Card, CardGroup, Accordion, etc.)
- Reference the [Mintlify documentation](https://mintlify.com/docs) for available components
### Fumadocs MDX
## Deployment
A `source.config.ts` config file has been included, you can customise different options like frontmatter schema.
Docs are automatically deployed when changes are pushed to the main branch.
Read the [Introduction](https://fumadocs.dev/docs/mdx) for further details.
To manually deploy:
## Learn More
```bash
mintlify deploy
```
To learn more about Next.js and Fumadocs, take a look at the following
resources:
## Contributing
See [CONTRIBUTING.md](../CONTRIBUTING.md) for contribution guidelines.
- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js
features and API.
- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.
- [Fumadocs](https://fumadocs.dev) - learn about Fumadocs
-55
View File
@@ -1,55 +0,0 @@
---
title: "Authentication"
description: "API authentication and security"
---
## Current Status
<Warning>
Authentication is not currently implemented in Voicebox. The API is intended for local use only.
</Warning>
## Local Usage
For local development and usage:
- API runs on `localhost:17493`
- No authentication required
- Access restricted to local machine
## Future Implementation
Authentication will be added in a future release for:
- Remote deployments
- Multi-user access
- Production environments
Planned authentication methods:
- API keys
- OAuth 2.0
- JWT tokens
## Security Best Practices
Until authentication is implemented:
<CardGroup cols={2}>
<Card title="Use VPN" icon="shield">
Use WireGuard or Tailscale for remote access
</Card>
<Card title="Reverse Proxy" icon="server">
Run behind nginx with basic auth
</Card>
<Card title="Firewall" icon="fire">
Restrict access to trusted IPs only
</Card>
<Card title="Local Only" icon="laptop">
Don't expose to public internet
</Card>
</CardGroup>
## Coming Soon
- API key management
- User accounts
- Rate limiting
- Access control
-119
View File
@@ -1,119 +0,0 @@
---
title: "Generation API"
description: "Generate speech from text"
---
## Generate Speech
```http
POST /generate
```
**Request:**
```json
{
"text": "Hello world",
"profile_id": "abc123",
"language": "en"
}
```
**Response:**
```json
{
"id": "gen123",
"text": "Hello world",
"profile_id": "abc123",
"language": "en",
"audio_url": "/audio/gen123.wav",
"duration": 2.3,
"created_at": "2024-01-29T12:00:00Z"
}
```
## List History
```http
GET /history
```
**Query Parameters:**
- `profile_id` (optional) - Filter by voice profile
- `limit` (optional) - Number of results (default: 50)
- `offset` (optional) - Pagination offset
**Response:**
```json
{
"generations": [
{
"id": "gen123",
"text": "Hello world",
"profile_id": "abc123",
"duration": 2.3,
"created_at": "2024-01-29T12:00:00Z"
}
],
"total": 100
}
```
## Get Generation
```http
GET /history/{id}
```
**Response:**
```json
{
"id": "gen123",
"text": "Hello world",
"profile_id": "abc123",
"language": "en",
"audio_url": "/audio/gen123.wav",
"duration": 2.3,
"created_at": "2024-01-29T12:00:00Z"
}
```
## Delete Generation
```http
DELETE /history/{id}
```
**Response:**
```json
{
"success": true
}
```
## TypeScript Example
```typescript
import { VoiceboxClient } from '@/lib/api'
const client = new VoiceboxClient({
baseUrl: 'http://localhost:17493'
})
// Generate speech
const generation = await client.generate({
text: 'Hello world',
profile_id: 'abc123',
language: 'en'
})
// Get audio URL
const audioUrl = generation.audio_url
// List history
const history = await client.listHistory({
profile_id: 'abc123',
limit: 20
})
```
For full API documentation, visit `http://localhost:17493/docs` when the server is running.
-219
View File
@@ -1,219 +0,0 @@
---
title: "API Overview"
description: "Integrate voice synthesis into your applications with the Voicebox REST API"
---
## Introduction
Voicebox exposes a full REST API that allows you to integrate voice synthesis into your own applications. The API runs on `http://localhost:17493` by default.
<Card title="Interactive API Docs" icon="book" href="http://localhost:17493/docs">
When Voicebox is running, visit the auto-generated API documentation at `http://localhost:17493/docs`
</Card>
## Base URL
```
http://localhost:17493
```
For remote deployments, replace `localhost` with your server's IP or hostname.
## Authentication
<Note>
Currently, the API does not require authentication for local development. Authentication will be added in a future release for production deployments.
</Note>
## Quick Example
Here's a simple example of generating speech:
```bash
# Generate speech
curl -X POST http://localhost:17493/generate \
-H "Content-Type: application/json" \
-d '{
"text": "Hello world",
"profile_id": "abc123",
"language": "en"
}'
```
## API Endpoints
The Voicebox API is organized into several categories:
<CardGroup cols={2}>
<Card title="Voice Profiles" icon="user" href="/api/voice-profiles">
Create, list, update, and delete voice profiles
</Card>
<Card title="Generation" icon="waveform" href="/api/generation">
Generate speech from text using voice profiles
</Card>
<Card title="Recordings" icon="microphone" href="/api/recordings">
Record and transcribe audio
</Card>
<Card title="Stories" icon="film">
Create and manage multi-voice stories (coming soon)
</Card>
</CardGroup>
## Core Endpoints
### Voice Profiles
```http
GET /profiles # List all profiles
POST /profiles # Create a new profile
GET /profiles/{id} # Get profile details
PUT /profiles/{id} # Update a profile
DELETE /profiles/{id} # Delete a profile
POST /profiles/{id}/samples # Add voice sample
```
### Generation
```http
POST /generate # Generate speech
GET /history # List generation history
GET /history/{id} # Get generation details
DELETE /history/{id} # Delete from history
```
### Recordings
```http
POST /recordings # Start recording
POST /recordings/stop # Stop recording
POST /transcribe # Transcribe audio
```
## Response Format
All API responses follow a consistent JSON format:
```json
{
"success": true,
"data": {
// Response data
},
"error": null
}
```
Error responses:
```json
{
"success": false,
"data": null,
"error": {
"message": "Error description",
"code": "ERROR_CODE"
}
}
```
## Data Models
### Voice Profile
```json
{
"id": "abc123",
"name": "John Smith",
"language": "en",
"description": "Professional narrator voice",
"created_at": "2024-01-29T12:00:00Z",
"samples": [
{
"id": "sample123",
"audio_path": "/path/to/sample.wav",
"duration": 15.5
}
]
}
```
### Generation
```json
{
"id": "gen123",
"text": "Hello world",
"profile_id": "abc123",
"language": "en",
"audio_path": "/path/to/output.wav",
"duration": 2.3,
"created_at": "2024-01-29T12:00:00Z"
}
```
## TypeScript Client
Voicebox provides an auto-generated TypeScript client with full type safety:
```typescript
import { VoiceboxClient } from '@/lib/api'
const client = new VoiceboxClient({
baseUrl: 'http://localhost:17493'
})
// Create a profile
const profile = await client.createProfile({
name: 'John Smith',
language: 'en'
})
// Generate speech
const generation = await client.generate({
text: 'Hello world',
profile_id: profile.id,
language: 'en'
})
```
The client is automatically generated from the OpenAPI schema. See [Development Setup](/development/setup#generate-openapi-client) for details.
## Rate Limiting
<Info>
Currently, there are no rate limits for local usage. Rate limiting will be added in a future release for production deployments.
</Info>
## WebSocket Support
<Note>
Real-time streaming generation via WebSockets is planned for a future release.
</Note>
## Use Cases
<CardGroup cols={2}>
<Card title="Game Development" icon="gamepad">
Generate dynamic dialogue for NPCs and characters
</Card>
<Card title="Content Creation" icon="video">
Automate voiceovers for videos and podcasts
</Card>
<Card title="Accessibility" icon="universal-access">
Build text-to-speech tools for visually impaired users
</Card>
<Card title="Voice Assistants" icon="robot">
Create custom voice interfaces
</Card>
</CardGroup>
## Next Steps
<CardGroup cols={2}>
<Card title="Voice Profiles API" icon="user" href="/api/voice-profiles">
Learn how to manage voice profiles
</Card>
<Card title="Generation API" icon="waveform" href="/api/generation">
Generate speech from text
</Card>
</CardGroup>
-95
View File
@@ -1,95 +0,0 @@
---
title: "Recordings API"
description: "Record and transcribe audio"
---
## Start Recording
```http
POST /recordings/start
```
**Request:**
```json
{
"source": "microphone"
}
```
**Response:**
```json
{
"recording_id": "rec123",
"status": "recording"
}
```
## Stop Recording
```http
POST /recordings/stop
```
**Request:**
```json
{
"recording_id": "rec123"
}
```
**Response:**
```json
{
"recording_id": "rec123",
"audio_url": "/audio/rec123.wav",
"duration": 15.5
}
```
## Transcribe Audio
```http
POST /transcribe
```
**Request:** (multipart/form-data)
```
audio: <file>
language: "en" (optional)
```
**Response:**
```json
{
"text": "Transcribed speech text here",
"language": "en",
"duration": 15.5,
"confidence": 0.95
}
```
## TypeScript Example
```typescript
import { VoiceboxClient } from '@/lib/api'
const client = new VoiceboxClient({
baseUrl: 'http://localhost:17493'
})
// Start recording
const recording = await client.startRecording({
source: 'microphone'
})
// ... record audio ...
// Stop recording
const result = await client.stopRecording(recording.id)
// Transcribe
const transcription = await client.transcribe(audioFile, 'en')
console.log(transcription.text)
```
For full API documentation, visit `http://localhost:17493/docs` when the server is running.
-149
View File
@@ -1,149 +0,0 @@
---
title: "Voice Profiles API"
description: "Manage voice profiles programmatically"
---
## Endpoints
### List Profiles
```http
GET /profiles
```
**Response:**
```json
{
"profiles": [
{
"id": "abc123",
"name": "John Smith",
"language": "en",
"description": "Professional narrator",
"created_at": "2024-01-29T12:00:00Z",
"sample_count": 2
}
]
}
```
### Get Profile
```http
GET /profiles/{id}
```
**Response:**
```json
{
"id": "abc123",
"name": "John Smith",
"language": "en",
"description": "Professional narrator",
"created_at": "2024-01-29T12:00:00Z",
"samples": [
{
"id": "sample123",
"duration": 15.5,
"created_at": "2024-01-29T12:00:00Z"
}
]
}
```
### Create Profile
```http
POST /profiles
```
**Request:**
```json
{
"name": "John Smith",
"language": "en",
"description": "Professional narrator"
}
```
**Response:**
```json
{
"id": "abc123",
"name": "John Smith",
"language": "en",
"description": "Professional narrator",
"created_at": "2024-01-29T12:00:00Z"
}
```
### Update Profile
```http
PUT /profiles/{id}
```
**Request:**
```json
{
"name": "Updated Name",
"description": "Updated description"
}
```
### Delete Profile
```http
DELETE /profiles/{id}
```
**Response:**
```json
{
"success": true
}
```
### Add Voice Sample
```http
POST /profiles/{id}/samples
```
**Request:** (multipart/form-data)
```
audio: <file>
```
**Response:**
```json
{
"sample_id": "sample123",
"duration": 15.5
}
```
## TypeScript Example
```typescript
import { VoiceboxClient } from '@/lib/api'
const client = new VoiceboxClient({
baseUrl: 'http://localhost:17493'
})
// Create profile
const profile = await client.createProfile({
name: 'John Smith',
language: 'en',
description: 'Professional narrator'
})
// Add sample
await client.addSample(profile.id, audioFile)
// List all profiles
const profiles = await client.listProfiles()
```
For full API documentation, visit `http://localhost:17493/docs` when the server is running.
+11
View File
@@ -0,0 +1,11 @@
import { DocsLayout } from 'fumadocs-ui/layouts/docs';
import { baseOptions } from '@/lib/layout.shared';
import { source } from '@/lib/source';
export default function Layout({ children }: LayoutProps<'/[[...slug]]'>) {
return (
<DocsLayout tree={source.pageTree} {...baseOptions()}>
{children}
</DocsLayout>
);
}
+74
View File
@@ -0,0 +1,74 @@
import { createRelativeLink } from 'fumadocs-ui/mdx';
import { DocsBody, DocsDescription, DocsPage, DocsTitle } from 'fumadocs-ui/page';
import type { Metadata } from 'next';
import { notFound } from 'next/navigation';
import { MarkdownCopyButton, ViewOptionsPopover } from '@/components/ai/page-actions';
import { APIPage } from '@/components/api-page';
import { getPageImage, source } from '@/lib/source';
import { getMDXComponents } from '@/mdx-components';
export default async function Page(props: PageProps<'/[[...slug]]'>) {
const params = await props.params;
const page = source.getPage(params.slug);
if (!page) notFound();
const MDX = page.data.body;
const markdownUrl = `${page.url}.mdx`;
const githubUrl = `https://github.com/jamiepine/voicebox/blob/main/docs/content/docs/${page.path}`;
return (
<DocsPage
toc={page.data.toc}
full={page.data.full}
editOnGithub={{
owner: 'jamiepine',
repo: 'voicebox',
sha: 'main',
path: `docs/content/docs/${page.path}`,
}}
lastUpdate={page.data.lastModified}
>
<DocsTitle>{page.data.title}</DocsTitle>
<DocsDescription className="mb-0">{page.data.description}</DocsDescription>
<div className="flex flex-row gap-2 items-center">
<MarkdownCopyButton markdownUrl={markdownUrl} />
<ViewOptionsPopover markdownUrl={markdownUrl} githubUrl={githubUrl} />
</div>
<div
role="separator"
style={{
height: '1px',
background: 'currentColor',
opacity: 0.15,
marginTop: '8px',
marginBottom: '24px',
}}
/>
<DocsBody>
<MDX
components={getMDXComponents({
a: createRelativeLink(source, page),
})}
/>
</DocsBody>
</DocsPage>
);
}
export async function generateStaticParams() {
return source.generateParams();
}
export async function generateMetadata(props: PageProps<'/[[...slug]]'>): Promise<Metadata> {
const params = await props.params;
const page = source.getPage(params.slug);
if (!page) notFound();
return {
title: page.data.title,
description: page.data.description,
openGraph: {
images: getPageImage(page).url,
},
};
}
+7
View File
@@ -0,0 +1,7 @@
import { source } from '@/lib/source';
import { createFromSource } from 'fumadocs-core/search/server';
export const { GET } = createFromSource(source, {
// https://docs.orama.com/docs/orama-js/supported-languages
language: 'english',
});
+14
View File
@@ -0,0 +1,14 @@
@import 'tailwindcss';
@import 'fumadocs-ui/css/neutral.css';
@import 'fumadocs-ui/css/preset.css';
@import 'fumadocs-openapi/css/preset.css';
:root {
--color-fd-primary: hsl(43, 50%, 50%);
--color-fd-primary-foreground: hsl(222.2, 47.4%, 11.2%);
}
.dark {
--color-fd-primary: hsl(43, 50%, 45%);
--color-fd-primary-foreground: hsl(0, 0%, 95%);
}
+17
View File
@@ -0,0 +1,17 @@
import { RootProvider } from 'fumadocs-ui/provider/next';
import './global.css';
import { Inter } from 'next/font/google';
const inter = Inter({
subsets: ['latin'],
});
export default function Layout({ children }: LayoutProps<'/'>) {
return (
<html lang="en" className={inter.className} suppressHydrationWarning>
<body className="flex flex-col min-h-screen">
<RootProvider>{children}</RootProvider>
</body>
</html>
);
}
+10
View File
@@ -0,0 +1,10 @@
import { getLLMText, source } from '@/lib/source';
export const revalidate = false;
export async function GET() {
const scan = source.getPages().map(getLLMText);
const scanned = await Promise.all(scan);
return new Response(scanned.join('\n\n'));
}
@@ -0,0 +1,20 @@
import { notFound } from 'next/navigation';
import { getLLMText, source } from '@/lib/source';
export const revalidate = false;
export async function GET(_req: Request, { params }: RouteContext<'/llms.mdx/docs/[[...slug]]'>) {
const { slug } = await params;
const page = source.getPage(slug);
if (!page) notFound();
return new Response(await getLLMText(page), {
headers: {
'Content-Type': 'text/markdown',
},
});
}
export function generateStaticParams() {
return source.generateParams();
}
+36
View File
@@ -0,0 +1,36 @@
import { getPageImage, source } from '@/lib/source';
import { notFound } from 'next/navigation';
import { ImageResponse } from 'next/og';
import { generate as DefaultImage } from 'fumadocs-ui/og';
export const revalidate = false;
export async function GET(
_req: Request,
{ params }: RouteContext<'/og/docs/[...slug]'>,
) {
const { slug } = await params;
const page = source.getPage(slug.slice(0, -1));
if (!page) notFound();
return new ImageResponse(
(
<DefaultImage
title={page.data.title}
description={page.data.description}
site="My App"
/>
),
{
width: 1200,
height: 630,
},
);
}
export function generateStaticParams() {
return source.getPages().map((page) => ({
lang: page.locale,
slug: getPageImage(page).segments,
}));
}
+277 -1278
View File
File diff suppressed because it is too large Load Diff
+13
View File
@@ -0,0 +1,13 @@
{
"$schema": "node_modules/@fumadocs/cli/dist/schema/default.json",
"aliases": {
"uiDir": "./components/ui",
"componentsDir": "./components",
"blockDir": "./components",
"cssDir": "./styles",
"libDir": "./lib"
},
"baseDir": "",
"uiLibrary": "radix-ui",
"commands": {}
}
+247
View File
@@ -0,0 +1,247 @@
'use client';
import { type ComponentProps, useMemo, useState } from 'react';
import { Check, ChevronDown, Copy, ExternalLinkIcon, TextIcon } from 'lucide-react';
import { cn } from '../../lib/cn';
import { useCopyButton } from 'fumadocs-ui/utils/use-copy-button';
import { Popover, PopoverTrigger, PopoverContent } from '../ui/popover';
import { buttonVariants } from '../ui/button';
const cache = new Map<string, Promise<string>>();
export function MarkdownCopyButton({
markdownUrl,
...props
}: ComponentProps<'button'> & {
/**
* A URL to fetch the raw Markdown/MDX content of page
*/
markdownUrl: string;
}) {
const [isLoading, setLoading] = useState(false);
const [checked, onClick] = useCopyButton(async () => {
const cached = cache.get(markdownUrl);
if (cached) return navigator.clipboard.writeText(await cached);
setLoading(true);
try {
const promise = fetch(markdownUrl).then((res) => res.text());
cache.set(markdownUrl, promise);
await navigator.clipboard.write([
new ClipboardItem({
'text/plain': promise,
}),
]);
} finally {
setLoading(false);
}
});
return (
<button
disabled={isLoading}
onClick={onClick}
{...props}
className={cn(
buttonVariants({
variant: 'secondary',
size: 'sm',
className: 'gap-2 [&_svg]:size-3.5 [&_svg]:text-fd-muted-foreground',
}),
props.className,
)}
>
{checked ? <Check /> : <Copy />}
Copy Markdown
</button>
);
}
export function ViewOptionsPopover({
markdownUrl,
githubUrl,
...props
}: ComponentProps<typeof PopoverTrigger> & {
/**
* A URL to the raw Markdown/MDX content of page
*/
markdownUrl: string;
/**
* Source file URL on GitHub
*/
githubUrl: string;
}) {
const items = useMemo(() => {
const pageUrl = typeof window !== 'undefined' ? window.location.href : 'loading';
const q = `Read ${pageUrl}, I want to ask questions about it.`;
return [
{
title: 'Open in GitHub',
href: githubUrl,
icon: (
<svg fill="currentColor" role="img" viewBox="0 0 24 24">
<title>GitHub</title>
<path d="M12 .297c-6.63 0-12 5.373-12 12 0 5.303 3.438 9.8 8.205 11.385.6.113.82-.258.82-.577 0-.285-.01-1.04-.015-2.04-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729 1.205.084 1.838 1.236 1.838 1.236 1.07 1.835 2.809 1.305 3.495.998.108-.776.417-1.305.76-1.605-2.665-.3-5.466-1.332-5.466-5.93 0-1.31.465-2.38 1.235-3.22-.135-.303-.54-1.523.105-3.176 0 0 1.005-.322 3.3 1.23.96-.267 1.98-.399 3-.405 1.02.006 2.04.138 3 .405 2.28-1.552 3.285-1.23 3.285-1.23.645 1.653.24 2.873.12 3.176.765.84 1.23 1.91 1.23 3.22 0 4.61-2.805 5.625-5.475 5.92.42.36.81 1.096.81 2.22 0 1.606-.015 2.896-.015 3.286 0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12" />
</svg>
),
},
{
title: 'View as Markdown',
href: markdownUrl,
icon: <TextIcon />,
},
{
title: 'Open in Scira AI',
href: `https://scira.ai/?${new URLSearchParams({
q,
})}`,
icon: (
<svg
width="910"
height="934"
viewBox="0 0 910 934"
fill="none"
xmlns="http://www.w3.org/2000/svg"
>
<title>Scira AI</title>
<path
d="M647.664 197.775C569.13 189.049 525.5 145.419 516.774 66.8849C508.048 145.419 464.418 189.049 385.884 197.775C464.418 206.501 508.048 250.131 516.774 328.665C525.5 250.131 569.13 206.501 647.664 197.775Z"
fill="currentColor"
stroke="currentColor"
strokeWidth="8"
strokeLinejoin="round"
/>
<path
d="M516.774 304.217C510.299 275.491 498.208 252.087 480.335 234.214C462.462 216.341 439.058 204.251 410.333 197.775C439.059 191.3 462.462 179.209 480.335 161.336C498.208 143.463 510.299 120.06 516.774 91.334C523.25 120.059 535.34 143.463 553.213 161.336C571.086 179.209 594.49 191.3 623.216 197.775C594.49 204.251 571.086 216.341 553.213 234.214C535.34 252.087 523.25 275.491 516.774 304.217Z"
fill="currentColor"
stroke="currentColor"
strokeWidth="8"
strokeLinejoin="round"
/>
<path
d="M857.5 508.116C763.259 497.644 710.903 445.288 700.432 351.047C689.961 445.288 637.605 497.644 543.364 508.116C637.605 518.587 689.961 570.943 700.432 665.184C710.903 570.943 763.259 518.587 857.5 508.116Z"
stroke="currentColor"
strokeWidth="20"
strokeLinejoin="round"
/>
<path
d="M700.432 615.957C691.848 589.05 678.575 566.357 660.383 548.165C642.191 529.973 619.499 516.7 592.593 508.116C619.499 499.533 642.191 486.258 660.383 468.066C678.575 449.874 691.848 427.181 700.432 400.274C709.015 427.181 722.289 449.874 740.481 468.066C758.673 486.258 781.365 499.533 808.271 508.116C781.365 516.7 758.673 529.973 740.481 548.165C722.289 566.357 709.015 589.05 700.432 615.957Z"
stroke="currentColor"
strokeWidth="20"
strokeLinejoin="round"
/>
<path
d="M889.949 121.237C831.049 114.692 798.326 81.9698 791.782 23.0692C785.237 81.9698 752.515 114.692 693.614 121.237C752.515 127.781 785.237 160.504 791.782 219.404C798.326 160.504 831.049 127.781 889.949 121.237Z"
fill="currentColor"
stroke="currentColor"
strokeWidth="8"
strokeLinejoin="round"
/>
<path
d="M791.782 196.795C786.697 176.937 777.869 160.567 765.16 147.858C752.452 135.15 736.082 126.322 716.226 121.237C736.082 116.152 752.452 107.324 765.16 94.6152C777.869 81.9065 786.697 65.5368 791.782 45.6797C796.867 65.5367 805.695 81.9066 818.403 94.6152C831.112 107.324 847.481 116.152 867.338 121.237C847.481 126.322 831.112 135.15 818.403 147.858C805.694 160.567 796.867 176.937 791.782 196.795Z"
fill="currentColor"
stroke="currentColor"
strokeWidth="8"
strokeLinejoin="round"
/>
<path
d="M760.632 764.337C720.719 814.616 669.835 855.1 611.872 882.692C553.91 910.285 490.404 924.255 426.213 923.533C362.022 922.812 298.846 907.419 241.518 878.531C184.19 849.643 134.228 808.026 95.4548 756.863C56.6815 705.7 30.1238 646.346 17.8129 583.343C5.50207 520.339 7.76433 455.354 24.4266 393.359C41.089 331.364 71.7099 274.001 113.947 225.658C156.184 177.315 208.919 139.273 268.117 114.442"
stroke="currentColor"
strokeWidth="30"
strokeLinecap="round"
strokeLinejoin="round"
/>
</svg>
),
},
{
title: 'Open in ChatGPT',
href: `https://chatgpt.com/?${new URLSearchParams({
hints: 'search',
q,
})}`,
icon: (
<svg
role="img"
viewBox="0 0 24 24"
fill="currentColor"
xmlns="http://www.w3.org/2000/svg"
>
<title>OpenAI</title>
<path d="M22.2819 9.8211a5.9847 5.9847 0 0 0-.5157-4.9108 6.0462 6.0462 0 0 0-6.5098-2.9A6.0651 6.0651 0 0 0 4.9807 4.1818a5.9847 5.9847 0 0 0-3.9977 2.9 6.0462 6.0462 0 0 0 .7427 7.0966 5.98 5.98 0 0 0 .511 4.9107 6.051 6.051 0 0 0 6.5146 2.9001A5.9847 5.9847 0 0 0 13.2599 24a6.0557 6.0557 0 0 0 5.7718-4.2058 5.9894 5.9894 0 0 0 3.9977-2.9001 6.0557 6.0557 0 0 0-.7475-7.0729zm-9.022 12.6081a4.4755 4.4755 0 0 1-2.8764-1.0408l.1419-.0804 4.7783-2.7582a.7948.7948 0 0 0 .3927-.6813v-6.7369l2.02 1.1686a.071.071 0 0 1 .038.052v5.5826a4.504 4.504 0 0 1-4.4945 4.4944zm-9.6607-4.1254a4.4708 4.4708 0 0 1-.5346-3.0137l.142.0852 4.783 2.7582a.7712.7712 0 0 0 .7806 0l5.8428-3.3685v2.3324a.0804.0804 0 0 1-.0332.0615L9.74 19.9502a4.4992 4.4992 0 0 1-6.1408-1.6464zM2.3408 7.8956a4.485 4.485 0 0 1 2.3655-1.9728V11.6a.7664.7664 0 0 0 .3879.6765l5.8144 3.3543-2.0201 1.1685a.0757.0757 0 0 1-.071 0l-4.8303-2.7865A4.504 4.504 0 0 1 2.3408 7.872zm16.5963 3.8558L13.1038 8.364 15.1192 7.2a.0757.0757 0 0 1 .071 0l4.8303 2.7913a4.4944 4.4944 0 0 1-.6765 8.1042v-5.6772a.79.79 0 0 0-.407-.667zm2.0107-3.0231l-.142-.0852-4.7735-2.7818a.7759.7759 0 0 0-.7854 0L9.409 9.2297V6.8974a.0662.0662 0 0 1 .0284-.0615l4.8303-2.7866a4.4992 4.4992 0 0 1 6.6802 4.66zM8.3065 12.863l-2.02-1.1638a.0804.0804 0 0 1-.038-.0567V6.0742a4.4992 4.4992 0 0 1 7.3757-3.4537l-.142.0805L8.704 5.459a.7948.7948 0 0 0-.3927.6813zm1.0976-2.3654l2.602-1.4998 2.6069 1.4998v2.9994l-2.5974 1.4997-2.6067-1.4997Z" />
</svg>
),
},
{
title: 'Open in Claude',
href: `https://claude.ai/new?${new URLSearchParams({
q,
})}`,
icon: (
<svg
fill="currentColor"
role="img"
viewBox="0 0 24 24"
xmlns="http://www.w3.org/2000/svg"
>
<title>Anthropic</title>
<path d="M17.3041 3.541h-3.6718l6.696 16.918H24Zm-10.6082 0L0 20.459h3.7442l1.3693-3.5527h7.0052l1.3693 3.5528h3.7442L10.5363 3.5409Zm-.3712 10.2232 2.2914-5.9456 2.2914 5.9456Z" />
</svg>
),
},
{
title: 'Open in Cursor',
icon: (
<svg
fill="currentColor"
role="img"
viewBox="0 0 24 24"
xmlns="http://www.w3.org/2000/svg"
>
<title>Cursor</title>
<path d="M11.503.131 1.891 5.678a.84.84 0 0 0-.42.726v11.188c0 .3.162.575.42.724l9.609 5.55a1 1 0 0 0 .998 0l9.61-5.55a.84.84 0 0 0 .42-.724V6.404a.84.84 0 0 0-.42-.726L12.497.131a1.01 1.01 0 0 0-.996 0M2.657 6.338h18.55c.263 0 .43.287.297.515L12.23 22.918c-.062.107-.229.064-.229-.06V12.335a.59.59 0 0 0-.295-.51l-9.11-5.257c-.109-.063-.064-.23.061-.23" />
</svg>
),
href: `https://cursor.com/link/prompt?${new URLSearchParams({
text: q,
})}`,
},
];
}, [githubUrl, markdownUrl]);
return (
<Popover>
<PopoverTrigger
{...props}
className={cn(
buttonVariants({
variant: 'secondary',
size: 'sm',
}),
'gap-2 data-[state=open]:bg-fd-accent data-[state=open]:text-fd-accent-foreground',
props.className,
)}
>
Open
<ChevronDown className="size-3.5 text-fd-muted-foreground" />
</PopoverTrigger>
<PopoverContent className="flex flex-col">
{items.map((item) => (
<a
key={item.href}
href={item.href}
rel="noreferrer noopener"
target="_blank"
className="text-sm p-2 rounded-lg inline-flex items-center gap-2 hover:text-fd-accent-foreground hover:bg-fd-accent [&_svg]:size-4"
>
{item.icon}
{item.title}
<ExternalLinkIcon className="text-fd-muted-foreground size-3.5 ms-auto" />
</a>
))}
</PopoverContent>
</Popover>
);
}
+6
View File
@@ -0,0 +1,6 @@
'use client';
import { defineClientConfig } from 'fumadocs-openapi/ui/client';
export default defineClientConfig({
// Client-side configuration for API playground
});
+7
View File
@@ -0,0 +1,7 @@
import { openapi } from '@/lib/openapi';
import { createAPIPage } from 'fumadocs-openapi/ui';
import client from './api-page.client';
export const APIPage = createAPIPage(openapi, {
client,
});
+29
View File
@@ -0,0 +1,29 @@
import { cva, type VariantProps } from 'class-variance-authority';
const variants = {
primary:
'bg-fd-primary text-fd-primary-foreground hover:bg-fd-primary/80 disabled:bg-fd-secondary disabled:text-fd-secondary-foreground',
outline: 'border hover:bg-fd-accent hover:text-fd-accent-foreground',
ghost: 'hover:bg-fd-accent hover:text-fd-accent-foreground',
secondary:
'border bg-fd-secondary text-fd-secondary-foreground hover:bg-fd-accent hover:text-fd-accent-foreground',
} as const;
export const buttonVariants = cva(
'inline-flex items-center justify-center rounded-md p-2 text-sm font-medium transition-colors duration-100 disabled:pointer-events-none disabled:opacity-50 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-fd-ring',
{
variants: {
variant: variants,
// fumadocs use `color` instead of `variant`
color: variants,
size: {
sm: 'gap-1 px-2 py-1.5 text-xs',
icon: 'p-1.5 [&_svg]:size-5',
'icon-sm': 'p-1.5 [&_svg]:size-4.5',
'icon-xs': 'p-1 [&_svg]:size-4',
},
},
},
);
export type ButtonProps = VariantProps<typeof buttonVariants>;
+32
View File
@@ -0,0 +1,32 @@
'use client';
import * as PopoverPrimitive from '@radix-ui/react-popover';
import * as React from 'react';
import { cn } from '../../lib/cn';
const Popover = PopoverPrimitive.Root;
const PopoverTrigger = PopoverPrimitive.Trigger;
const PopoverContent = React.forwardRef<
React.ComponentRef<typeof PopoverPrimitive.Content>,
React.ComponentPropsWithoutRef<typeof PopoverPrimitive.Content>
>(({ className, align = 'center', sideOffset = 4, ...props }, ref) => (
<PopoverPrimitive.Portal>
<PopoverPrimitive.Content
ref={ref}
align={align}
sideOffset={sideOffset}
side="bottom"
className={cn(
'z-50 origin-(--radix-popover-content-transform-origin) overflow-y-auto max-h-(--radix-popover-content-available-height) min-w-[240px] max-w-[98vw] rounded-xl border bg-fd-popover/60 backdrop-blur-lg p-2 text-sm text-fd-popover-foreground shadow-lg focus-visible:outline-none data-[state=closed]:animate-fd-popover-out data-[state=open]:animate-fd-popover-in',
className,
)}
{...props}
/>
</PopoverPrimitive.Portal>
));
PopoverContent.displayName = PopoverPrimitive.Content.displayName;
const PopoverClose = PopoverPrimitive.PopoverClose;
export { Popover, PopoverTrigger, PopoverContent, PopoverClose };
+67
View File
@@ -0,0 +1,67 @@
---
title: "Documentation README"
description: "Voicebox documentation development guide"
---
This directory contains the documentation for Voicebox, built with [Fumadocs](https://fumadocs.dev).
## Development
### Prerequisites
Install Mintlify globally using bun:
```bash
bun add -g mintlify
```
Or use the helper script:
```bash
bun run install:mintlify
```
### Running Locally
```bash
bun run dev
```
This will start the Mintlify dev server.
The docs will be available at `http://localhost:3000`
### Structure
```
docs/
├── mint.json # Mintlify configuration
├── custom.css # Custom styles
├── overview/ # Getting started & feature docs
├── guides/ # User guides
├── api/ # API reference
├── development/ # Developer documentation
├── logo/ # Logo assets
└── public/ # Static assets
```
### Writing Docs
- Use `.mdx` files for all documentation pages
- Follow the existing structure in `mint.json` for navigation
- Use Mintlify components for enhanced formatting (Card, CardGroup, Accordion, etc.)
- Reference the [Mintlify documentation](https://mintlify.com/docs) for available components
## Deployment
Docs are automatically deployed when changes are pushed to the main branch.
To manually deploy:
```bash
mintlify deploy
```
## Contributing
See [CONTRIBUTING.md](../CONTRIBUTING.md) for contribution guidelines.
@@ -1,4 +1,7 @@
# Troubleshooting Guide
---
title: "Troubleshooting Guide"
description: "Common issues and solutions for Voicebox"
---
Common issues and solutions for Voicebox.
@@ -0,0 +1,16 @@
---
title: Health
description: Health check endpoint.
full: true
_openapi:
method: GET
toc: []
structuredData:
headings: []
contents:
- content: Health check endpoint.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/health","method":"get"}]} />
@@ -0,0 +1,4 @@
{
"title": "General",
"pages": ["root__get", "health_health_get"]
}
@@ -0,0 +1,16 @@
---
title: Root
description: Root endpoint.
full: true
_openapi:
method: GET
toc: []
structuredData:
headings: []
contents:
- content: Root endpoint.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/","method":"get"}]} />
@@ -0,0 +1,16 @@
---
title: Generate Speech
description: Generate speech from text using a voice profile.
full: true
_openapi:
method: POST
toc: []
structuredData:
headings: []
contents:
- content: Generate speech from text using a voice profile.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/generate","method":"post"}]} />
@@ -0,0 +1,16 @@
---
title: Get Audio
description: Serve generated audio file.
full: true
_openapi:
method: GET
toc: []
structuredData:
headings: []
contents:
- content: Serve generated audio file.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/audio/{generation_id}","method":"get"}]} />
@@ -0,0 +1,8 @@
{
"title": "Generation",
"pages": [
"generate_speech_generate_post",
"transcribe_audio_transcribe_post",
"get_audio_audio__generation_id__get"
]
}
@@ -0,0 +1,16 @@
---
title: Transcribe Audio
description: Transcribe audio file to text.
full: true
_openapi:
method: POST
toc: []
structuredData:
headings: []
contents:
- content: Transcribe audio file to text.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/transcribe","method":"post"}]} />

Some files were not shown because too many files have changed in this diff Show More