mirror of
https://github.com/jamiepine/voicebox.git
synced 2026-09-28 06:35:18 -07:00
Compare commits
50
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2e95b7c5d8 | ||
|
|
ffc1b54812 | ||
|
|
fc5ed1ff40 | ||
|
|
c9f38dd496 | ||
|
|
58b19e4e9f | ||
|
|
0245c31dba | ||
|
|
81864e831a | ||
|
|
7bd72ea9f7 | ||
|
|
f96eae2567 | ||
|
|
7d53699c96 | ||
|
|
28e91ce2c1 | ||
|
|
88be097b62 | ||
|
|
564d787927 | ||
|
|
2c1ee94891 | ||
|
|
e789c937ad | ||
|
|
273483ffcf | ||
|
|
5774a168a9 | ||
|
|
6bf40bd2d0 | ||
|
|
12cda2e090 | ||
|
|
7a90290a76 | ||
|
|
b02ce8e2f3 | ||
|
|
4e7772a21d | ||
|
|
51fb320b8c | ||
|
|
8ac202aa58 | ||
|
|
ac68052945 | ||
|
|
e601fd2ca4 | ||
|
|
7b25e0ba0b | ||
|
|
a6817cd082 | ||
|
|
df50b8a925 | ||
|
|
a672ac5279 | ||
|
|
d35e6f0cc5 | ||
|
|
b1069b4521 | ||
|
|
f9e1aa153d | ||
|
|
01800f196f | ||
|
|
606da1c894 | ||
|
|
664178f0cf | ||
|
|
f1541701fb | ||
|
|
a2adc3b506 | ||
|
|
15ba824472 | ||
|
|
2ad4776a76 | ||
|
|
a8469b39f1 | ||
|
|
7dd70a52e4 | ||
|
|
1526f2de26 | ||
|
|
2c63dfff25 | ||
|
|
e0a798dc0d | ||
|
|
5933cba8e9 | ||
|
|
1b2d492398 | ||
|
|
faa825290f | ||
|
|
4a8a9eac14 | ||
|
|
3e4d9ff641 |
@@ -0,0 +1,120 @@
|
|||||||
|
---
|
||||||
|
name: add-tts-engine
|
||||||
|
description: Use this skill to add a new TTS engine to Voicebox. It walks through dependency research, backend implementation, frontend wiring, PyInstaller bundling, and frozen-build testing. Always start with Phase 0 (dependency audit) before writing any code.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Add TTS Engine
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Integrate a new text-to-speech engine into Voicebox end-to-end: dependency research, backend protocol implementation, frontend UI wiring, PyInstaller bundling, and frozen-build verification. The user should only need to test the final build locally.
|
||||||
|
|
||||||
|
## Reference Doc
|
||||||
|
|
||||||
|
The full phased guide lives at `docs/content/docs/developer/tts-engines.mdx`. **Read this file in its entirety before starting.** It contains:
|
||||||
|
|
||||||
|
- Phase 0: Dependency research (mandatory before writing code)
|
||||||
|
- Phase 1: Backend implementation (`TTSBackend` protocol)
|
||||||
|
- Phase 2: Route and service integration (usually zero changes)
|
||||||
|
- Phase 3: Frontend integration (5 files)
|
||||||
|
- Phase 4: Dependencies (`requirements.txt`, justfile, CI, Docker)
|
||||||
|
- Phase 5: PyInstaller bundling (`build_binary.py` + `server.py`)
|
||||||
|
- Phase 6: Common upstream workarounds
|
||||||
|
- Implementation checklist (gate between phases)
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
### 1. Read the guide
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Read the full TTS engines doc
|
||||||
|
cat docs/content/docs/developer/tts-engines.mdx
|
||||||
|
```
|
||||||
|
|
||||||
|
Internalize all phases, especially Phase 0 and Phase 5. The v0.2.3 release was three patch releases because Phase 0 was skipped.
|
||||||
|
|
||||||
|
### 2. Dependency research (Phase 0)
|
||||||
|
|
||||||
|
Clone the model library into a temporary directory and audit it. Do NOT skip this.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mkdir /tmp/engine-research && cd /tmp/engine-research
|
||||||
|
git clone <model-library-url>
|
||||||
|
```
|
||||||
|
|
||||||
|
Run the grep searches from Phase 0.2 in the guide against the cloned source and its transitive dependencies. Produce a written dependency audit covering:
|
||||||
|
|
||||||
|
1. PyPI vs non-PyPI packages
|
||||||
|
2. PyInstaller directives needed (`--collect-all`, `--copy-metadata`, `--hidden-import`)
|
||||||
|
3. Runtime data files that must be bundled
|
||||||
|
4. Native library paths that need env var overrides in frozen builds
|
||||||
|
5. Monkey-patches needed (`torch.load`, float64, MPS, HF token)
|
||||||
|
6. Sample rate
|
||||||
|
7. Model download method (`from_pretrained` vs `snapshot_download` + `from_local`)
|
||||||
|
|
||||||
|
Test model loading and generation on CPU in the throwaway venv before proceeding.
|
||||||
|
|
||||||
|
### 3. Implement (Phases 1–4)
|
||||||
|
|
||||||
|
Follow the guide's phases in order. Key files to modify:
|
||||||
|
|
||||||
|
**Backend (Phase 1):**
|
||||||
|
- Create `backend/backends/<engine>_backend.py`
|
||||||
|
- Register in `backend/backends/__init__.py` (ModelConfig + TTS_ENGINES + factory)
|
||||||
|
- Update regex in `backend/models.py`
|
||||||
|
|
||||||
|
**Frontend (Phase 3):**
|
||||||
|
- `app/src/lib/api/types.ts` — engine union type
|
||||||
|
- `app/src/lib/constants/languages.ts` — ENGINE_LANGUAGES
|
||||||
|
- `app/src/components/Generation/EngineModelSelector.tsx` — ENGINE_OPTIONS, ENGINE_DESCRIPTIONS
|
||||||
|
- `app/src/lib/hooks/useGenerationForm.ts` — Zod schema, model-name mapping
|
||||||
|
- `app/src/components/ServerSettings/ModelManagement.tsx` — MODEL_DESCRIPTIONS
|
||||||
|
|
||||||
|
**Dependencies (Phase 4):**
|
||||||
|
- `backend/requirements.txt`
|
||||||
|
- `justfile` (setup-python, setup-python-release targets)
|
||||||
|
- `.github/workflows/release.yml`
|
||||||
|
- `Dockerfile` (if applicable)
|
||||||
|
|
||||||
|
### 4. PyInstaller bundling (Phase 5)
|
||||||
|
|
||||||
|
Register the engine in `backend/build_binary.py`:
|
||||||
|
- `--hidden-import` for the backend module and model package
|
||||||
|
- `--collect-all` for packages using `inspect.getsource`, shipping data files, or native libraries
|
||||||
|
- `--copy-metadata` for packages using `importlib.metadata`
|
||||||
|
|
||||||
|
If the engine has native data paths, add `os.environ.setdefault()` in `backend/server.py` inside the `if getattr(sys, 'frozen', False):` block.
|
||||||
|
|
||||||
|
### 5. Verify in dev mode
|
||||||
|
|
||||||
|
```bash
|
||||||
|
just dev
|
||||||
|
```
|
||||||
|
|
||||||
|
Test the full chain: model download → load → generate → voice cloning.
|
||||||
|
|
||||||
|
### 6. Use the checklist
|
||||||
|
|
||||||
|
Walk through the Implementation Checklist at the bottom of `tts-engines.mdx`. Every item must be checked before handing the build to the user.
|
||||||
|
|
||||||
|
## Key Lessons (from v0.2.3)
|
||||||
|
|
||||||
|
These are the most common failure modes. Phase 0 research catches all of them:
|
||||||
|
|
||||||
|
| Pattern | Symptom in Frozen Build | Fix |
|
||||||
|
|---------|------------------------|-----|
|
||||||
|
| `@typechecked` / `inspect.getsource()` | "could not get source code" | `--collect-all <package>` |
|
||||||
|
| Package ships pretrained model files | `FileNotFoundError` for `.pth.tar`, `.yaml` | `--collect-all <package>` |
|
||||||
|
| C library with hardcoded system paths | `FileNotFoundError` for `/usr/share/...` | `--collect-all` + env var in `server.py` |
|
||||||
|
| `importlib.metadata.version()` | "No package metadata found" | `--copy-metadata <package>` |
|
||||||
|
| `torch.load` without `map_location` | CUDA device not available on CPU build | Monkey-patch `torch.load` |
|
||||||
|
| `torch.from_numpy` on float64 data | dtype mismatch RuntimeError | Cast to `.float()` |
|
||||||
|
| `token=True` in HF download calls | Auth failure without stored HF token | Use `snapshot_download(token=None)` + `from_local()` |
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- The route and service layers have zero per-engine dispatch points. `main.py` requires zero changes.
|
||||||
|
- The model config registry in `backends/__init__.py` handles all dispatch automatically.
|
||||||
|
- Use `get_torch_device()` and `model_load_progress()` from `backends/base.py` — don't reimplement device detection or progress tracking.
|
||||||
|
- Always test with a **clean HuggingFace cache** (no pre-downloaded models from dev).
|
||||||
|
- Do NOT push or create a release. Hand the build to the user for local testing.
|
||||||
@@ -0,0 +1,94 @@
|
|||||||
|
---
|
||||||
|
name: draft-release-notes
|
||||||
|
description: Use this skill to draft or update the [Unreleased] section of CHANGELOG.md from the actual changes since the last tag. Run this at any point during development to keep a working copy of the release narrative. Does NOT bump versions or create tags.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Draft Release Notes
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Update the `[Unreleased]` section at the top of `CHANGELOG.md` with a narrative release story based on the real changes since the last tag. This is a **non-destructive working copy** — run it as many times as you want during development.
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
1. **Identify the last release tag and gather changes.**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
LAST_TAG=$(git tag --list "v*" --sort=-v:refname | head -n 1)
|
||||||
|
echo "Last tag: $LAST_TAG"
|
||||||
|
```
|
||||||
|
|
||||||
|
Then collect raw material from three sources:
|
||||||
|
|
||||||
|
a. **Commit log since last tag:**
|
||||||
|
```bash
|
||||||
|
git log --oneline "$LAST_TAG"..HEAD
|
||||||
|
```
|
||||||
|
|
||||||
|
b. **GitHub-generated release notes preview** (PR titles, new contributors):
|
||||||
|
```bash
|
||||||
|
gh api repos/:owner/:repo/releases/generate-notes \
|
||||||
|
-f tag_name="vNEXT" \
|
||||||
|
-f target_commitish="$(git rev-parse HEAD)" \
|
||||||
|
-f previous_tag_name="$LAST_TAG" \
|
||||||
|
--jq '.body'
|
||||||
|
```
|
||||||
|
|
||||||
|
c. **Diff stat for theme analysis:**
|
||||||
|
```bash
|
||||||
|
git diff --stat "$LAST_TAG"..HEAD
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Draft the release narrative.**
|
||||||
|
|
||||||
|
Write markdown for the `[Unreleased]` section following the format below. Do not include the `## [Unreleased]` heading itself — just the body content.
|
||||||
|
|
||||||
|
3. **Update CHANGELOG.md.**
|
||||||
|
|
||||||
|
Replace everything between `## [Unreleased]` and the next `## [` heading with the new draft. Preserve the HTML comment header and all existing release sections below.
|
||||||
|
|
||||||
|
The `[Unreleased]` section must always exist and always be the first section after the header comments.
|
||||||
|
|
||||||
|
4. **Do NOT commit, tag, or bump versions.** Just leave the file modified in the working tree.
|
||||||
|
|
||||||
|
## Release Story Format
|
||||||
|
|
||||||
|
Structure the `[Unreleased]` section like this:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
## [Unreleased]
|
||||||
|
|
||||||
|
<One strong opening paragraph: what this release is about and why it matters.
|
||||||
|
Tie it to concrete shipped changes. No vague hype.>
|
||||||
|
|
||||||
|
<One paragraph on major technical shifts, if applicable.>
|
||||||
|
|
||||||
|
### <Feature/Theme Group>
|
||||||
|
- Bullet points with specifics
|
||||||
|
- Reference PRs where available: ([#123](https://github.com/jamiepine/voicebox/pull/123))
|
||||||
|
|
||||||
|
### <Another Group>
|
||||||
|
- ...
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
- ...
|
||||||
|
```
|
||||||
|
|
||||||
|
### Style Guidelines
|
||||||
|
|
||||||
|
- **Factual and specific.** Every claim should trace to a real commit or PR.
|
||||||
|
- **Narrative over list.** Lead with paragraphs that tell the story, then support with bullets.
|
||||||
|
- **Group by theme, not by commit.** Cluster related changes under descriptive headings.
|
||||||
|
- **Reference PRs** where they exist, but don't fabricate them.
|
||||||
|
- **Skip trivial chores** (typo fixes, CI tweaks) unless they're the bulk of the release.
|
||||||
|
- **Match the voice of existing releases** — look at the v0.2.1 and v0.2.3 entries in CHANGELOG.md for tone reference.
|
||||||
|
|
||||||
|
## When There Are No Changes
|
||||||
|
|
||||||
|
If `git log "$LAST_TAG"..HEAD` is empty, leave the `[Unreleased]` section empty (just the heading) and tell the user there's nothing to draft.
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- This skill only touches the `[Unreleased]` section. It never modifies stamped release sections.
|
||||||
|
- The agent can be asked to run this skill at any point — mid-feature, before a PR, or right before cutting a release.
|
||||||
|
- The `release-bump` skill depends on this draft being up to date before it finalizes.
|
||||||
@@ -0,0 +1,124 @@
|
|||||||
|
---
|
||||||
|
name: release-bump
|
||||||
|
description: Use this skill to finalize a release. It stamps the [Unreleased] changelog section with a version and date, runs bumpversion to update all version files, and creates the release commit and tag. Only run this when you're ready to ship.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Release Bump
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Finalize the changelog draft, bump the version across all tracked files, and create a tagged release commit. After this skill runs, the repo has a clean release commit and tag ready to push.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- `gh` CLI installed and authenticated (`gh auth status`).
|
||||||
|
- `bumpversion` installed (`pip install bumpversion` or available in the project venv).
|
||||||
|
- The `[Unreleased]` section of `CHANGELOG.md` should already contain the release narrative. If it's empty or stale, run the `draft-release-notes` skill first.
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
1. **Verify the working tree is clean** (except `CHANGELOG.md` which may have the draft).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git status --porcelain
|
||||||
|
```
|
||||||
|
|
||||||
|
Only `CHANGELOG.md` (and optionally `.agents/` files) should be modified. If there are other uncommitted changes, stop and ask the user to commit or stash them first.
|
||||||
|
|
||||||
|
2. **Determine the bump level.**
|
||||||
|
|
||||||
|
Ask the user if not specified: `patch`, `minor`, or `major`. Check the current version:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
grep '^current_version' .bumpversion.cfg
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Stamp the changelog.**
|
||||||
|
|
||||||
|
Read the current `[Unreleased]` content from `CHANGELOG.md`. Compute the new version (based on bump level and current version). Then:
|
||||||
|
|
||||||
|
a. Replace the `## [Unreleased]` section body with an empty placeholder.
|
||||||
|
b. Insert a new stamped section immediately after `## [Unreleased]`:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
## [Unreleased]
|
||||||
|
|
||||||
|
## [X.Y.Z] - YYYY-MM-DD
|
||||||
|
|
||||||
|
<the content that was in [Unreleased]>
|
||||||
|
```
|
||||||
|
|
||||||
|
c. Update the reference links at the bottom of the file:
|
||||||
|
- Change the `[Unreleased]` link to compare against the new tag
|
||||||
|
- Add a new link for the new version
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
[Unreleased]: https://github.com/jamiepine/voicebox/compare/vX.Y.Z...HEAD
|
||||||
|
[X.Y.Z]: https://github.com/jamiepine/voicebox/compare/vPREVIOUS...vX.Y.Z
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Stage the changelog.**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add CHANGELOG.md
|
||||||
|
```
|
||||||
|
|
||||||
|
5. **Run bumpversion.**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bumpversion --allow-dirty <patch|minor|major>
|
||||||
|
```
|
||||||
|
|
||||||
|
The `--allow-dirty` flag is needed because `CHANGELOG.md` is already staged. bumpversion will:
|
||||||
|
- Update version strings in all tracked files (see `.bumpversion.cfg`)
|
||||||
|
- Create a commit with message `Bump version: X.Y.Z -> A.B.C`
|
||||||
|
- Create a tag `vA.B.C`
|
||||||
|
|
||||||
|
The staged `CHANGELOG.md` will be included in this commit automatically.
|
||||||
|
|
||||||
|
6. **Verify results.**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git show --name-only --stat HEAD
|
||||||
|
git tag --list "v*" --sort=-v:refname | head -n 5
|
||||||
|
```
|
||||||
|
|
||||||
|
Confirm the commit contains:
|
||||||
|
- `CHANGELOG.md`
|
||||||
|
- `.bumpversion.cfg`
|
||||||
|
- `tauri/src-tauri/tauri.conf.json`
|
||||||
|
- `tauri/src-tauri/Cargo.toml`
|
||||||
|
- `package.json`
|
||||||
|
- `app/package.json`
|
||||||
|
- `tauri/package.json`
|
||||||
|
- `landing/package.json`
|
||||||
|
- `web/package.json`
|
||||||
|
- `backend/__init__.py`
|
||||||
|
|
||||||
|
Confirm the new tag exists.
|
||||||
|
|
||||||
|
7. **Do NOT push** unless the user explicitly asks. Report the tag name and suggest:
|
||||||
|
|
||||||
|
```
|
||||||
|
Ready to push. When you're ready:
|
||||||
|
git push origin main --follow-tags
|
||||||
|
```
|
||||||
|
|
||||||
|
## Version Calculation Reference
|
||||||
|
|
||||||
|
Given current version `X.Y.Z`:
|
||||||
|
- `patch` -> `X.Y.(Z+1)`
|
||||||
|
- `minor` -> `X.(Y+1).0`
|
||||||
|
- `major` -> `(X+1).0.0`
|
||||||
|
|
||||||
|
## Error Recovery
|
||||||
|
|
||||||
|
- If bumpversion fails, the tag won't exist. Fix the issue and re-run — bumpversion is idempotent as long as the tag doesn't already exist.
|
||||||
|
- If you need to undo a release commit (before pushing): `git tag -d vX.Y.Z && git reset --soft HEAD~1`
|
||||||
|
- Never amend a release commit that has been pushed.
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- When the tag is pushed, the release CI (`.github/workflows/release.yml`) automatically extracts the matching version section from `CHANGELOG.md` and uses it as the GitHub Release body. No manual copy-paste needed.
|
||||||
|
- The release commit message is controlled by `.bumpversion.cfg` (`Bump version: X.Y.Z -> A.B.C`). Do not override it.
|
||||||
|
- If you need to manually update the GitHub Release body after the fact: `gh release edit vX.Y.Z --notes-file <(sed -n '/## \[X.Y.Z\]/,/## \[/p' CHANGELOG.md | head -n -1)`
|
||||||
+1
-1
@@ -1,5 +1,5 @@
|
|||||||
[bumpversion]
|
[bumpversion]
|
||||||
current_version = 0.2.3
|
current_version = 0.3.1
|
||||||
commit = True
|
commit = True
|
||||||
tag = True
|
tag = True
|
||||||
tag_name = v{new_version}
|
tag_name = v{new_version}
|
||||||
|
|||||||
@@ -62,6 +62,7 @@ jobs:
|
|||||||
pip install pyinstaller
|
pip install pyinstaller
|
||||||
pip install -r backend/requirements.txt
|
pip install -r backend/requirements.txt
|
||||||
pip install --no-deps chatterbox-tts
|
pip install --no-deps chatterbox-tts
|
||||||
|
pip install --no-deps hume-tada
|
||||||
|
|
||||||
- name: Install MLX dependencies (Apple Silicon only)
|
- name: Install MLX dependencies (Apple Silicon only)
|
||||||
if: matrix.backend == 'mlx'
|
if: matrix.backend == 'mlx'
|
||||||
@@ -123,6 +124,29 @@ jobs:
|
|||||||
p12-file-base64: ${{ secrets.APPLE_CERTIFICATE }}
|
p12-file-base64: ${{ secrets.APPLE_CERTIFICATE }}
|
||||||
p12-password: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
|
p12-password: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
|
||||||
|
|
||||||
|
- name: Extract release notes from CHANGELOG.md
|
||||||
|
id: changelog
|
||||||
|
shell: bash
|
||||||
|
run: |
|
||||||
|
# Get the version from the tag (strip leading 'v')
|
||||||
|
VERSION="${GITHUB_REF_NAME#v}"
|
||||||
|
|
||||||
|
# Extract the section for this version from CHANGELOG.md
|
||||||
|
# Matches from "## [X.Y.Z]" until the next "## [" heading
|
||||||
|
NOTES=$(sed -n "/^## \[${VERSION}\]/,/^## \[/{/^## \[${VERSION}\]/d;/^## \[/d;p;}" CHANGELOG.md)
|
||||||
|
|
||||||
|
# Fall back to a placeholder if the version isn't in the changelog
|
||||||
|
if [ -z "$(echo "$NOTES" | tr -d '[:space:]')" ]; then
|
||||||
|
NOTES="See the assets below to download and install this version."
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Use multiline output syntax
|
||||||
|
{
|
||||||
|
echo "notes<<CHANGELOG_EOF"
|
||||||
|
echo "$NOTES"
|
||||||
|
echo "CHANGELOG_EOF"
|
||||||
|
} >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
- uses: tauri-apps/[email protected]
|
- uses: tauri-apps/[email protected]
|
||||||
env:
|
env:
|
||||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||||
@@ -139,17 +163,7 @@ jobs:
|
|||||||
projectPath: tauri
|
projectPath: tauri
|
||||||
tagName: v__VERSION__
|
tagName: v__VERSION__
|
||||||
releaseName: "voicebox v__VERSION__"
|
releaseName: "voicebox v__VERSION__"
|
||||||
releaseBody: |
|
releaseBody: ${{ steps.changelog.outputs.notes }}
|
||||||
## What's Changed
|
|
||||||
See the assets below to download and install this version.
|
|
||||||
|
|
||||||
### Installation
|
|
||||||
- **macOS (Apple Silicon)**: Download the `aarch64.dmg` file - uses MLX for fast native inference
|
|
||||||
- **macOS (Intel)**: Download the `x64.dmg` file - uses PyTorch
|
|
||||||
- **Windows**: Download the `.msi` installer
|
|
||||||
- **Linux**: Compile from source (see README)
|
|
||||||
|
|
||||||
The app includes automatic updates - future updates will be installed automatically.
|
|
||||||
releaseDraft: true
|
releaseDraft: true
|
||||||
prerelease: false
|
prerelease: false
|
||||||
args: ${{ matrix.args }}
|
args: ${{ matrix.args }}
|
||||||
@@ -175,43 +189,48 @@ jobs:
|
|||||||
pip install pyinstaller
|
pip install pyinstaller
|
||||||
pip install -r backend/requirements.txt
|
pip install -r backend/requirements.txt
|
||||||
pip install --no-deps chatterbox-tts
|
pip install --no-deps chatterbox-tts
|
||||||
|
pip install --no-deps hume-tada
|
||||||
|
|
||||||
- name: Install PyTorch with CUDA 12.1
|
- name: Install PyTorch with CUDA 12.8
|
||||||
run: |
|
run: |
|
||||||
pip install torch --index-url https://download.pytorch.org/whl/cu121 --force-reinstall --no-deps
|
pip install torch --index-url https://download.pytorch.org/whl/cu128 --force-reinstall --no-deps
|
||||||
pip install torchaudio --index-url https://download.pytorch.org/whl/cu121
|
pip install torchaudio --index-url https://download.pytorch.org/whl/cu128 --force-reinstall --no-deps
|
||||||
|
|
||||||
- name: Verify CUDA support in torch
|
- name: Verify CUDA support in torch
|
||||||
run: |
|
run: |
|
||||||
python -c "import torch; print(f'CUDA available in build: {torch.cuda.is_available()}'); print(f'CUDA version: {torch.version.cuda}')"
|
python -c "import torch; print(f'CUDA available in build: {torch.cuda.is_available()}'); print(f'CUDA version: {torch.version.cuda}')"
|
||||||
|
|
||||||
- name: Build CUDA server binary
|
- name: Build CUDA server binary (onedir)
|
||||||
shell: bash
|
shell: bash
|
||||||
working-directory: backend
|
working-directory: backend
|
||||||
run: python build_binary.py --cuda
|
run: python build_binary.py --cuda
|
||||||
|
|
||||||
- name: Split binary for GitHub Releases
|
- name: Package into server core + CUDA libs archives
|
||||||
shell: bash
|
shell: bash
|
||||||
run: |
|
run: |
|
||||||
python scripts/split_binary.py \
|
python scripts/package_cuda.py \
|
||||||
backend/dist/voicebox-server-cuda.exe \
|
backend/dist/voicebox-server-cuda/ \
|
||||||
--output release-assets/
|
--output release-assets/ \
|
||||||
|
--cuda-libs-version cu128-v1 \
|
||||||
|
--torch-compat ">=2.7.0,<2.11.0"
|
||||||
|
|
||||||
- name: Upload split parts to GitHub Release
|
- name: Upload archives to GitHub Release
|
||||||
if: startsWith(github.ref, 'refs/tags/')
|
if: startsWith(github.ref, 'refs/tags/')
|
||||||
uses: softprops/action-gh-release@v1
|
uses: softprops/action-gh-release@v2
|
||||||
with:
|
with:
|
||||||
files: |
|
files: |
|
||||||
release-assets/voicebox-server-cuda.part*.exe
|
release-assets/voicebox-server-cuda.tar.gz
|
||||||
release-assets/voicebox-server-cuda.sha256
|
release-assets/voicebox-server-cuda.tar.gz.sha256
|
||||||
release-assets/voicebox-server-cuda.manifest
|
release-assets/cuda-libs-cu128-v1.tar.gz
|
||||||
|
release-assets/cuda-libs-cu128-v1.tar.gz.sha256
|
||||||
|
release-assets/cuda-libs.json
|
||||||
draft: true
|
draft: true
|
||||||
env:
|
env:
|
||||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||||
|
|
||||||
- name: Upload binary as workflow artifact
|
- name: Upload onedir as workflow artifact
|
||||||
uses: actions/upload-artifact@v4
|
uses: actions/upload-artifact@v4
|
||||||
with:
|
with:
|
||||||
name: voicebox-server-cuda-windows
|
name: voicebox-server-cuda-windows
|
||||||
path: backend/dist/voicebox-server-cuda.exe
|
path: backend/dist/voicebox-server-cuda/
|
||||||
retention-days: 7
|
retention-days: 7
|
||||||
|
|||||||
@@ -50,6 +50,14 @@ logs/
|
|||||||
app/openapi.json
|
app/openapi.json
|
||||||
tauri/src-tauri/binaries/*
|
tauri/src-tauri/binaries/*
|
||||||
tauri/src-tauri/gen/Assets.car
|
tauri/src-tauri/gen/Assets.car
|
||||||
|
tauri/src-tauri/gen/voicebox.icns
|
||||||
|
tauri/src-tauri/gen/partial.plist
|
||||||
|
|
||||||
|
# PyInstaller
|
||||||
|
*.spec
|
||||||
|
|
||||||
|
# Windows artifacts
|
||||||
|
nul
|
||||||
|
|
||||||
# Temporary
|
# Temporary
|
||||||
tmp/
|
tmp/
|
||||||
|
|||||||
+437
-69
@@ -1,96 +1,464 @@
|
|||||||
|
<!-- This file is compiled automatically during the release workflow. -->
|
||||||
|
<!-- Do not edit manually — your changes will be overwritten. -->
|
||||||
|
<!-- To update the draft: ask the agent to use the draft-release-notes skill. -->
|
||||||
|
<!-- To finalize a release: ask the agent to use the release-bump skill. -->
|
||||||
|
|
||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
All notable changes to Voicebox will be documented in this file.
|
|
||||||
|
|
||||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
||||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
### Fixed
|
## [0.3.0] - 2026-03-17
|
||||||
- **Profile Name Validation** - Added proper validation to prevent duplicate profile names ([#134](https://github.com/jamiepine/voicebox/issues/134))
|
|
||||||
- Users now receive clear error messages when attempting to create or update profiles with duplicate names
|
|
||||||
- Improved error handling in create and update profile API endpoints
|
|
||||||
- Added comprehensive test suite for duplicate name validation
|
|
||||||
|
|
||||||
## [0.1.0] - 2026-01-25
|
This release rewrites the backend into a modular architecture, overhauls the settings UI into routed sub-pages, fixes audio player freezing, migrates documentation to Fumadocs, and ships a batch of bug fixes targeting the most-reported issues from the tracker.
|
||||||
|
|
||||||
### Added
|
The backend's 3,000-line monolith `main.py` has been decomposed into domain routers, a services layer, and a proper database package. A style guide and ruff configuration now enforce consistency. On the frontend, settings have been split into dedicated routed pages with server logs, a changelog viewer, and an about page. The audio player no longer freezes mid-playback, and model loading status is now visible in the UI. Seven user-reported bugs have been fixed, including server crashes during sample uploads, generation list staleness, cryptic error messages, and CUDA support for RTX 50-series GPUs.
|
||||||
|
|
||||||
#### Core Features
|
### Settings Overhaul ([#294](https://github.com/jamiepine/voicebox/pull/294))
|
||||||
- **Voice Cloning** - Clone voices from audio samples using Qwen3-TTS (1.7B and 0.6B models)
|
- Split settings into routed sub-tabs: General, Generation, GPU, Logs, Changelog, About
|
||||||
- **Voice Profile Management** - Create, edit, and organize voice profiles with multiple samples
|
- Added live server log viewer with auto-scroll
|
||||||
- **Speech Generation** - Generate high-quality speech from text using cloned voices
|
- Added in-app changelog page that parses `CHANGELOG.md` at build time
|
||||||
- **Generation History** - Track all generations with search and filtering capabilities
|
- Added About page with version info, license, and generation folder quick-open
|
||||||
- **Audio Transcription** - Automatic transcription powered by Whisper
|
- Extracted reusable `SettingRow` component for consistent setting layouts
|
||||||
- **In-App Recording** - Record audio samples directly in the app with waveform visualization
|
|
||||||
|
|
||||||
#### Desktop App
|
### Audio Player Fix ([#293](https://github.com/jamiepine/voicebox/pull/293))
|
||||||
- **Tauri Desktop App** - Native desktop application for macOS, Windows, and Linux
|
- Fixed audio player freezing during playback
|
||||||
- **Local Server Mode** - Embedded Python server runs automatically
|
- Improved playback UX with better state management and listener cleanup
|
||||||
- **Remote Server Mode** - Connect to a remote Voicebox server on your network
|
- Fixed restart race condition during regeneration
|
||||||
- **Auto-Updates** - Automatic update notifications and installation
|
- Added stable keys for audio element re-rendering
|
||||||
|
- Improved accessibility across player controls
|
||||||
|
|
||||||
#### API
|
### Backend Refactor ([#285](https://github.com/jamiepine/voicebox/pull/285))
|
||||||
- **REST API** - Full REST API for voice synthesis and profile management
|
- Extracted all routes from `main.py` into 13 domain routers under `backend/routes/` — `main.py` dropped from ~3,100 lines to ~10
|
||||||
- **OpenAPI Documentation** - Interactive API docs at `/docs` endpoint
|
- Moved CRUD and service modules into `backend/services/`, platform detection into `backend/utils/`
|
||||||
- **Type-Safe Client** - Auto-generated TypeScript client from OpenAPI schema
|
- Split monolithic `database.py` into a `database/` package with separate `models`, `session`, `migrations`, and `seed` modules
|
||||||
|
- Added `backend/STYLE_GUIDE.md` and `pyproject.toml` with ruff linting config
|
||||||
|
- Removed dead code: unused `_get_cuda_dll_excludes`, stale `studio.py`, `example_usage.py`, old `Makefile`
|
||||||
|
- Deduplicated shared logic across TTS backends into `backends/base.py`
|
||||||
|
- Improved startup logging with version, platform, data directory, and database stats
|
||||||
|
- Fixed startup database session leak — sessions now rollback and close in `finally` block
|
||||||
|
- Isolated shutdown unload calls so one backend failure doesn't block the others
|
||||||
|
- Handled null duration in `story_items` migration
|
||||||
|
- Reject model migration when target is a subdirectory of source cache
|
||||||
|
|
||||||
#### Technical
|
### Documentation Rewrite ([#288](https://github.com/jamiepine/voicebox/pull/288))
|
||||||
- **Voice Prompt Caching** - Fast regeneration with cached voice prompts
|
- Migrated docs site from Mintlify to Fumadocs (Next.js-based)
|
||||||
- **Multi-Sample Support** - Combine multiple audio samples for better voice quality
|
- Rewrote introduction and root page with content from README
|
||||||
- **GPU/CPU/MPS Support** - Automatic device detection and optimization
|
- Added "Edit on GitHub" links and last-updated timestamps on all pages
|
||||||
- **Model Management** - Lazy loading and VRAM management
|
- Generated OpenAPI spec and auto-generated API reference pages
|
||||||
- **SQLite Database** - Local data persistence
|
- Removed stale planning docs (`CUDA_BACKEND_SWAP`, `EXTERNAL_PROVIDERS`, `MLX_AUDIO`, `TTS_PROVIDER_ARCHITECTURE`, etc.)
|
||||||
|
- Sidebar groups now expand by default; root redirects to `/docs`
|
||||||
|
- Added OG image metadata and `/og` preview page
|
||||||
|
|
||||||
### Technical Details
|
### UI & Frontend
|
||||||
|
- Added model loading status indicator and effects preset dropdown ([3187344](https://github.com/jamiepine/voicebox/commit/3187344))
|
||||||
|
- Fixed take-label race condition during regeneration
|
||||||
|
- Added accessible focus styling to select component
|
||||||
|
- Softened select focus indicator opacity
|
||||||
|
- Addressed 4 critical and 12 major issues from CodeRabbit review
|
||||||
|
|
||||||
- Built with Tauri v2 (Rust + React)
|
### Bug Fixes ([#295](https://github.com/jamiepine/voicebox/pull/295))
|
||||||
- FastAPI backend with async Python
|
- Fixed sample uploads crashing the server — audio decoding now runs in a thread pool instead of blocking the async event loop ([#278](https://github.com/jamiepine/voicebox/issues/278))
|
||||||
- TypeScript frontend with React Query and Zustand
|
- Fixed generation list not updating when a generation completes — switched to `refetchQueries` for reliable cache busting, added SSE error fallback, and page reset on completion ([#231](https://github.com/jamiepine/voicebox/issues/231))
|
||||||
- Qwen3-TTS for voice cloning
|
- Fixed error toasts showing `[object Object]` instead of the actual error message ([#290](https://github.com/jamiepine/voicebox/issues/290))
|
||||||
- Whisper for transcription
|
- Added Whisper model selection (`base`, `small`, `medium`, `large`, `turbo`) and expanded language support to the `/transcribe` endpoint ([#233](https://github.com/jamiepine/voicebox/issues/233))
|
||||||
|
- Upgraded CUDA backend build from cu121 to cu126 for RTX 50-series (Blackwell) GPU support ([#289](https://github.com/jamiepine/voicebox/issues/289))
|
||||||
|
- Handled client disconnects in SSE and streaming endpoints to suppress `[Errno 32] Broken Pipe` errors ([#248](https://github.com/jamiepine/voicebox/issues/248))
|
||||||
|
- Fixed Docker build failure from pip hash mismatch on Qwen3-TTS dependencies ([#286](https://github.com/jamiepine/voicebox/issues/286))
|
||||||
|
- Added 50 MB upload size limit with chunked reads to prevent unbounded memory allocation on sample uploads
|
||||||
|
- Eliminated redundant double audio decode in sample processing pipeline
|
||||||
|
|
||||||
|
### Platform Fixes
|
||||||
|
- Replaced `netstat` with `TcpStream` + PowerShell for Windows port detection ([#277](https://github.com/jamiepine/voicebox/pull/277))
|
||||||
|
- Fixed Docker frontend build and cleaned up Docker docs
|
||||||
|
- Fixed macOS download links to use `.dmg` instead of `.app.tar.gz`
|
||||||
|
- Added dynamic download redirect routes to landing site
|
||||||
|
|
||||||
|
### Release Tooling
|
||||||
|
- Added `draft-release-notes` and `release-bump` agent skills
|
||||||
|
- Wired CI release workflow to extract notes from `CHANGELOG.md` for GitHub Releases
|
||||||
|
- Backfilled changelog with all historical releases
|
||||||
|
|
||||||
|
## [0.2.3] - 2026-03-15
|
||||||
|
|
||||||
|
The "it works in dev but not in prod" release. This version fixes a series of PyInstaller bundling issues that prevented model downloading, loading, generation, and progress tracking from working in production builds.
|
||||||
|
|
||||||
|
### Model Downloads Now Actually Work
|
||||||
|
|
||||||
|
The v0.2.1/v0.2.2 builds could not download or load models that weren't already cached from a dev install. This release fixes the entire chain:
|
||||||
|
|
||||||
|
- **Chatterbox, Chatterbox Turbo, and LuxTTS** all download, load, and generate correctly in bundled builds
|
||||||
|
- **Real-time download progress** — byte-level progress bars now work in production. The root cause: `huggingface_hub` silently disables tqdm progress bars based on logger level, which prevented our progress tracker from receiving byte updates. We now force-enable the internal counter regardless.
|
||||||
|
- **Fixed Python 3.12.0 `code.replace()` bug** — the macOS build was on Python 3.12.0, which has a [known CPython bug](https://github.com/pyinstaller/pyinstaller/issues/7992) that corrupts bytecode when PyInstaller rewrites code objects. This caused `NameError: name 'obj' is not defined` crashes during scipy/torch imports. Upgraded to Python 3.12.13.
|
||||||
|
|
||||||
|
### PyInstaller Fixes
|
||||||
|
|
||||||
|
- Collect all `inflect` files — `typeguard`'s `@typechecked` decorator calls `inspect.getsource()` at import time, which needs `.py` source files, not just bytecode. Fixes LuxTTS "could not get source code" error.
|
||||||
|
- Collect all `perth` files — bundles the pretrained watermark model (`hparams.yaml`, `.pth.tar`) needed by Chatterbox at runtime
|
||||||
|
- Collect all `piper_phonemize` files — bundles `espeak-ng-data/` (phoneme tables, language dicts) needed by LuxTTS for text-to-phoneme conversion
|
||||||
|
- Set `ESPEAK_DATA_PATH` in frozen builds so the espeak-ng C library finds the bundled data instead of looking at `/usr/share/espeak-ng-data/`
|
||||||
|
- Collect all `linacodec` files — fixes `inspect.getsource` error in Vocos codec
|
||||||
|
- Collect all `zipvoice` files — fixes source code lookup in LuxTTS voice cloning
|
||||||
|
- Copy metadata for `requests`, `transformers`, `huggingface-hub`, `tokenizers`, `safetensors`, `tqdm` — fixes `importlib.metadata` lookups in frozen binary
|
||||||
|
- Add hidden imports for `chatterbox`, `chatterbox_turbo`, `luxtts`, `zipvoice` backends
|
||||||
|
- Add `multiprocessing.freeze_support()` to fix resource_tracker subprocess crash in frozen binary
|
||||||
|
- `--noconsole` now only applied on Windows — macOS/Linux need stdout/stderr for Tauri sidecar log capture
|
||||||
|
- Hardened `sys.stdout`/`sys.stderr` devnull redirect to test writability, not just `None` check
|
||||||
|
|
||||||
|
### Updater
|
||||||
|
|
||||||
|
- Fixed updater artifact generation with `v1Compatible` for `tauri-action` signature files
|
||||||
|
- Updated `tauri-action` to v0.6 to fix updater JSON and `.sig` generation
|
||||||
|
|
||||||
|
### Other Fixes
|
||||||
|
|
||||||
|
- Full traceback logging on all backend model loading errors (was just `str(e)` before)
|
||||||
|
|
||||||
|
## [0.2.2] - 2026-03-15
|
||||||
|
|
||||||
|
- Fix Chatterbox model support in bundled builds
|
||||||
|
- Fix LuxTTS/ZipVoice support in bundled builds
|
||||||
|
- Auto-update CUDA binary when app version changes
|
||||||
|
- CUDA download progress bar
|
||||||
|
- Fix server process staying alive on macOS (SIGHUP handling, watchdog grace period)
|
||||||
|
- Hide console window when running CUDA binary on Windows
|
||||||
|
|
||||||
|
## [0.2.1] - 2026-03-15
|
||||||
|
|
||||||
|
Voicebox v0.1.x was a single-engine voice cloning app built around Qwen3-TTS. v0.2.0 is a ground-up rethink: four TTS engines, 23 languages, paralinguistic emotion controls, a post-processing effects pipeline, unlimited generation length, an async generation queue, and support for every major GPU vendor. Plus Docker.
|
||||||
|
|
||||||
|
### New TTS Engines
|
||||||
|
|
||||||
|
#### Multi-Engine Architecture
|
||||||
|
|
||||||
|
Voicebox now runs **four independent TTS engines** behind a thread-safe per-engine backend registry. Switch engines per-generation from a single dropdown — no restart required.
|
||||||
|
|
||||||
|
| Engine | Languages | Size | Key Strengths |
|
||||||
|
| --------------------------- | --------- | ------- | --------------------------------------------- |
|
||||||
|
| **Qwen3-TTS 1.7B** | 10 | ~3.5 GB | Highest quality, delivery instructions |
|
||||||
|
| **Qwen3-TTS 0.6B** | 10 | ~1.2 GB | Lighter, faster variant |
|
||||||
|
| **LuxTTS** | English | ~300 MB | CPU-friendly, 48 kHz output, 150x realtime |
|
||||||
|
| **Chatterbox Multilingual** | 23 | ~3.2 GB | Broadest language coverage, zero-shot cloning |
|
||||||
|
| **Chatterbox Turbo** | English | ~1.5 GB | 350M params, low latency, paralinguistic tags |
|
||||||
|
|
||||||
|
#### Chatterbox Multilingual — 23 Languages ([#257](https://github.com/jamiepine/voicebox/pull/257))
|
||||||
|
|
||||||
|
Zero-shot voice cloning in Arabic, Chinese, Danish, Dutch, English, Finnish, French, German, Greek, Hebrew, Hindi, Italian, Japanese, Korean, Malay, Norwegian, Polish, Portuguese, Russian, Spanish, Swahili, Swedish, and Turkish.
|
||||||
|
|
||||||
|
#### LuxTTS — Lightweight English TTS ([#254](https://github.com/jamiepine/voicebox/pull/254))
|
||||||
|
|
||||||
|
A fast, CPU-friendly English engine. ~300 MB download, 48 kHz output, runs at 150x realtime on CPU.
|
||||||
|
|
||||||
|
#### Chatterbox Turbo — Expressive English ([#258](https://github.com/jamiepine/voicebox/pull/258))
|
||||||
|
|
||||||
|
A fast 350M-parameter English model with inline paralinguistic tags.
|
||||||
|
|
||||||
|
#### Paralinguistic Tags Autocomplete ([#265](https://github.com/jamiepine/voicebox/pull/265))
|
||||||
|
|
||||||
|
Type `/` in the text input with Chatterbox Turbo selected to open an autocomplete for **9 expressive tags**: `[laugh]` `[chuckle]` `[gasp]` `[cough]` `[sigh]` `[groan]` `[sniff]` `[shush]` `[clear throat]`
|
||||||
|
|
||||||
|
### Generation
|
||||||
|
|
||||||
|
#### Unlimited Generation Length — Auto-Chunking ([#266](https://github.com/jamiepine/voicebox/pull/266))
|
||||||
|
|
||||||
|
Long text is now automatically split at sentence boundaries, generated per-chunk, and crossfaded back together. Engine-agnostic.
|
||||||
|
|
||||||
|
- Auto-chunking limit slider — 100–5,000 chars (default 800)
|
||||||
|
- Crossfade slider — 0–200ms (default 50ms)
|
||||||
|
- Max text length raised to 50,000 characters
|
||||||
|
- Smart splitting respects abbreviations, CJK punctuation, and `[tags]`
|
||||||
|
|
||||||
|
#### Asynchronous Generation Queue ([#269](https://github.com/jamiepine/voicebox/pull/269))
|
||||||
|
|
||||||
|
Generation is now fully non-blocking. Serial execution queue prevents GPU contention. Real-time SSE status streaming.
|
||||||
|
|
||||||
|
#### Generation Versions
|
||||||
|
|
||||||
|
Every generation now supports multiple versions with provenance tracking — original, effects versions, takes, source tracking, version pinning in stories, and favorites.
|
||||||
|
|
||||||
|
### Post-Processing Effects ([#271](https://github.com/jamiepine/voicebox/pull/271))
|
||||||
|
|
||||||
|
A full audio effects system powered by Spotify's `pedalboard` library: Pitch Shift, Reverb, Delay, Chorus/Flanger, Compressor, Gain, High-Pass Filter, Low-Pass Filter. 4 built-in presets, custom presets, per-profile default effects, and live preview.
|
||||||
|
|
||||||
### Platform Support
|
### Platform Support
|
||||||
|
|
||||||
- macOS (Apple Silicon and Intel)
|
- **Windows Support** ([#272](https://github.com/jamiepine/voicebox/pull/272)) — Full Windows support with CUDA GPU detection
|
||||||
- Windows
|
- **Linux** ([#262](https://github.com/jamiepine/voicebox/pull/262)) — AMD ROCm, NVIDIA GBM fix, WebKitGTK mic access (build from source)
|
||||||
- Linux (AppImage)
|
- **NVIDIA CUDA Backend Swap** ([#252](https://github.com/jamiepine/voicebox/pull/252)) — Download and swap in CUDA backend from within the app
|
||||||
|
- **Intel Arc (XPU) and DirectML** — PyTorch backend supports Intel Arc and DirectML
|
||||||
|
- **Docker + Web Deployment** ([#161](https://github.com/jamiepine/voicebox/pull/161)) — 3-stage build, non-root runtime, health checks
|
||||||
|
- **Whisper Turbo** — Added `openai/whisper-large-v3-turbo` as a transcription model option
|
||||||
|
|
||||||
---
|
### Model Management ([#268](https://github.com/jamiepine/voicebox/pull/268))
|
||||||
|
|
||||||
## [Unreleased]
|
Per-model unload, custom models directory, model folder migration, download cancel/clear UI ([#238](https://github.com/jamiepine/voicebox/pull/238)), restructured settings UI.
|
||||||
|
|
||||||
### Fixed
|
### Security & Reliability
|
||||||
- Audio export failing when Tauri save dialog returns object instead of string path
|
|
||||||
- OpenAPI client generator script now documents the local backend port and avoids an unused loop variable warning
|
|
||||||
|
|
||||||
### Added
|
- CORS hardening ([#88](https://github.com/jamiepine/voicebox/pull/88))
|
||||||
- **justfile** - Comprehensive development workflow automation with commands for setup, development, building, testing, and code quality checks
|
- Network access toggle ([#133](https://github.com/jamiepine/voicebox/pull/133))
|
||||||
- Cross-platform support (macOS, Linux, Windows)
|
- Offline crash fix ([#152](https://github.com/jamiepine/voicebox/pull/152))
|
||||||
- Python version detection and compatibility warnings
|
- Atomic audio saves ([#263](https://github.com/jamiepine/voicebox/pull/263))
|
||||||
- Self-documenting help system with `just --list`
|
- Filesystem health endpoint
|
||||||
|
- Chatterbox float64 dtype fix ([#264](https://github.com/jamiepine/voicebox/pull/264))
|
||||||
|
|
||||||
### Changed
|
### Accessibility ([#243](https://github.com/jamiepine/voicebox/pull/243))
|
||||||
- **README** - Updated Quick Start with justfile-based setup instructions
|
|
||||||
|
|
||||||
### Removed
|
Screen reader support, keyboard navigation, state-aware `aria-label` attributes on all interactive controls.
|
||||||
- **Makefile** - Replaced by justfile (cross-platform, simpler syntax)
|
|
||||||
|
|
||||||
---
|
### UI Polish
|
||||||
|
|
||||||
## [Unreleased - Planned]
|
- Redesigned landing page ([#274](https://github.com/jamiepine/voicebox/pull/274))
|
||||||
|
- Voices tab overhaul with inline inspector
|
||||||
|
- Responsive layout improvements
|
||||||
|
- Duplicate profile name validation ([#175](https://github.com/jamiepine/voicebox/pull/175))
|
||||||
|
|
||||||
### Planned
|
### Community Contributors
|
||||||
- Real-time streaming synthesis
|
|
||||||
- Conversation mode with multiple speakers
|
|
||||||
- Voice effects (pitch shift, reverb, M3GAN-style)
|
|
||||||
- Timeline-based audio editor
|
|
||||||
- Additional voice models (XTTS, Bark)
|
|
||||||
- Voice design from text descriptions
|
|
||||||
- Project system for saving sessions
|
|
||||||
- Plugin architecture
|
|
||||||
|
|
||||||
---
|
[@haosenwang1018](https://github.com/haosenwang1018), [@Balneario-de-Cofrentes](https://github.com/Balneario-de-Cofrentes), [@ageofalgo](https://github.com/ageofalgo), [@mikeswann](https://github.com/mikeswann), [@rayl15](https://github.com/rayl15), [@mpecanha](https://github.com/mpecanha), [@ways2read](https://github.com/ways2read), [@ieguiguren](https://github.com/ieguiguren), [@Vaibhavee89](https://github.com/Vaibhavee89), [@pandego](https://github.com/pandego), [@luminest-llc](https://github.com/luminest-llc)
|
||||||
|
|
||||||
|
## [0.1.13] - 2026-02-23
|
||||||
|
|
||||||
|
### Stability and reliability
|
||||||
|
|
||||||
|
- [#95](https://github.com/jamiepine/voicebox/pull/95) Fix: selecting 0.6B model still downloads and uses 1.7B
|
||||||
|
- [#93](https://github.com/jamiepine/voicebox/pull/93) fix(mlx): bundle native libs and broaden error handling for Apple Silicon
|
||||||
|
- [#79](https://github.com/jamiepine/voicebox/pull/79) fix: handle non-ASCII filenames in Content-Disposition headers
|
||||||
|
- [#78](https://github.com/jamiepine/voicebox/pull/78) fix: guard getUserMedia call against undefined mediaDevices in non-secure contexts
|
||||||
|
- [#77](https://github.com/jamiepine/voicebox/pull/77) fix: await for confirmation before deleting voices and channels
|
||||||
|
- [#128](https://github.com/jamiepine/voicebox/pull/128) fix: resolve multiple issues (#96, #119, #111, #108, #121, #125, #127)
|
||||||
|
- [#40](https://github.com/jamiepine/voicebox/pull/40) Fix: audio export path resolution
|
||||||
|
|
||||||
|
### Build and packaging
|
||||||
|
|
||||||
|
- [#122](https://github.com/jamiepine/voicebox/pull/122) fix(web): add @tailwindcss/vite plugin to web config
|
||||||
|
- [#126](https://github.com/jamiepine/voicebox/pull/126) Create requirements.txt
|
||||||
|
|
||||||
|
### UX and docs
|
||||||
|
|
||||||
|
- [#44](https://github.com/jamiepine/voicebox/pull/44) Enhances floating generate box UX
|
||||||
|
- [#57](https://github.com/jamiepine/voicebox/pull/57) chore: updates repo URL in README
|
||||||
|
- [#146](https://github.com/jamiepine/voicebox/pull/146) Add Spacebot banner to landing page
|
||||||
|
- [#1](https://github.com/jamiepine/voicebox/pull/1) Improvements
|
||||||
|
|
||||||
|
## [0.1.12] - 2026-01-31
|
||||||
|
|
||||||
|
### Model Download UX Overhaul
|
||||||
|
|
||||||
|
- Real-time download progress tracking with accurate percentage and speed info
|
||||||
|
- No more downloading notifications during generation even when its not downloading
|
||||||
|
- Better error handling and status reporting throughout the download process
|
||||||
|
|
||||||
|
### Other Improvements
|
||||||
|
|
||||||
|
- Enhanced health check endpoint with GPU type information
|
||||||
|
- Improved model caching verification
|
||||||
|
- More reliable SSE progress updates
|
||||||
|
- Actual update notifications — no need to manually check in settings anymore
|
||||||
|
|
||||||
|
## [0.1.11] - 2026-01-30
|
||||||
|
|
||||||
|
- Fixed transcriptions on MLX
|
||||||
|
- Fixed model download progress (finally)
|
||||||
|
|
||||||
|
## [0.1.10] - 2026-01-30
|
||||||
|
|
||||||
|
### Faster generation on Apple Silicon
|
||||||
|
|
||||||
|
Massive speed gains, from around 20s per generation to 2-3s. Added native MLX backend support for Apple Silicon, providing significantly faster TTS and STT generation on M-series macOS machines.
|
||||||
|
|
||||||
|
- **MLX Backend** — New backend implementation optimized for Apple Silicon using MLX framework
|
||||||
|
- **Dynamic Backend Selection** — Automatically detects platform and selects between MLX (macOS) and PyTorch (other platforms)
|
||||||
|
- Refactored TTS and STT logic into modular backend implementations
|
||||||
|
- Updated build process to include MLX-specific dependencies for macOS builds
|
||||||
|
|
||||||
|
## [0.1.9] - 2026-01-30
|
||||||
|
|
||||||
|
### Improved voice profile creation flow
|
||||||
|
|
||||||
|
- Voice create drafts: No longer lose work if you close the modal
|
||||||
|
- Fixed whisper only transcribing English or Chinese, now has support for all languages
|
||||||
|
|
||||||
|
### Improved Stories editor
|
||||||
|
|
||||||
|
- Added spacebar for play/pause
|
||||||
|
- Timeline now auto-scrolls to follow playhead during playback
|
||||||
|
- Fixed misalignment of the items with mouse when picking up
|
||||||
|
- Fixed hitbox for selecting an item
|
||||||
|
- Fixed playhead jumping forward when pressing play
|
||||||
|
|
||||||
|
### Generation box improvements
|
||||||
|
|
||||||
|
- Instruct mode no longer wipes prompt text
|
||||||
|
- Improved UI cleanliness
|
||||||
|
|
||||||
|
### Misc
|
||||||
|
|
||||||
|
- Fixed "Model downloading" toast during generation when model is already downloaded
|
||||||
|
|
||||||
|
## [0.1.8] - 2026-01-29
|
||||||
|
|
||||||
|
### Model Download Timeout Issues
|
||||||
|
|
||||||
|
Fixed critical issue where model downloads would fail with "Failed to fetch" errors on Windows. Refactored download endpoints to return immediately and continue downloads in background.
|
||||||
|
|
||||||
|
### Cross-Platform Cache Path Issues
|
||||||
|
|
||||||
|
Fixed hardcoded `~/.cache/huggingface/hub` paths that don't work on Windows. All cache paths now use `hf_constants.HF_HUB_CACHE` for proper cross-platform support.
|
||||||
|
|
||||||
|
### Windows Process Management
|
||||||
|
|
||||||
|
- Added `/shutdown` endpoint for graceful server shutdown on Windows
|
||||||
|
- Added `gpu_type` field to health check response
|
||||||
|
|
||||||
|
## [0.1.7] - 2026-01-29
|
||||||
|
|
||||||
|
- Trim and split audio clips in Story Editor
|
||||||
|
- Auto-activation of stories in Story Editor with visible playhead
|
||||||
|
- Conditional auto-play support in AudioPlayer for better user control
|
||||||
|
- Refactored audio loading across HistoryTable, SampleList, and generation forms
|
||||||
|
- Audio now only auto-plays when explicitly intended, preventing unexpected playback
|
||||||
|
|
||||||
|
## [0.1.6] - 2026-01-29
|
||||||
|
|
||||||
|
### Introducing Stories
|
||||||
|
|
||||||
|
A full voice editor for composing podcasts and generated conversations.
|
||||||
|
|
||||||
|
- **Stories Editor** — Create multi-voice narratives, podcasts, or conversations with a timeline-based editor
|
||||||
|
- Compose tracks with different voices
|
||||||
|
- Edit and arrange audio segments inline
|
||||||
|
- Build generated conversations with multiple participants
|
||||||
|
- **Improved Voice Generation UI** — Auto-resizing input, default voice selection, better layout
|
||||||
|
- **Track Editor Integration** — Inline track editing within story items
|
||||||
|
|
||||||
|
## [0.1.5] - 2026-01-28
|
||||||
|
|
||||||
|
Fixed recording length limit at 0:29 to auto stop instead of passing the limit and getting an error, which would cause users to lose their recording.
|
||||||
|
|
||||||
|
## [0.1.4] - 2026-01-28
|
||||||
|
|
||||||
|
- Audio channel management system
|
||||||
|
- Native audio playback handling in AudioPlayer component
|
||||||
|
- Refactored ConnectionForm and Checkbox components
|
||||||
|
- Improved layout consistency and responsiveness
|
||||||
|
- Added safe area constants for better responsive design
|
||||||
|
|
||||||
|
## [0.1.3] - 2026-01-27
|
||||||
|
|
||||||
|
- Improved the generate textbox
|
||||||
|
- Maybe fixed Windows autoupdate restarting entire computer
|
||||||
|
|
||||||
|
## [0.1.2] - 2026-01-27
|
||||||
|
|
||||||
|
### Audio Capture & Format Conversion
|
||||||
|
|
||||||
|
- Added audio format conversion util
|
||||||
|
- Enhanced system audio capture on macOS and Windows
|
||||||
|
- Improved audio recording hooks
|
||||||
|
- Added audio input entitlement for macOS
|
||||||
|
- Added audio capture tests
|
||||||
|
|
||||||
|
### Update System
|
||||||
|
|
||||||
|
- Enhanced auto-updater functionality and update status display
|
||||||
|
|
||||||
|
## [0.1.1] - 2026-01-27
|
||||||
|
|
||||||
|
### Platform Support
|
||||||
|
|
||||||
|
- **macOS Audio Capture** — Native audio capture support for sample creation
|
||||||
|
- **Windows Audio Capture** — WASAPI implementation with improved thread safety
|
||||||
|
- **Linux Support** — Temporarily removed builds due to runner disk space constraints
|
||||||
|
|
||||||
|
### Audio Features
|
||||||
|
|
||||||
|
- Play/pause for audio samples across all components
|
||||||
|
- Three new sample components: Recording, System capture, Upload with drag-and-drop
|
||||||
|
- Audio validation, error handling, and consistent cleanup
|
||||||
|
|
||||||
|
### Voice Profile Management
|
||||||
|
|
||||||
|
- Profile import with file size validation (100MB limit)
|
||||||
|
- Enhanced profile form with new audio sample components
|
||||||
|
- Drag-and-drop support for audio file uploads
|
||||||
|
|
||||||
|
### Server Management
|
||||||
|
|
||||||
|
- Changed default URL from `localhost:8000` to `127.0.0.1:17493`
|
||||||
|
- Server reuse logic, "keep server running" preference, orphaned process handling
|
||||||
|
|
||||||
|
### Build & Release
|
||||||
|
|
||||||
|
- Added `.bumpversion.cfg` for automated version management
|
||||||
|
- Enhanced icon generation script for multi-size Windows icons
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
|
||||||
|
- Fixed date formatting for timezone-less date strings
|
||||||
|
- Fixed getLatestRelease file filtering
|
||||||
|
- Improved audio duration metadata on Windows
|
||||||
|
|
||||||
|
## [0.1.0] - 2026-01-27
|
||||||
|
|
||||||
|
The first public release of Voicebox — an open-source voice synthesis studio powered by Qwen3-TTS.
|
||||||
|
|
||||||
|
### Voice Cloning with Qwen3-TTS
|
||||||
|
|
||||||
|
- Automatic model download from HuggingFace
|
||||||
|
- Multiple model sizes (1.7B and 0.6B)
|
||||||
|
- Voice prompt caching for instant regeneration
|
||||||
|
- English and Chinese support
|
||||||
|
|
||||||
|
### Voice Profile Management
|
||||||
|
|
||||||
|
- Create profiles from audio files or record directly in the app
|
||||||
|
- Multiple samples per profile for higher quality cloning
|
||||||
|
- Import/Export profiles
|
||||||
|
- Automatic transcription via Whisper
|
||||||
|
|
||||||
|
### Speech Generation
|
||||||
|
|
||||||
|
- Simple text-to-speech with profile selection
|
||||||
|
- Seed control for reproducible generations
|
||||||
|
- Long-form support up to 5,000 characters
|
||||||
|
|
||||||
|
### Generation History
|
||||||
|
|
||||||
|
- Full history with metadata
|
||||||
|
- Search by text content
|
||||||
|
- Inline playback and download
|
||||||
|
|
||||||
|
### Flexible Deployment
|
||||||
|
|
||||||
|
- Local mode with bundled backend
|
||||||
|
- Remote mode for GPU servers on your network
|
||||||
|
- One-click server setup
|
||||||
|
|
||||||
|
### Desktop Experience
|
||||||
|
|
||||||
|
- Built with Tauri v2 (Rust) — native performance, not Electron
|
||||||
|
- Cross-platform: macOS and Windows
|
||||||
|
- No Python installation required
|
||||||
|
|
||||||
|
### Tech Stack
|
||||||
|
|
||||||
|
Tauri v2, React, TypeScript, Tailwind CSS, FastAPI, Qwen3-TTS, Whisper, SQLite
|
||||||
|
|
||||||
|
[Unreleased]: https://github.com/jamiepine/voicebox/compare/v0.2.3...HEAD
|
||||||
|
[0.2.3]: https://github.com/jamiepine/voicebox/compare/v0.2.2...v0.2.3
|
||||||
|
[0.2.2]: https://github.com/jamiepine/voicebox/compare/v0.2.1...v0.2.2
|
||||||
|
[0.2.1]: https://github.com/jamiepine/voicebox/compare/v0.1.13...v0.2.1
|
||||||
|
[0.1.13]: https://github.com/jamiepine/voicebox/compare/v0.1.12...v0.1.13
|
||||||
|
[0.1.12]: https://github.com/jamiepine/voicebox/compare/v0.1.11...v0.1.12
|
||||||
|
[0.1.11]: https://github.com/jamiepine/voicebox/compare/v0.1.10...v0.1.11
|
||||||
|
[0.1.10]: https://github.com/jamiepine/voicebox/compare/v0.1.9...v0.1.10
|
||||||
|
[0.1.9]: https://github.com/jamiepine/voicebox/compare/v0.1.8...v0.1.9
|
||||||
|
[0.1.8]: https://github.com/jamiepine/voicebox/compare/v0.1.7...v0.1.8
|
||||||
|
[0.1.7]: https://github.com/jamiepine/voicebox/compare/v0.1.6...v0.1.7
|
||||||
|
[0.1.6]: https://github.com/jamiepine/voicebox/compare/v0.1.5...v0.1.6
|
||||||
|
[0.1.5]: https://github.com/jamiepine/voicebox/compare/v0.1.4...v0.1.5
|
||||||
|
[0.1.4]: https://github.com/jamiepine/voicebox/compare/v0.1.3...v0.1.4
|
||||||
|
[0.1.3]: https://github.com/jamiepine/voicebox/compare/v0.1.2...v0.1.3
|
||||||
|
[0.1.2]: https://github.com/jamiepine/voicebox/compare/v0.1.1...v0.1.2
|
||||||
|
[0.1.1]: https://github.com/jamiepine/voicebox/compare/v0.1.0...v0.1.1
|
||||||
[0.1.0]: https://github.com/jamiepine/voicebox/releases/tag/v0.1.0
|
[0.1.0]: https://github.com/jamiepine/voicebox/releases/tag/v0.1.0
|
||||||
|
|||||||
@@ -31,8 +31,12 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
|||||||
build-essential \
|
build-essential \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
RUN pip install --no-cache-dir --upgrade pip
|
||||||
|
|
||||||
COPY backend/requirements.txt .
|
COPY backend/requirements.txt .
|
||||||
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt
|
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt
|
||||||
|
RUN pip install --no-cache-dir --prefix=/install --no-deps chatterbox-tts
|
||||||
|
RUN pip install --no-cache-dir --prefix=/install --no-deps hume-tada
|
||||||
RUN pip install --no-cache-dir --prefix=/install \
|
RUN pip install --no-cache-dir --prefix=/install \
|
||||||
git+https://github.com/QwenLM/Qwen3-TTS.git
|
git+https://github.com/QwenLM/Qwen3-TTS.git
|
||||||
|
|
||||||
|
|||||||
@@ -1,58 +0,0 @@
|
|||||||
# Voicebox Offline Mode Fix
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
Voicebox crashes when generating speech if HuggingFace is unreachable, even when models are fully cached locally.
|
|
||||||
|
|
||||||
**Root Cause:**
|
|
||||||
- Voicebox downloads `mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16` (MLX optimized version)
|
|
||||||
- But `mlx_audio.tts.load()` tries to fetch `config.json` from original repo `Qwen/Qwen3-TTS-12Hz-1.7B-Base`
|
|
||||||
- This network request fails → server crashes with `RemoteDisconnected`
|
|
||||||
|
|
||||||
**Related Issues:**
|
|
||||||
- Issue #150: "Internet connection required, even though models are downloaded?"
|
|
||||||
- Issue #151: "API Stability Issues: Model Loading Hangs and Server Crashes"
|
|
||||||
|
|
||||||
## Solution
|
|
||||||
Two-part fix:
|
|
||||||
|
|
||||||
### 1. Monkey-patch huggingface_hub (`backend/utils/hf_offline_patch.py`)
|
|
||||||
- Intercepts cache lookup functions
|
|
||||||
- Forces offline mode early (before mlx_audio imports)
|
|
||||||
- Adds debug logging for cache hits/misses
|
|
||||||
|
|
||||||
### 2. Symlink original repo to MLX version (`ensure_original_qwen_config_cached()`)
|
|
||||||
- When original `Qwen/Qwen3-TTS-12Hz-1.7B-Base` cache doesn't exist
|
|
||||||
- But MLX `mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16` does exist
|
|
||||||
- Creates a symlink so cache lookups succeed
|
|
||||||
|
|
||||||
## Files Changed
|
|
||||||
- `backend/backends/mlx_backend.py` - Added patch imports at top
|
|
||||||
- `backend/utils/hf_offline_patch.py` - New patch module
|
|
||||||
|
|
||||||
## Testing
|
|
||||||
To test this fix:
|
|
||||||
1. Build Voicebox from source: `just build`
|
|
||||||
2. Disconnect from internet
|
|
||||||
3. Try generating speech
|
|
||||||
4. Should work without network requests
|
|
||||||
|
|
||||||
## Build Instructions
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Install dependencies
|
|
||||||
just setup
|
|
||||||
|
|
||||||
# Build the app
|
|
||||||
just build
|
|
||||||
|
|
||||||
# Or build just the server
|
|
||||||
just build-server
|
|
||||||
```
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
- The patch is applied automatically when `mlx_backend.py` is imported
|
|
||||||
- Set `VOICEBOX_OFFLINE_PATCH=0` to disable the patch
|
|
||||||
- The symlink approach works because the config.json is compatible between versions
|
|
||||||
|
|
||||||
---
|
|
||||||
*Patch contributed by community*
|
|
||||||
@@ -27,10 +27,10 @@
|
|||||||
|
|
||||||
<p align="center">
|
<p align="center">
|
||||||
<a href="https://voicebox.sh">voicebox.sh</a> •
|
<a href="https://voicebox.sh">voicebox.sh</a> •
|
||||||
|
<a href="https://docs.voicebox.sh">Docs</a> •
|
||||||
<a href="#download">Download</a> •
|
<a href="#download">Download</a> •
|
||||||
<a href="#features">Features</a> •
|
<a href="#features">Features</a> •
|
||||||
<a href="#api">API</a> •
|
<a href="#api">API</a>
|
||||||
<a href="#roadmap">Roadmap</a>
|
|
||||||
</p>
|
</p>
|
||||||
|
|
||||||
<br/>
|
<br/>
|
||||||
@@ -59,10 +59,10 @@
|
|||||||
|
|
||||||
## What is Voicebox?
|
## What is Voicebox?
|
||||||
|
|
||||||
Voicebox is a **local-first voice cloning studio** — a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
|
Voicebox is a **local-first voice cloning studio** — a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 5 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
|
||||||
|
|
||||||
- **Complete privacy** — models and voice data stay on your machine
|
- **Complete privacy** — models and voice data stay on your machine
|
||||||
- **4 TTS engines** — Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo
|
- **5 TTS engines** — Qwen3-TTS, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo, and HumeAI TADA
|
||||||
- **23 languages** — from English to Arabic, Japanese, Hindi, Swahili, and more
|
- **23 languages** — from English to Arabic, Japanese, Hindi, Swahili, and more
|
||||||
- **Post-processing effects** — pitch shift, reverb, delay, chorus, compression, and filters
|
- **Post-processing effects** — pitch shift, reverb, delay, chorus, compression, and filters
|
||||||
- **Expressive speech** — paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo
|
- **Expressive speech** — paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo
|
||||||
@@ -76,12 +76,12 @@ Voicebox is a **local-first voice cloning studio** — a free and open-source al
|
|||||||
|
|
||||||
## Download
|
## Download
|
||||||
|
|
||||||
| Platform | Download |
|
| Platform | Download |
|
||||||
|----------|----------|
|
| --------------------- | ------------------------------------------------------ |
|
||||||
| macOS (Apple Silicon) | [Download DMG](https://voicebox.sh/download/mac-arm) |
|
| macOS (Apple Silicon) | [Download DMG](https://voicebox.sh/download/mac-arm) |
|
||||||
| macOS (Intel) | [Download DMG](https://voicebox.sh/download/mac-intel) |
|
| macOS (Intel) | [Download DMG](https://voicebox.sh/download/mac-intel) |
|
||||||
| Windows | [Download MSI](https://voicebox.sh/download/windows) |
|
| Windows | [Download MSI](https://voicebox.sh/download/windows) |
|
||||||
| Docker | `docker compose up` |
|
| Docker | `docker compose up` |
|
||||||
|
|
||||||
> **[View all binaries →](https://github.com/jamiepine/voicebox/releases/latest)**
|
> **[View all binaries →](https://github.com/jamiepine/voicebox/releases/latest)**
|
||||||
|
|
||||||
@@ -93,14 +93,15 @@ Voicebox is a **local-first voice cloning studio** — a free and open-source al
|
|||||||
|
|
||||||
### Multi-Engine Voice Cloning
|
### Multi-Engine Voice Cloning
|
||||||
|
|
||||||
Four TTS engines with different strengths, switchable per-generation:
|
Five TTS engines with different strengths, switchable per-generation:
|
||||||
|
|
||||||
| Engine | Languages | Strengths |
|
| Engine | Languages | Strengths |
|
||||||
|--------|-----------|-----------|
|
| --------------------------- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| **Qwen3-TTS** (0.6B / 1.7B) | 10 | High-quality multilingual cloning, delivery instructions ("speak slowly", "whisper") |
|
| **Qwen3-TTS** (0.6B / 1.7B) | 10 | High-quality multilingual cloning, delivery instructions ("speak slowly", "whisper") |
|
||||||
| **LuxTTS** | English | Lightweight (~1GB VRAM), 48kHz output, 150x realtime on CPU |
|
| **LuxTTS** | English | Lightweight (~1GB VRAM), 48kHz output, 150x realtime on CPU |
|
||||||
| **Chatterbox Multilingual** | 23 | Broadest language coverage — Arabic, Danish, Finnish, Greek, Hebrew, Hindi, Malay, Norwegian, Polish, Swahili, Swedish, Turkish and more |
|
| **Chatterbox Multilingual** | 23 | Broadest language coverage — Arabic, Danish, Finnish, Greek, Hebrew, Hindi, Malay, Norwegian, Polish, Swahili, Swedish, Turkish and more |
|
||||||
| **Chatterbox Turbo** | English | Fast 350M model with paralinguistic emotion/sound tags |
|
| **Chatterbox Turbo** | English | Fast 350M model with paralinguistic emotion/sound tags |
|
||||||
|
| **TADA** (1B / 3B) | 10 | HumeAI speech-language model — 700s+ coherent audio, text-acoustic dual alignment |
|
||||||
|
|
||||||
### Emotions & Paralinguistic Tags
|
### Emotions & Paralinguistic Tags
|
||||||
|
|
||||||
@@ -112,16 +113,16 @@ Type `/` in the text input to insert expressive tags that the model synthesizes
|
|||||||
|
|
||||||
8 audio effects powered by Spotify's `pedalboard` library. Apply after generation, preview in real time, build reusable presets.
|
8 audio effects powered by Spotify's `pedalboard` library. Apply after generation, preview in real time, build reusable presets.
|
||||||
|
|
||||||
| Effect | Description |
|
| Effect | Description |
|
||||||
|--------|-------------|
|
| ---------------- | --------------------------------------------- |
|
||||||
| Pitch Shift | Up or down by up to 12 semitones |
|
| Pitch Shift | Up or down by up to 12 semitones |
|
||||||
| Reverb | Configurable room size, damping, wet/dry mix |
|
| Reverb | Configurable room size, damping, wet/dry mix |
|
||||||
| Delay | Echo with adjustable time, feedback, and mix |
|
| Delay | Echo with adjustable time, feedback, and mix |
|
||||||
| Chorus / Flanger | Modulated delay for metallic or lush textures |
|
| Chorus / Flanger | Modulated delay for metallic or lush textures |
|
||||||
| Compressor | Dynamic range compression |
|
| Compressor | Dynamic range compression |
|
||||||
| Gain | Volume adjustment (-40 to +40 dB) |
|
| Gain | Volume adjustment (-40 to +40 dB) |
|
||||||
| High-Pass Filter | Remove low frequencies |
|
| High-Pass Filter | Remove low frequencies |
|
||||||
| Low-Pass Filter | Remove high frequencies |
|
| Low-Pass Filter | Remove high frequencies |
|
||||||
|
|
||||||
Ships with 4 built-in presets (Robotic, Radio, Echo Chamber, Deep Voice) and supports custom presets. Effects can be assigned per-profile as defaults.
|
Ships with 4 built-in presets (Robotic, Radio, Echo Chamber, Deep Voice) and supports custom presets. Effects can be assigned per-profile as defaults.
|
||||||
|
|
||||||
@@ -186,14 +187,14 @@ Multi-voice timeline editor for conversations, podcasts, and narratives.
|
|||||||
|
|
||||||
### GPU Support
|
### GPU Support
|
||||||
|
|
||||||
| Platform | Backend | Notes |
|
| Platform | Backend | Notes |
|
||||||
|----------|---------|-------|
|
| ------------------------ | -------------- | ---------------------------------------------- |
|
||||||
| macOS (Apple Silicon) | MLX (Metal) | 4-5x faster via Neural Engine |
|
| macOS (Apple Silicon) | MLX (Metal) | 4-5x faster via Neural Engine |
|
||||||
| Windows / Linux (NVIDIA) | PyTorch (CUDA) | Auto-downloads CUDA binary from within the app |
|
| Windows / Linux (NVIDIA) | PyTorch (CUDA) | Auto-downloads CUDA binary from within the app |
|
||||||
| Linux (AMD) | PyTorch (ROCm) | Auto-configures HSA_OVERRIDE_GFX_VERSION |
|
| Linux (AMD) | PyTorch (ROCm) | Auto-configures HSA_OVERRIDE_GFX_VERSION |
|
||||||
| Windows (any GPU) | DirectML | Universal Windows GPU support |
|
| Windows (any GPU) | DirectML | Universal Windows GPU support |
|
||||||
| Intel Arc | IPEX/XPU | Intel discrete GPU acceleration |
|
| Intel Arc | IPEX/XPU | Intel discrete GPU acceleration |
|
||||||
| Any | CPU | Works everywhere, just slower |
|
| Any | CPU | Works everywhere, just slower |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -224,30 +225,30 @@ Full API documentation available at `http://localhost:17493/docs`.
|
|||||||
|
|
||||||
## Tech Stack
|
## Tech Stack
|
||||||
|
|
||||||
| Layer | Technology |
|
| Layer | Technology |
|
||||||
|-------|------------|
|
| ------------- | ------------------------------------------------- |
|
||||||
| Desktop App | Tauri (Rust) |
|
| Desktop App | Tauri (Rust) |
|
||||||
| Frontend | React, TypeScript, Tailwind CSS |
|
| Frontend | React, TypeScript, Tailwind CSS |
|
||||||
| State | Zustand, React Query |
|
| State | Zustand, React Query |
|
||||||
| Backend | FastAPI (Python) |
|
| Backend | FastAPI (Python) |
|
||||||
| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo |
|
| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo, TADA |
|
||||||
| Effects | Pedalboard (Spotify) |
|
| Effects | Pedalboard (Spotify) |
|
||||||
| Transcription | Whisper / Whisper Turbo (PyTorch or MLX) |
|
| Transcription | Whisper / Whisper Turbo (PyTorch or MLX) |
|
||||||
| Inference | MLX (Apple Silicon) / PyTorch (CUDA/ROCm/XPU/CPU) |
|
| Inference | MLX (Apple Silicon) / PyTorch (CUDA/ROCm/XPU/CPU) |
|
||||||
| Database | SQLite |
|
| Database | SQLite |
|
||||||
| Audio | WaveSurfer.js, librosa |
|
| Audio | WaveSurfer.js, librosa |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Roadmap
|
## Roadmap
|
||||||
|
|
||||||
| Feature | Description |
|
| Feature | Description |
|
||||||
|---------|-------------|
|
| ----------------------- | ---------------------------------------------- |
|
||||||
| **Real-time Streaming** | Stream audio as it generates, word by word |
|
| **Real-time Streaming** | Stream audio as it generates, word by word |
|
||||||
| **Voice Design** | Create new voices from text descriptions |
|
| **Voice Design** | Create new voices from text descriptions |
|
||||||
| **More Models** | XTTS, Bark, and other open-source voice models |
|
| **More Models** | XTTS, Bark, and other open-source voice models |
|
||||||
| **Plugin Architecture** | Extend with custom models and effects |
|
| **Plugin Architecture** | Extend with custom models and effects |
|
||||||
| **Mobile Companion** | Control Voicebox from your phone |
|
| **Mobile Companion** | Control Voicebox from your phone |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -276,6 +277,12 @@ just build # Build CPU server binary + Tauri app
|
|||||||
just build-local # (Windows) Build CPU + CUDA server binaries + Tauri app
|
just build-local # (Windows) Build CPU + CUDA server binaries + Tauri app
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Adding New Voice Models
|
||||||
|
|
||||||
|
The multi-engine architecture makes adding new TTS engines straightforward. A [step-by-step guide](docs/content/docs/developer/tts-engines.mdx) covers the full process: dependency research, backend protocol implementation, frontend wiring, and PyInstaller bundling.
|
||||||
|
|
||||||
|
The guide is optimized for AI coding agents. An [agent skill](.agents/skills/add-tts-engine/SKILL.md) can pick up a model name and handle the entire integration autonomously — you just test the build locally.
|
||||||
|
|
||||||
### Project Structure
|
### Project Structure
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@voicebox/app",
|
"name": "@voicebox/app",
|
||||||
"version": "0.2.3",
|
"version": "0.3.1",
|
||||||
"private": true,
|
"private": true,
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
|
|||||||
@@ -0,0 +1,23 @@
|
|||||||
|
import { readFileSync } from 'node:fs';
|
||||||
|
import path from 'node:path';
|
||||||
|
import type { Plugin } from 'vite';
|
||||||
|
|
||||||
|
/** Vite plugin that exposes CHANGELOG.md as `virtual:changelog`. */
|
||||||
|
export function changelogPlugin(repoRoot: string): Plugin {
|
||||||
|
const virtualId = 'virtual:changelog';
|
||||||
|
const resolvedId = '\0' + virtualId;
|
||||||
|
const changelogPath = path.resolve(repoRoot, 'CHANGELOG.md');
|
||||||
|
|
||||||
|
return {
|
||||||
|
name: 'changelog',
|
||||||
|
resolveId(id) {
|
||||||
|
if (id === virtualId) return resolvedId;
|
||||||
|
},
|
||||||
|
load(id) {
|
||||||
|
if (id === resolvedId) {
|
||||||
|
const raw = readFileSync(changelogPath, 'utf-8');
|
||||||
|
return `export default ${JSON.stringify(raw)};`;
|
||||||
|
}
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -8,6 +8,7 @@ import { TOP_SAFE_AREA_PADDING } from '@/lib/constants/ui';
|
|||||||
import { cn } from '@/lib/utils/cn';
|
import { cn } from '@/lib/utils/cn';
|
||||||
import { usePlatform } from '@/platform/PlatformContext';
|
import { usePlatform } from '@/platform/PlatformContext';
|
||||||
import { router } from '@/router';
|
import { router } from '@/router';
|
||||||
|
import { useLogStore } from '@/stores/logStore';
|
||||||
import { useServerStore } from '@/stores/serverStore';
|
import { useServerStore } from '@/stores/serverStore';
|
||||||
|
|
||||||
const LOADING_MESSAGES = [
|
const LOADING_MESSAGES = [
|
||||||
@@ -63,6 +64,14 @@ function App() {
|
|||||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||||
}, [platform.lifecycle]);
|
}, [platform.lifecycle]);
|
||||||
|
|
||||||
|
// Subscribe to server logs
|
||||||
|
useEffect(() => {
|
||||||
|
const unsubscribe = platform.lifecycle.subscribeToServerLogs((entry) => {
|
||||||
|
useLogStore.getState().addEntry(entry);
|
||||||
|
});
|
||||||
|
return unsubscribe;
|
||||||
|
}, [platform.lifecycle]);
|
||||||
|
|
||||||
// Setup window close handler and auto-start server when running in Tauri (production only)
|
// Setup window close handler and auto-start server when running in Tauri (production only)
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
if (!platform.metadata.isTauri) {
|
if (!platform.metadata.isTauri) {
|
||||||
|
|||||||
@@ -17,7 +17,6 @@ export function AudioPlayer() {
|
|||||||
audioUrl,
|
audioUrl,
|
||||||
audioId,
|
audioId,
|
||||||
profileId,
|
profileId,
|
||||||
title,
|
|
||||||
isPlaying,
|
isPlaying,
|
||||||
currentTime,
|
currentTime,
|
||||||
duration,
|
duration,
|
||||||
@@ -63,7 +62,7 @@ export function AudioPlayer() {
|
|||||||
);
|
);
|
||||||
|
|
||||||
return shouldUseNative;
|
return shouldUseNative;
|
||||||
}, [profileChannels, channels, profileId]);
|
}, [profileChannels, channels, platform.metadata.isTauri]);
|
||||||
|
|
||||||
const waveformRef = useRef<HTMLDivElement>(null);
|
const waveformRef = useRef<HTMLDivElement>(null);
|
||||||
const wavesurferRef = useRef<WaveSurfer | null>(null);
|
const wavesurferRef = useRef<WaveSurfer | null>(null);
|
||||||
@@ -73,31 +72,21 @@ export function AudioPlayer() {
|
|||||||
const isUsingNativePlaybackRef = useRef(false);
|
const isUsingNativePlaybackRef = useRef(false);
|
||||||
const [isLoading, setIsLoading] = useState(false);
|
const [isLoading, setIsLoading] = useState(false);
|
||||||
const [error, setError] = useState<string | null>(null);
|
const [error, setError] = useState<string | null>(null);
|
||||||
|
const [wsReady, setWsReady] = useState(false);
|
||||||
|
|
||||||
// Initialize WaveSurfer (only when audioUrl exists and container is ready)
|
// Create WaveSurfer once when the player becomes visible (audioUrl is set).
|
||||||
|
// This instance is reused for all subsequent audio loads - never destroyed until unmount.
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
// Don't initialize if no audioUrl or already initialized
|
if (!audioUrl) return;
|
||||||
if (!audioUrl) {
|
if (wavesurferRef.current) return; // already created
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (wavesurferRef.current) {
|
|
||||||
debug.log('WaveSurfer already initialized, skipping');
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
debug.log('Creating NEW WaveSurfer instance');
|
|
||||||
|
|
||||||
// Wait for container to be properly rendered
|
|
||||||
const initWaveSurfer = () => {
|
const initWaveSurfer = () => {
|
||||||
const container = waveformRef.current;
|
const container = waveformRef.current;
|
||||||
if (!container) {
|
if (!container) {
|
||||||
// Container not ready yet, retry
|
|
||||||
setTimeout(initWaveSurfer, 50);
|
setTimeout(initWaveSurfer, 50);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Check if container has dimensions and is visible
|
|
||||||
const rect = container.getBoundingClientRect();
|
const rect = container.getBoundingClientRect();
|
||||||
const style = window.getComputedStyle(container);
|
const style = window.getComputedStyle(container);
|
||||||
const isVisible =
|
const isVisible =
|
||||||
@@ -107,412 +96,221 @@ export function AudioPlayer() {
|
|||||||
style.visibility !== 'hidden';
|
style.visibility !== 'hidden';
|
||||||
|
|
||||||
if (!isVisible) {
|
if (!isVisible) {
|
||||||
// Retry after a short delay
|
|
||||||
setTimeout(initWaveSurfer, 50);
|
setTimeout(initWaveSurfer, 50);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
debug.log('Initializing WaveSurfer...', {
|
debug.log('Creating WaveSurfer instance', {
|
||||||
container,
|
|
||||||
width: rect.width,
|
width: rect.width,
|
||||||
height: rect.height,
|
height: rect.height,
|
||||||
});
|
});
|
||||||
|
|
||||||
try {
|
try {
|
||||||
// Get computed CSS variable values
|
|
||||||
const root = document.documentElement;
|
const root = document.documentElement;
|
||||||
const getCSSVar = (varName: string) => {
|
const getCSSVar = (varName: string) => {
|
||||||
const value = getComputedStyle(root).getPropertyValue(varName).trim();
|
const value = getComputedStyle(root).getPropertyValue(varName).trim();
|
||||||
return value ? `hsl(${value})` : '';
|
return value ? `hsl(${value})` : '';
|
||||||
};
|
};
|
||||||
|
|
||||||
const waveColor = getCSSVar('--muted');
|
|
||||||
const progressColor = getCSSVar('--accent');
|
|
||||||
const cursorColor = getCSSVar('--accent');
|
|
||||||
|
|
||||||
const wavesurfer = WaveSurfer.create({
|
const wavesurfer = WaveSurfer.create({
|
||||||
container: container,
|
container,
|
||||||
waveColor: waveColor,
|
waveColor: getCSSVar('--muted'),
|
||||||
progressColor: progressColor,
|
progressColor: getCSSVar('--accent'),
|
||||||
cursorColor: cursorColor,
|
cursorColor: getCSSVar('--accent'),
|
||||||
|
cursorWidth: 3,
|
||||||
barWidth: 2,
|
barWidth: 2,
|
||||||
barRadius: 2,
|
barRadius: 2,
|
||||||
height: 80,
|
height: 80,
|
||||||
normalize: true,
|
normalize: true,
|
||||||
// Use MediaElement backend (default). Unlike the WebAudio backend,
|
interact: true,
|
||||||
// MediaElement uses a standard <audio> element for playback which
|
dragToSeek: { debounceTime: 0 },
|
||||||
// benefits from the browser/webview's built-in audio session recovery.
|
mediaControls: false,
|
||||||
// This prevents audio loss when another app steals audio output or
|
backend: 'WebAudio',
|
||||||
// the system audio session is interrupted.
|
|
||||||
interact: true, // Enable interaction (click to seek)
|
|
||||||
mediaControls: false, // Don't show native controls
|
|
||||||
});
|
});
|
||||||
|
|
||||||
wavesurferRef.current = wavesurfer;
|
// Wire up event handlers (these persist for the lifetime of the instance)
|
||||||
debug.log('WaveSurfer created successfully');
|
wavesurfer.on('timeupdate', (time) => {
|
||||||
} catch (error) {
|
const dur = usePlayerStore.getState().duration;
|
||||||
debug.error('Failed to create WaveSurfer:', error);
|
if (dur > 0 && time >= dur) {
|
||||||
setError(
|
setCurrentTime(dur);
|
||||||
`Failed to initialize waveform: ${error instanceof Error ? error.message : String(error)}`,
|
const loop = usePlayerStore.getState().isLooping;
|
||||||
);
|
if (loop) {
|
||||||
return;
|
wavesurfer.seekTo(0);
|
||||||
}
|
wavesurfer.play().catch((err) => debug.error('Loop play failed:', err));
|
||||||
|
} else {
|
||||||
|
wavesurfer.pause();
|
||||||
|
setIsPlaying(false);
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
setCurrentTime(time);
|
||||||
|
});
|
||||||
|
|
||||||
const wavesurfer = wavesurferRef.current;
|
wavesurfer.on('ready', () => {
|
||||||
if (!wavesurfer) return;
|
const dur = wavesurfer.getDuration();
|
||||||
|
setDuration(dur);
|
||||||
|
loadingRef.current = false;
|
||||||
|
setIsLoading(false);
|
||||||
|
setError(null);
|
||||||
|
debug.log('Audio ready, duration:', dur);
|
||||||
|
|
||||||
// Update store when time changes, stop if past duration
|
wavesurfer.setVolume(usePlayerStore.getState().volume);
|
||||||
wavesurfer.on('timeupdate', (time) => {
|
wavesurfer.setMuted(false);
|
||||||
const dur = usePlayerStore.getState().duration;
|
|
||||||
if (dur > 0 && time >= dur) {
|
// Auto-play if the flag is set (story mode advance or explicit play)
|
||||||
setCurrentTime(dur);
|
const shouldAutoPlayNow = usePlayerStore.getState().shouldAutoPlay;
|
||||||
|
if (shouldAutoPlayNow) {
|
||||||
|
usePlayerStore.getState().clearAutoPlayFlag();
|
||||||
|
wavesurfer.play().catch((err) => {
|
||||||
|
debug.error('Failed to autoplay:', err);
|
||||||
|
});
|
||||||
|
} else {
|
||||||
|
debug.log('Skipping auto-play - shouldAutoPlay is false');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
wavesurfer.on('play', () => setIsPlaying(true));
|
||||||
|
wavesurfer.on('pause', () => {
|
||||||
|
setIsPlaying(false);
|
||||||
|
setCurrentTime(wavesurfer.getCurrentTime());
|
||||||
|
});
|
||||||
|
|
||||||
|
wavesurfer.on('seeking', (time) => setCurrentTime(time));
|
||||||
|
|
||||||
|
// Mute audio during drag-to-seek to prevent popping from the WebAudio
|
||||||
|
// backend's hard stop/start cycle on each seek. Unmute with a short
|
||||||
|
// fade-in when the drag ends.
|
||||||
|
const seekMedia = wavesurfer.getMediaElement() as any;
|
||||||
|
const seekGain: GainNode | null = seekMedia?.getGainNode?.() ?? null;
|
||||||
|
if (seekGain) {
|
||||||
|
const ctx = seekGain.context as AudioContext;
|
||||||
|
wavesurfer.on('dragstart', () => {
|
||||||
|
seekGain.gain.cancelScheduledValues(ctx.currentTime);
|
||||||
|
seekGain.gain.setTargetAtTime(0, ctx.currentTime, 0.002);
|
||||||
|
});
|
||||||
|
wavesurfer.on('dragend', () => {
|
||||||
|
seekGain.gain.cancelScheduledValues(ctx.currentTime);
|
||||||
|
seekGain.gain.setTargetAtTime(1, ctx.currentTime, 0.01);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
wavesurfer.on('finish', () => {
|
||||||
const loop = usePlayerStore.getState().isLooping;
|
const loop = usePlayerStore.getState().isLooping;
|
||||||
if (loop) {
|
if (loop) {
|
||||||
wavesurfer.seekTo(0);
|
wavesurfer.seekTo(0);
|
||||||
wavesurfer.play();
|
wavesurfer.play().catch((err) => debug.error('Loop play failed:', err));
|
||||||
} else {
|
} else {
|
||||||
wavesurfer.pause();
|
|
||||||
setIsPlaying(false);
|
setIsPlaying(false);
|
||||||
|
const onFinish = usePlayerStore.getState().onFinish;
|
||||||
|
if (onFinish) onFinish();
|
||||||
}
|
}
|
||||||
return;
|
|
||||||
}
|
|
||||||
setCurrentTime(time);
|
|
||||||
});
|
|
||||||
|
|
||||||
// Update store when duration is loaded
|
|
||||||
wavesurfer.on('ready', async () => {
|
|
||||||
const dur = wavesurfer.getDuration();
|
|
||||||
setDuration(dur);
|
|
||||||
loadingRef.current = false;
|
|
||||||
setIsLoading(false);
|
|
||||||
setError(null);
|
|
||||||
debug.log('Audio ready, duration:', dur);
|
|
||||||
debug.log('Waveform should be visible now');
|
|
||||||
|
|
||||||
// Ensure volume is set
|
|
||||||
const currentVolume = usePlayerStore.getState().volume;
|
|
||||||
wavesurfer.setVolume(currentVolume);
|
|
||||||
|
|
||||||
// Auto-play when ready - check if we should use native playback
|
|
||||||
// Get current values from the store and queries at runtime (not captured closure values)
|
|
||||||
const currentAudioUrl = usePlayerStore.getState().audioUrl;
|
|
||||||
const currentProfileId = usePlayerStore.getState().profileId;
|
|
||||||
|
|
||||||
debug.log('Auto-play check - capturing runtime values...');
|
|
||||||
|
|
||||||
// Fetch profile channels at runtime (not using captured value)
|
|
||||||
let runtimeProfileChannels = null;
|
|
||||||
let runtimeChannels = null;
|
|
||||||
|
|
||||||
if (platform.metadata.isTauri && currentProfileId) {
|
|
||||||
try {
|
|
||||||
runtimeProfileChannels = await apiClient.getProfileChannels(currentProfileId);
|
|
||||||
debug.log('Runtime profileChannels:', runtimeProfileChannels);
|
|
||||||
|
|
||||||
if (runtimeProfileChannels && runtimeProfileChannels.channel_ids.length > 0) {
|
|
||||||
runtimeChannels = await apiClient.listChannels();
|
|
||||||
debug.log('Runtime channels:', runtimeChannels);
|
|
||||||
}
|
|
||||||
} catch (error) {
|
|
||||||
debug.error('Failed to fetch runtime channel data:', error);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
debug.log('Auto-play check:', {
|
|
||||||
isTauri: platform.metadata.isTauri,
|
|
||||||
currentAudioUrl,
|
|
||||||
currentProfileId,
|
|
||||||
hasProfileChannels: !!runtimeProfileChannels,
|
|
||||||
hasChannels: !!runtimeChannels,
|
|
||||||
});
|
});
|
||||||
|
|
||||||
if (
|
wavesurfer.on('error', (err) => {
|
||||||
platform.metadata.isTauri &&
|
debug.error('WaveSurfer error:', err);
|
||||||
currentAudioUrl &&
|
|
||||||
currentProfileId &&
|
|
||||||
runtimeProfileChannels &&
|
|
||||||
runtimeChannels
|
|
||||||
) {
|
|
||||||
debug.log('Attempting native audio playback...');
|
|
||||||
|
|
||||||
// Stop any existing native playback first
|
|
||||||
if (isUsingNativePlaybackRef.current) {
|
|
||||||
try {
|
|
||||||
platform.audio.stopPlayback();
|
|
||||||
debug.log('Stopped existing native playback before starting new one');
|
|
||||||
} catch (error) {
|
|
||||||
debug.error('Failed to stop existing playback:', error);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
// Collect all device IDs from assigned channels
|
|
||||||
const assignedChannels = runtimeChannels.filter((ch: any) =>
|
|
||||||
runtimeProfileChannels.channel_ids.includes(ch.id),
|
|
||||||
);
|
|
||||||
debug.log('Assigned channels for playback:', assignedChannels);
|
|
||||||
|
|
||||||
// Check if any assigned channel has non-default devices
|
|
||||||
const shouldUseNative = assignedChannels.some(
|
|
||||||
(ch: any) => ch.device_ids.length > 0 && !ch.is_default,
|
|
||||||
);
|
|
||||||
debug.log('Should use native playback:', shouldUseNative);
|
|
||||||
|
|
||||||
if (!shouldUseNative) {
|
|
||||||
debug.log('No custom devices assigned, using standard playback');
|
|
||||||
isUsingNativePlaybackRef.current = false;
|
|
||||||
} else {
|
|
||||||
const deviceIds = assignedChannels.flatMap((ch: any) => ch.device_ids);
|
|
||||||
debug.log('Device IDs to play to:', deviceIds);
|
|
||||||
|
|
||||||
if (deviceIds.length > 0) {
|
|
||||||
debug.log('Fetching audio data from:', currentAudioUrl);
|
|
||||||
// Fetch audio data
|
|
||||||
const response = await fetch(currentAudioUrl);
|
|
||||||
const audioData = new Uint8Array(await response.arrayBuffer());
|
|
||||||
debug.log('Audio data size:', audioData.length);
|
|
||||||
|
|
||||||
// Play via native audio
|
|
||||||
debug.log('Invoking play_audio_to_devices...');
|
|
||||||
try {
|
|
||||||
await platform.audio.playToDevices(audioData, deviceIds);
|
|
||||||
debug.log('play_audio_to_devices completed successfully');
|
|
||||||
|
|
||||||
// Mark that we're using native playback
|
|
||||||
isUsingNativePlaybackRef.current = true;
|
|
||||||
|
|
||||||
// Mute WaveSurfer's audio output — native handles the actual sound
|
|
||||||
// Keep WaveSurfer running for waveform visualization
|
|
||||||
wavesurfer.setVolume(0);
|
|
||||||
wavesurfer.setMuted(true);
|
|
||||||
|
|
||||||
// Start WaveSurfer playback for visualization (muted)
|
|
||||||
wavesurfer.play().catch((error) => {
|
|
||||||
debug.error('Failed to start WaveSurfer visualization:', error);
|
|
||||||
});
|
|
||||||
|
|
||||||
setIsPlaying(true);
|
|
||||||
debug.log('Auto-playing via native audio routing - SUCCESS');
|
|
||||||
return;
|
|
||||||
} catch (invokeError) {
|
|
||||||
debug.error('play_audio_to_devices invoke failed:', invokeError);
|
|
||||||
throw invokeError;
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
debug.log('No device IDs found, falling back to WaveSurfer');
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} catch (error) {
|
|
||||||
debug.error(
|
|
||||||
'Native playback failed during auto-play, falling back to WaveSurfer:',
|
|
||||||
error,
|
|
||||||
);
|
|
||||||
isUsingNativePlaybackRef.current = false;
|
|
||||||
// Fall through to WaveSurfer playback
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Standard playback path — ensure WaveSurfer is unmuted
|
|
||||||
if (!isUsingNativePlaybackRef.current) {
|
|
||||||
wavesurfer.setMuted(false);
|
|
||||||
wavesurfer.setVolume(usePlayerStore.getState().volume);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Only auto-play if shouldAutoPlay flag is set (user explicitly clicked to play)
|
|
||||||
const shouldAutoPlayNow = usePlayerStore.getState().shouldAutoPlay;
|
|
||||||
if (shouldAutoPlayNow) {
|
|
||||||
// Clear the flag first
|
|
||||||
usePlayerStore.getState().clearAutoPlayFlag();
|
|
||||||
|
|
||||||
// Use a small delay to ensure audio element is fully ready
|
|
||||||
setTimeout(() => {
|
|
||||||
wavesurfer.play().catch((error) => {
|
|
||||||
debug.error('Failed to autoplay:', error);
|
|
||||||
// Don't show error for autoplay failures (browser restrictions)
|
|
||||||
});
|
|
||||||
}, 100);
|
|
||||||
} else {
|
|
||||||
debug.log('Skipping auto-play - shouldAutoPlay is false');
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Handle play/pause
|
|
||||||
wavesurfer.on('play', () => {
|
|
||||||
setIsPlaying(true);
|
|
||||||
});
|
|
||||||
wavesurfer.on('pause', () => setIsPlaying(false));
|
|
||||||
wavesurfer.on('finish', () => {
|
|
||||||
// Check loop state from store
|
|
||||||
const loop = usePlayerStore.getState().isLooping;
|
|
||||||
if (loop) {
|
|
||||||
wavesurfer.seekTo(0);
|
|
||||||
wavesurfer.play();
|
|
||||||
} else {
|
|
||||||
setIsPlaying(false);
|
|
||||||
// Trigger finish callback if set
|
|
||||||
const onFinish = usePlayerStore.getState().onFinish;
|
|
||||||
if (onFinish) {
|
|
||||||
onFinish();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Handle errors
|
|
||||||
wavesurfer.on('error', (error) => {
|
|
||||||
debug.error('WaveSurfer error:', error);
|
|
||||||
setIsLoading(false);
|
|
||||||
setError(`Audio error: ${error instanceof Error ? error.message : String(error)}`);
|
|
||||||
});
|
|
||||||
|
|
||||||
// Handle loading
|
|
||||||
wavesurfer.on('loading', (percent) => {
|
|
||||||
setIsLoading(true);
|
|
||||||
if (percent === 100) {
|
|
||||||
setIsLoading(false);
|
setIsLoading(false);
|
||||||
}
|
setError(`Audio error: ${err instanceof Error ? err.message : String(err)}`);
|
||||||
});
|
});
|
||||||
|
|
||||||
// Load audio immediately if audioUrl is already set
|
wavesurfer.on('loading', (percent) => {
|
||||||
if (audioUrl) {
|
setIsLoading(true);
|
||||||
debug.log('WaveSurfer ready, loading audio:', audioUrl);
|
if (percent === 100) setIsLoading(false);
|
||||||
loadingRef.current = true;
|
});
|
||||||
setIsLoading(true);
|
|
||||||
// Stop any current playback before loading new audio
|
wavesurferRef.current = wavesurfer;
|
||||||
if (wavesurfer.isPlaying()) {
|
setWsReady(true);
|
||||||
wavesurfer.pause();
|
debug.log('WaveSurfer created successfully');
|
||||||
}
|
} catch (err) {
|
||||||
wavesurfer
|
debug.error('Failed to create WaveSurfer:', err);
|
||||||
.load(audioUrl)
|
setError(
|
||||||
.then(() => {
|
`Failed to initialize waveform: ${err instanceof Error ? err.message : String(err)}`,
|
||||||
debug.log('Audio loaded into WaveSurfer');
|
);
|
||||||
loadingRef.current = false;
|
|
||||||
})
|
|
||||||
.catch((error) => {
|
|
||||||
debug.error('Failed to load audio into WaveSurfer:', error);
|
|
||||||
loadingRef.current = false;
|
|
||||||
setIsLoading(false);
|
|
||||||
setError(
|
|
||||||
`Failed to load audio: ${error instanceof Error ? error.message : String(error)}`,
|
|
||||||
);
|
|
||||||
});
|
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
// Use double requestAnimationFrame to ensure DOM is fully rendered
|
let rafId: number;
|
||||||
let rafId1: number;
|
rafId = requestAnimationFrame(() => {
|
||||||
let rafId2: number;
|
initWaveSurfer();
|
||||||
let timeoutId: number | null = null;
|
|
||||||
|
|
||||||
rafId1 = requestAnimationFrame(() => {
|
|
||||||
rafId2 = requestAnimationFrame(() => {
|
|
||||||
// Add a small delay to ensure container is fully laid out
|
|
||||||
timeoutId = setTimeout(() => {
|
|
||||||
initWaveSurfer();
|
|
||||||
}, 10);
|
|
||||||
});
|
|
||||||
});
|
});
|
||||||
|
|
||||||
return () => {
|
return () => {
|
||||||
debug.log('Cleaning up WaveSurfer initialization effect');
|
cancelAnimationFrame(rafId);
|
||||||
if (rafId1) cancelAnimationFrame(rafId1);
|
};
|
||||||
if (rafId2) cancelAnimationFrame(rafId2);
|
// Only run on mount-like conditions. audioUrl is here so we create the instance
|
||||||
if (timeoutId) clearTimeout(timeoutId);
|
// when the player first appears, but we guard against re-creation above.
|
||||||
|
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||||
|
}, [audioUrl, setIsPlaying, setDuration, setCurrentTime]);
|
||||||
|
|
||||||
|
// Destroy WaveSurfer only on unmount
|
||||||
|
useEffect(() => {
|
||||||
|
return () => {
|
||||||
if (wavesurferRef.current) {
|
if (wavesurferRef.current) {
|
||||||
debug.log('Destroying WaveSurfer instance');
|
debug.log('Destroying WaveSurfer instance (unmount)');
|
||||||
try {
|
try {
|
||||||
wavesurferRef.current.destroy();
|
wavesurferRef.current.destroy();
|
||||||
} catch (error) {
|
} catch (err) {
|
||||||
debug.error('Error destroying WaveSurfer:', error);
|
debug.error('Error destroying WaveSurfer:', err);
|
||||||
}
|
}
|
||||||
wavesurferRef.current = null;
|
wavesurferRef.current = null;
|
||||||
|
setWsReady(false);
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
}, [audioUrl, setIsPlaying, setCurrentTime, setDuration]);
|
}, []);
|
||||||
|
|
||||||
// Load audio when URL changes (only if WaveSurfer is already initialized)
|
// Load audio when URL changes (reuses the existing WaveSurfer instance)
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
const wavesurfer = wavesurferRef.current;
|
const wavesurfer = wavesurferRef.current;
|
||||||
|
if (!wavesurfer || !wsReady) return;
|
||||||
|
|
||||||
if (!audioUrl || !wavesurfer) {
|
if (!audioUrl) {
|
||||||
// Reset state when no audio or WaveSurfer not ready
|
// No audio - pause and reset
|
||||||
if (!audioUrl && wavesurfer) {
|
wavesurfer.pause();
|
||||||
wavesurfer.pause();
|
wavesurfer.seekTo(0);
|
||||||
wavesurfer.seekTo(0);
|
loadingRef.current = false;
|
||||||
loadingRef.current = false;
|
setIsLoading(false);
|
||||||
setIsLoading(false);
|
setDuration(0);
|
||||||
setDuration(0);
|
setCurrentTime(0);
|
||||||
setCurrentTime(0);
|
setError(null);
|
||||||
setError(null);
|
isUsingNativePlaybackRef.current = false;
|
||||||
// Reset native playback flag
|
|
||||||
isUsingNativePlaybackRef.current = false;
|
|
||||||
}
|
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Stop native playback if it was active
|
// Reset native playback state
|
||||||
if (isUsingNativePlaybackRef.current && platform.metadata.isTauri) {
|
|
||||||
try {
|
|
||||||
platform.audio.stopPlayback();
|
|
||||||
debug.log('Stopped native audio playback');
|
|
||||||
} catch (error) {
|
|
||||||
debug.error('Failed to stop native playback:', error);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Reset native playback flag when loading new audio
|
|
||||||
// Unmute WaveSurfer if it was muted for native playback
|
|
||||||
if (isUsingNativePlaybackRef.current) {
|
|
||||||
wavesurfer.setMuted(false);
|
|
||||||
wavesurfer.setVolume(usePlayerStore.getState().volume);
|
|
||||||
}
|
|
||||||
isUsingNativePlaybackRef.current = false;
|
isUsingNativePlaybackRef.current = false;
|
||||||
|
wavesurfer.setMuted(false);
|
||||||
|
wavesurfer.setVolume(usePlayerStore.getState().volume);
|
||||||
|
|
||||||
// CRITICAL: Force stop any current playback and cancel any pending loads
|
// Stop current playback and reset position before loading new audio.
|
||||||
// This must happen BEFORE any early returns
|
// With the WebAudio backend, pause() accumulates playedDuration internally.
|
||||||
debug.log('Audio URL changed to:', audioUrl);
|
// seekTo(0) resets it so the new track starts from the beginning.
|
||||||
|
debug.log('Loading new audio URL:', audioUrl);
|
||||||
// COMPLETELY stop and destroy the current audio
|
|
||||||
try {
|
try {
|
||||||
// First pause if playing
|
|
||||||
if (wavesurfer.isPlaying()) {
|
if (wavesurfer.isPlaying()) {
|
||||||
debug.log('Pausing current playback');
|
|
||||||
wavesurfer.pause();
|
wavesurfer.pause();
|
||||||
}
|
}
|
||||||
|
wavesurfer.seekTo(0);
|
||||||
// Use empty() to completely destroy the waveform and reset media
|
} catch (err) {
|
||||||
debug.log('Calling wavesurfer.empty() to destroy audio');
|
debug.error('Error resetting before load:', err);
|
||||||
wavesurfer.empty();
|
|
||||||
} catch (error) {
|
|
||||||
debug.error('Error stopping previous audio:', error);
|
|
||||||
// Continue anyway to load new audio
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Reset loading state to allow new load (cancel any pending loads)
|
|
||||||
loadingRef.current = false;
|
|
||||||
|
|
||||||
// Now start the new load
|
|
||||||
loadingRef.current = true;
|
loadingRef.current = true;
|
||||||
setIsLoading(true);
|
setIsLoading(true);
|
||||||
setError(null);
|
setError(null);
|
||||||
setCurrentTime(0);
|
setCurrentTime(0);
|
||||||
setDuration(0);
|
setDuration(0);
|
||||||
|
|
||||||
// Load new audio
|
|
||||||
debug.log('Starting new audio load for:', audioUrl);
|
|
||||||
wavesurfer
|
wavesurfer
|
||||||
.load(audioUrl)
|
.load(audioUrl)
|
||||||
.then(() => {
|
.then(() => {
|
||||||
debug.log('Audio load promise resolved');
|
debug.log('Audio loaded into WaveSurfer');
|
||||||
// Don't set loading to false here - wait for 'ready' event
|
loadingRef.current = false;
|
||||||
})
|
})
|
||||||
.catch((error) => {
|
.catch((err) => {
|
||||||
debug.error('Failed to load audio:', error);
|
debug.error('Failed to load audio:', err);
|
||||||
debug.error('Audio URL:', audioUrl);
|
|
||||||
loadingRef.current = false;
|
loadingRef.current = false;
|
||||||
setIsLoading(false);
|
setIsLoading(false);
|
||||||
setError(`Failed to load audio: ${error instanceof Error ? error.message : String(error)}`);
|
setError(`Failed to load audio: ${err instanceof Error ? err.message : String(err)}`);
|
||||||
});
|
});
|
||||||
}, [audioUrl, setCurrentTime, setDuration]);
|
}, [audioUrl, wsReady, setCurrentTime, setDuration]);
|
||||||
|
|
||||||
// Sync play/pause state (only when user clicks play/pause button, not auto-sync)
|
// Sync play/pause state (only when user clicks play/pause button, not auto-sync)
|
||||||
// This effect is kept for external state changes but should be minimal
|
// This effect is kept for external state changes but should be minimal
|
||||||
@@ -520,7 +318,6 @@ export function AudioPlayer() {
|
|||||||
if (!wavesurferRef.current || duration === 0) return;
|
if (!wavesurferRef.current || duration === 0) return;
|
||||||
|
|
||||||
if (isPlaying && wavesurferRef.current.isPlaying() === false) {
|
if (isPlaying && wavesurferRef.current.isPlaying() === false) {
|
||||||
// Only auto-play if audio is ready
|
|
||||||
wavesurferRef.current.play().catch((error) => {
|
wavesurferRef.current.play().catch((error) => {
|
||||||
debug.error('Failed to play:', error);
|
debug.error('Failed to play:', error);
|
||||||
setIsPlaying(false);
|
setIsPlaying(false);
|
||||||
@@ -534,14 +331,7 @@ export function AudioPlayer() {
|
|||||||
// Sync volume
|
// Sync volume
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
if (wavesurferRef.current) {
|
if (wavesurferRef.current) {
|
||||||
// If using native playback, keep WaveSurfer muted regardless of volume setting
|
wavesurferRef.current.setVolume(volume);
|
||||||
if (isUsingNativePlaybackRef.current) {
|
|
||||||
wavesurferRef.current.setVolume(0);
|
|
||||||
debug.log('Volume sync: Using native playback, keeping WaveSurfer muted');
|
|
||||||
} else {
|
|
||||||
wavesurferRef.current.setVolume(volume);
|
|
||||||
debug.log('Volume synced:', volume);
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}, [volume]);
|
}, [volume]);
|
||||||
|
|
||||||
@@ -566,7 +356,6 @@ export function AudioPlayer() {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Reset to beginning and play
|
|
||||||
debug.log('Restarting current audio from beginning');
|
debug.log('Restarting current audio from beginning');
|
||||||
wavesurfer.seekTo(0);
|
wavesurfer.seekTo(0);
|
||||||
wavesurfer.play().catch((error) => {
|
wavesurfer.play().catch((error) => {
|
||||||
@@ -575,34 +364,35 @@ export function AudioPlayer() {
|
|||||||
setError(`Playback error: ${error instanceof Error ? error.message : String(error)}`);
|
setError(`Playback error: ${error instanceof Error ? error.message : String(error)}`);
|
||||||
});
|
});
|
||||||
|
|
||||||
// Clear the restart flag
|
|
||||||
clearRestartFlag();
|
clearRestartFlag();
|
||||||
}, [shouldRestart, duration, setIsPlaying, clearRestartFlag]);
|
}, [shouldRestart, duration, setIsPlaying, clearRestartFlag]);
|
||||||
|
|
||||||
// Handle shouldAutoPlay flag - for story mode auto-advance
|
// Auto-play is handled exclusively in the WaveSurfer 'ready' event handler.
|
||||||
const shouldAutoPlay = usePlayerStore((state) => state.shouldAutoPlay);
|
// A separate effect here would race with the ready event since the WebAudio
|
||||||
const clearAutoPlayFlag = usePlayerStore((state) => state.clearAutoPlayFlag);
|
// backend needs to fully decode the audio before play() works correctly.
|
||||||
|
|
||||||
|
// Spacebar to play/pause (capture phase so it fires before focused elements)
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
const wavesurfer = wavesurferRef.current;
|
const onKeyDown = (e: KeyboardEvent) => {
|
||||||
if (!wavesurfer || !shouldAutoPlay || duration === 0) {
|
if (e.code !== 'Space') return;
|
||||||
return;
|
// Ignore if user is typing in an input/textarea
|
||||||
}
|
const tag = (e.target as HTMLElement)?.tagName;
|
||||||
|
if (tag === 'INPUT' || tag === 'TEXTAREA' || (e.target as HTMLElement)?.isContentEditable) {
|
||||||
// Auto-play the newly loaded audio
|
return;
|
||||||
debug.log('Auto-playing next track in story mode');
|
}
|
||||||
wavesurfer.seekTo(0);
|
if (audioUrl && duration > 0 && wavesurferRef.current) {
|
||||||
wavesurfer.play().catch((error) => {
|
e.preventDefault();
|
||||||
debug.error('Failed to auto-play:', error);
|
e.stopPropagation();
|
||||||
setIsPlaying(false);
|
if (wavesurferRef.current.isPlaying()) {
|
||||||
setError(`Playback error: ${error instanceof Error ? error.message : String(error)}`);
|
wavesurferRef.current.pause();
|
||||||
});
|
} else {
|
||||||
|
wavesurferRef.current.play().catch((err) => debug.error('Spacebar play failed:', err));
|
||||||
// Clear the auto-play flag
|
}
|
||||||
clearAutoPlayFlag();
|
}
|
||||||
}, [shouldAutoPlay, duration, setIsPlaying, clearAutoPlayFlag]);
|
};
|
||||||
|
document.addEventListener('keydown', onKeyDown, true);
|
||||||
// Handle loop - WaveSurfer handles this via the 'finish' event
|
return () => document.removeEventListener('keydown', onKeyDown, true);
|
||||||
|
}, [audioUrl, duration]);
|
||||||
|
|
||||||
const handlePlayPause = async () => {
|
const handlePlayPause = async () => {
|
||||||
// Standard WaveSurfer playback (works for both normal and native playback modes)
|
// Standard WaveSurfer playback (works for both normal and native playback modes)
|
||||||
@@ -741,32 +531,32 @@ export function AudioPlayer() {
|
|||||||
size="icon"
|
size="icon"
|
||||||
onClick={handlePlayPause}
|
onClick={handlePlayPause}
|
||||||
disabled={isLoading || duration === 0}
|
disabled={isLoading || duration === 0}
|
||||||
className="shrink-0"
|
className={`shrink-0 -mt-2 ${isPlaying ? 'bg-accent text-accent-foreground' : ''}`}
|
||||||
title={duration === 0 && !isLoading ? 'Audio not loaded' : ''}
|
title={duration === 0 && !isLoading ? 'Audio not loaded' : ''}
|
||||||
aria-label={
|
aria-label={
|
||||||
duration === 0 && !isLoading ? 'Audio not loaded' : isPlaying ? 'Pause' : 'Play'
|
duration === 0 && !isLoading ? 'Audio not loaded' : isPlaying ? 'Pause' : 'Play'
|
||||||
}
|
}
|
||||||
>
|
>
|
||||||
{isPlaying ? <Pause className="h-5 w-5" /> : <Play className="h-5 w-5" />}
|
{isPlaying ? (
|
||||||
|
<Pause className="h-5 w-5 fill-current" />
|
||||||
|
) : (
|
||||||
|
<Play className="h-5 w-5 fill-current" />
|
||||||
|
)}
|
||||||
</Button>
|
</Button>
|
||||||
|
|
||||||
{/* Waveform */}
|
{/* Waveform */}
|
||||||
<div className="flex-1 min-w-0 flex flex-col gap-1">
|
<div className="flex-1 min-w-0 flex flex-col gap-1">
|
||||||
<div ref={waveformRef} className="w-full min-h-[80px]" />
|
<div ref={waveformRef} className="w-full min-h-[80px] select-none" />
|
||||||
{duration > 0 && (
|
<Slider
|
||||||
<Slider
|
value={duration > 0 ? [(currentTime / duration) * 100] : [0]}
|
||||||
value={duration > 0 ? [(currentTime / duration) * 100] : [0]}
|
onValueChange={handleSeek}
|
||||||
onValueChange={handleSeek}
|
max={100}
|
||||||
max={100}
|
step={0.1}
|
||||||
step={0.1}
|
className="w-full"
|
||||||
className="w-full"
|
aria-label="Playback position"
|
||||||
aria-label="Playback position"
|
aria-valuetext={`${formatAudioDuration(currentTime)} of ${formatAudioDuration(duration)}`}
|
||||||
aria-valuetext={`${formatAudioDuration(currentTime)} of ${formatAudioDuration(duration)}`}
|
/>
|
||||||
/>
|
|
||||||
)}
|
|
||||||
{isLoading && (
|
|
||||||
<div className="text-xs text-muted-foreground text-center py-2">Loading audio...</div>
|
|
||||||
)}
|
|
||||||
{error && <div className="text-xs text-destructive text-center py-2">{error}</div>}
|
{error && <div className="text-xs text-destructive text-center py-2">{error}</div>}
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@@ -777,19 +567,12 @@ export function AudioPlayer() {
|
|||||||
<span className="font-mono">{formatAudioDuration(duration)}</span>
|
<span className="font-mono">{formatAudioDuration(duration)}</span>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
{/* Title */}
|
|
||||||
{title && (
|
|
||||||
<div className="text-sm font-medium truncate max-w-[200px] shrink-0 hidden lg:block">
|
|
||||||
{title}
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
|
|
||||||
{/* Loop Button */}
|
{/* Loop Button */}
|
||||||
<Button
|
<Button
|
||||||
variant="ghost"
|
variant="ghost"
|
||||||
size="icon"
|
size="icon"
|
||||||
onClick={toggleLoop}
|
onClick={toggleLoop}
|
||||||
className={isLooping ? 'text-primary' : ''}
|
className={isLooping ? 'bg-accent text-accent-foreground' : ''}
|
||||||
title="Toggle loop"
|
title="Toggle loop"
|
||||||
aria-label={isLooping ? 'Stop looping' : 'Loop'}
|
aria-label={isLooping ? 'Stop looping' : 'Loop'}
|
||||||
>
|
>
|
||||||
|
|||||||
@@ -20,6 +20,8 @@ const ENGINE_OPTIONS = [
|
|||||||
{ value: 'luxtts', label: 'LuxTTS' },
|
{ value: 'luxtts', label: 'LuxTTS' },
|
||||||
{ value: 'chatterbox', label: 'Chatterbox' },
|
{ value: 'chatterbox', label: 'Chatterbox' },
|
||||||
{ value: 'chatterbox_turbo', label: 'Chatterbox Turbo' },
|
{ value: 'chatterbox_turbo', label: 'Chatterbox Turbo' },
|
||||||
|
{ value: 'tada:1B', label: 'TADA 1B' },
|
||||||
|
{ value: 'tada:3B', label: 'TADA 3B Multilingual' },
|
||||||
] as const;
|
] as const;
|
||||||
|
|
||||||
const ENGINE_DESCRIPTIONS: Record<string, string> = {
|
const ENGINE_DESCRIPTIONS: Record<string, string> = {
|
||||||
@@ -27,6 +29,7 @@ const ENGINE_DESCRIPTIONS: Record<string, string> = {
|
|||||||
luxtts: 'Fast, English-focused',
|
luxtts: 'Fast, English-focused',
|
||||||
chatterbox: '23 languages, incl. Hebrew',
|
chatterbox: '23 languages, incl. Hebrew',
|
||||||
chatterbox_turbo: 'English, [laugh] [cough] tags',
|
chatterbox_turbo: 'English, [laugh] [cough] tags',
|
||||||
|
tada: 'HumeAI, 700s+ coherent audio',
|
||||||
};
|
};
|
||||||
|
|
||||||
/** Engines that only support English and should force language to 'en' on select. */
|
/** Engines that only support English and should force language to 'en' on select. */
|
||||||
@@ -34,6 +37,7 @@ const ENGLISH_ONLY_ENGINES = new Set(['luxtts', 'chatterbox_turbo']);
|
|||||||
|
|
||||||
function getSelectValue(engine: string, modelSize?: string): string {
|
function getSelectValue(engine: string, modelSize?: string): string {
|
||||||
if (engine === 'qwen') return `qwen:${modelSize || '1.7B'}`;
|
if (engine === 'qwen') return `qwen:${modelSize || '1.7B'}`;
|
||||||
|
if (engine === 'tada') return `tada:${modelSize || '1B'}`;
|
||||||
return engine;
|
return engine;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -48,6 +52,20 @@ function handleEngineChange(form: UseFormReturn<GenerationFormValues>, value: st
|
|||||||
if (!available.some((l) => l.value === currentLang)) {
|
if (!available.some((l) => l.value === currentLang)) {
|
||||||
form.setValue('language', available[0]?.value ?? 'en');
|
form.setValue('language', available[0]?.value ?? 'en');
|
||||||
}
|
}
|
||||||
|
} else if (value.startsWith('tada:')) {
|
||||||
|
const [, modelSize] = value.split(':');
|
||||||
|
form.setValue('engine', 'tada');
|
||||||
|
form.setValue('modelSize', modelSize as '1B' | '3B');
|
||||||
|
// TADA 1B is English-only; 3B is multilingual
|
||||||
|
if (modelSize === '1B') {
|
||||||
|
form.setValue('language', 'en');
|
||||||
|
} else {
|
||||||
|
const currentLang = form.getValues('language');
|
||||||
|
const available = getLanguageOptionsForEngine('tada');
|
||||||
|
if (!available.some((l) => l.value === currentLang)) {
|
||||||
|
form.setValue('language', available[0]?.value ?? 'en');
|
||||||
|
}
|
||||||
|
}
|
||||||
} else {
|
} else {
|
||||||
form.setValue('engine', value as GenerationFormValues['engine']);
|
form.setValue('engine', value as GenerationFormValues['engine']);
|
||||||
form.setValue('modelSize', undefined as unknown as '1.7B' | '0.6B');
|
form.setValue('modelSize', undefined as unknown as '1.7B' | '0.6B');
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ import {
|
|||||||
Wand2,
|
Wand2,
|
||||||
} from 'lucide-react';
|
} from 'lucide-react';
|
||||||
import { useEffect, useRef, useState } from 'react';
|
import { useEffect, useRef, useState } from 'react';
|
||||||
import Loader from 'react-loaders';
|
|
||||||
import { EffectsChainEditor } from '@/components/Effects/EffectsChainEditor';
|
import { EffectsChainEditor } from '@/components/Effects/EffectsChainEditor';
|
||||||
import { Button } from '@/components/ui/button';
|
import { Button } from '@/components/ui/button';
|
||||||
import {
|
import {
|
||||||
@@ -56,8 +56,35 @@ import { formatDate, formatDuration, formatEngineName } from '@/lib/utils/format
|
|||||||
import { useGenerationStore } from '@/stores/generationStore';
|
import { useGenerationStore } from '@/stores/generationStore';
|
||||||
import { usePlayerStore } from '@/stores/playerStore';
|
import { usePlayerStore } from '@/stores/playerStore';
|
||||||
|
|
||||||
// OLD TABLE-BASED COMPONENT - REMOVED (can be found in git history)
|
// ─── Audio Bars ─────────────────────────────────────────────────────────────
|
||||||
// This is the new alternate history view with fixed height rows
|
|
||||||
|
function AudioBars({ mode }: { mode: 'idle' | 'generating' | 'playing' }) {
|
||||||
|
const barColor = mode !== 'idle' ? 'bg-accent' : 'bg-muted-foreground/40';
|
||||||
|
return (
|
||||||
|
<div className="flex items-center gap-[2px] h-5">
|
||||||
|
{[0, 1, 2, 3, 4].map((i) => (
|
||||||
|
<motion.div
|
||||||
|
key={`${mode}-${i}`}
|
||||||
|
className={`w-[3px] rounded-full ${barColor}`}
|
||||||
|
animate={
|
||||||
|
mode === 'generating'
|
||||||
|
? { height: ['6px', '16px', '6px'] }
|
||||||
|
: mode === 'playing'
|
||||||
|
? { height: ['8px', '14px', '4px', '12px', '8px'] }
|
||||||
|
: { height: '8px' }
|
||||||
|
}
|
||||||
|
transition={
|
||||||
|
mode === 'generating'
|
||||||
|
? { duration: 0.6, repeat: Infinity, delay: i * 0.08, ease: 'easeInOut' }
|
||||||
|
: mode === 'playing'
|
||||||
|
? { duration: 1.2, repeat: Infinity, delay: i * 0.15, ease: 'easeInOut' }
|
||||||
|
: { duration: 0.4, ease: 'easeOut' }
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
// NEW ALTERNATE HISTORY VIEW - FIXED HEIGHT ROWS WITH INFINITE SCROLL
|
// NEW ALTERNATE HISTORY VIEW - FIXED HEIGHT ROWS WITH INFINITE SCROLL
|
||||||
export function HistoryTable() {
|
export function HistoryTable() {
|
||||||
@@ -126,7 +153,9 @@ export function HistoryTable() {
|
|||||||
}
|
}
|
||||||
}, [historyData, page]);
|
}, [historyData, page]);
|
||||||
|
|
||||||
// Reset to page 0 when deletions or imports occur
|
// Reset to page 0 when deletions, imports, or generation completions occur
|
||||||
|
const pendingCount = useGenerationStore((state) => state.pendingGenerationIds.size);
|
||||||
|
const prevPendingCountRef = useRef(pendingCount);
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
if (deleteGeneration.isSuccess || importGeneration.isSuccess) {
|
if (deleteGeneration.isSuccess || importGeneration.isSuccess) {
|
||||||
setPage(0);
|
setPage(0);
|
||||||
@@ -134,6 +163,19 @@ export function HistoryTable() {
|
|||||||
}
|
}
|
||||||
}, [deleteGeneration.isSuccess, importGeneration.isSuccess]);
|
}, [deleteGeneration.isSuccess, importGeneration.isSuccess]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
// A generation finished (pending count decreased) — scroll back to show it
|
||||||
|
if (
|
||||||
|
prevPendingCountRef.current > 0 &&
|
||||||
|
pendingCount < prevPendingCountRef.current &&
|
||||||
|
page !== 0
|
||||||
|
) {
|
||||||
|
setPage(0);
|
||||||
|
setAllHistory([]);
|
||||||
|
}
|
||||||
|
prevPendingCountRef.current = pendingCount;
|
||||||
|
}, [pendingCount, page]);
|
||||||
|
|
||||||
// Intersection Observer for infinite scroll
|
// Intersection Observer for infinite scroll
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
const loadMoreEl = loadMoreRef.current;
|
const loadMoreEl = loadMoreRef.current;
|
||||||
@@ -413,7 +455,7 @@ export function HistoryTable() {
|
|||||||
role={isPlayable ? 'button' : undefined}
|
role={isPlayable ? 'button' : undefined}
|
||||||
tabIndex={isPlayable ? 0 : undefined}
|
tabIndex={isPlayable ? 0 : undefined}
|
||||||
className={cn(
|
className={cn(
|
||||||
'flex items-stretch gap-4 h-26 p-3',
|
'flex items-stretch gap-4 h-26 p-3 outline-none',
|
||||||
isPlayable && 'hover:bg-muted/70 cursor-pointer rounded-md',
|
isPlayable && 'hover:bg-muted/70 cursor-pointer rounded-md',
|
||||||
isVersionsExpanded && 'rounded-b-none',
|
isVersionsExpanded && 'rounded-b-none',
|
||||||
)}
|
)}
|
||||||
@@ -446,12 +488,9 @@ export function HistoryTable() {
|
|||||||
>
|
>
|
||||||
{/* Status icon */}
|
{/* Status icon */}
|
||||||
<div className="flex items-center shrink-0 w-10 justify-center overflow-hidden">
|
<div className="flex items-center shrink-0 w-10 justify-center overflow-hidden">
|
||||||
<div className="scale-50">
|
<AudioBars
|
||||||
<Loader
|
mode={isGenerating ? 'generating' : isCurrentlyPlaying ? 'playing' : 'idle'}
|
||||||
type={isGenerating ? 'line-scale' : 'line-scale-pulse-out-rapid'}
|
/>
|
||||||
active={isGenerating || isCurrentlyPlaying}
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
{/* Left side - Meta information */}
|
{/* Left side - Meta information */}
|
||||||
|
|||||||
@@ -243,7 +243,40 @@ export function GpuAcceleration() {
|
|||||||
|
|
||||||
{/* Native GPU detected - no CUDA download needed */}
|
{/* Native GPU detected - no CUDA download needed */}
|
||||||
|
|
||||||
{/* CUDA download section - only show when no GPU is active (native or CUDA) */}
|
{/* Currently running CUDA - show switch back to CPU */}
|
||||||
|
{isCurrentlyCuda && platform.metadata.isTauri && (
|
||||||
|
<>
|
||||||
|
{restartPhase !== 'idle' ? (
|
||||||
|
<div className="flex items-center gap-2 p-3 rounded-lg bg-primary/5 border">
|
||||||
|
<Loader2 className="h-4 w-4 animate-spin" />
|
||||||
|
<span className="text-sm">
|
||||||
|
{restartPhase === 'stopping' && 'Stopping server...'}
|
||||||
|
{restartPhase === 'waiting' && 'Restarting server...'}
|
||||||
|
{restartPhase === 'ready' && 'Server restarted successfully!'}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
) : (
|
||||||
|
<div className="space-y-3">
|
||||||
|
<p className="text-sm text-muted-foreground">
|
||||||
|
Running with CUDA GPU acceleration. Switch back to CPU if needed (you can
|
||||||
|
re-download later).
|
||||||
|
</p>
|
||||||
|
<Button onClick={handleSwitchToCpu} variant="outline" className="w-full" size="sm">
|
||||||
|
<RotateCw className="h-4 w-4 mr-2" />
|
||||||
|
Switch to CPU Backend
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{error && (
|
||||||
|
<div className="flex items-center gap-2 text-sm text-destructive">
|
||||||
|
<AlertCircle className="h-4 w-4 shrink-0" />
|
||||||
|
<span>{error}</span>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* CUDA download/manage section - show when no native GPU and not currently running CUDA */}
|
||||||
{!hasNativeGpu && !isCurrentlyCuda && (
|
{!hasNativeGpu && !isCurrentlyCuda && (
|
||||||
<>
|
<>
|
||||||
{/* Download progress (manual download or auto-update) */}
|
{/* Download progress (manual download or auto-update) */}
|
||||||
@@ -315,7 +348,7 @@ export function GpuAcceleration() {
|
|||||||
)}
|
)}
|
||||||
|
|
||||||
{/* Downloaded but not active - show switch button */}
|
{/* Downloaded but not active - show switch button */}
|
||||||
{cudaAvailable && !isCurrentlyCuda && platform.metadata.isTauri && (
|
{cudaAvailable && platform.metadata.isTauri && (
|
||||||
<div className="space-y-3">
|
<div className="space-y-3">
|
||||||
<p className="text-sm text-muted-foreground">
|
<p className="text-sm text-muted-foreground">
|
||||||
CUDA backend is downloaded and ready. Restart the server to enable GPU
|
CUDA backend is downloaded and ready. Restart the server to enable GPU
|
||||||
@@ -328,27 +361,8 @@ export function GpuAcceleration() {
|
|||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
{/* Currently active - show switch back to CPU */}
|
|
||||||
{isCurrentlyCuda && platform.metadata.isTauri && (
|
|
||||||
<div className="space-y-3">
|
|
||||||
<p className="text-sm text-muted-foreground">
|
|
||||||
Running with CUDA GPU acceleration. Switch back to CPU if needed (you can
|
|
||||||
re-download later).
|
|
||||||
</p>
|
|
||||||
<Button
|
|
||||||
onClick={handleSwitchToCpu}
|
|
||||||
variant="outline"
|
|
||||||
className="w-full"
|
|
||||||
size="sm"
|
|
||||||
>
|
|
||||||
<RotateCw className="h-4 w-4 mr-2" />
|
|
||||||
Switch to CPU Backend
|
|
||||||
</Button>
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
|
|
||||||
{/* Delete option when downloaded (and not active) */}
|
{/* Delete option when downloaded (and not active) */}
|
||||||
{cudaAvailable && !isCurrentlyCuda && (
|
{cudaAvailable && (
|
||||||
<Button
|
<Button
|
||||||
onClick={handleDelete}
|
onClick={handleDelete}
|
||||||
variant="ghost"
|
variant="ghost"
|
||||||
|
|||||||
@@ -62,6 +62,10 @@ const MODEL_DESCRIPTIONS: Record<string, string> = {
|
|||||||
'Production-grade open source TTS by Resemble AI. Supports 23 languages with voice cloning and emotion exaggeration control.',
|
'Production-grade open source TTS by Resemble AI. Supports 23 languages with voice cloning and emotion exaggeration control.',
|
||||||
'chatterbox-turbo':
|
'chatterbox-turbo':
|
||||||
'Streamlined 350M parameter TTS by Resemble AI. High-quality English speech with less compute and VRAM than larger models.',
|
'Streamlined 350M parameter TTS by Resemble AI. High-quality English speech with less compute and VRAM than larger models.',
|
||||||
|
'tada-1b':
|
||||||
|
'HumeAI TADA 1B — English speech-language model built on Llama 3.2 1B. Generates 700s+ of coherent audio with synchronized text-acoustic alignment.',
|
||||||
|
'tada-3b-ml':
|
||||||
|
'HumeAI TADA 3B Multilingual — built on Llama 3.2 3B. Supports 10 languages with high-fidelity voice cloning via text-acoustic dual alignment.',
|
||||||
'whisper-base':
|
'whisper-base':
|
||||||
'Smallest Whisper model (74M parameters). Fast transcription with moderate accuracy.',
|
'Smallest Whisper model (74M parameters). Fast transcription with moderate accuracy.',
|
||||||
'whisper-small':
|
'whisper-small':
|
||||||
@@ -391,7 +395,8 @@ export function ModelManagement() {
|
|||||||
(m) =>
|
(m) =>
|
||||||
m.model_name.startsWith('qwen-tts') ||
|
m.model_name.startsWith('qwen-tts') ||
|
||||||
m.model_name.startsWith('luxtts') ||
|
m.model_name.startsWith('luxtts') ||
|
||||||
m.model_name.startsWith('chatterbox'),
|
m.model_name.startsWith('chatterbox') ||
|
||||||
|
m.model_name.startsWith('tada'),
|
||||||
) ?? [];
|
) ?? [];
|
||||||
const whisperModels = modelStatus?.models.filter((m) => m.model_name.startsWith('whisper')) ?? [];
|
const whisperModels = modelStatus?.models.filter((m) => m.model_name.startsWith('whisper')) ?? [];
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,135 @@
|
|||||||
|
import { ArrowUpRight } from 'lucide-react';
|
||||||
|
import type { CSSProperties, ReactNode } from 'react';
|
||||||
|
import { useEffect, useState } from 'react';
|
||||||
|
import voiceboxLogo from '@/assets/voicebox-logo.png';
|
||||||
|
import { usePlatform } from '@/platform/PlatformContext';
|
||||||
|
|
||||||
|
function FadeIn({ delay = 0, children }: { delay?: number; children: ReactNode }) {
|
||||||
|
return (
|
||||||
|
<div
|
||||||
|
className="animate-[fadeInUp_0.5s_ease_both]"
|
||||||
|
style={{ animationDelay: `${delay}ms` } as CSSProperties}
|
||||||
|
>
|
||||||
|
{children}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function AboutPage() {
|
||||||
|
const platform = usePlatform();
|
||||||
|
const [version, setVersion] = useState('');
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
platform.metadata
|
||||||
|
.getVersion()
|
||||||
|
.then(setVersion)
|
||||||
|
.catch(() => setVersion(''));
|
||||||
|
}, [platform]);
|
||||||
|
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
<style>{`
|
||||||
|
@keyframes fadeInUp {
|
||||||
|
from {
|
||||||
|
opacity: 0;
|
||||||
|
transform: translateY(8px);
|
||||||
|
}
|
||||||
|
to {
|
||||||
|
opacity: 1;
|
||||||
|
transform: translateY(0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
`}</style>
|
||||||
|
<div className="max-w-md mx-auto h-full flex items-center">
|
||||||
|
<div className="flex flex-col items-center text-center space-y-5">
|
||||||
|
<FadeIn delay={0}>
|
||||||
|
<img src={voiceboxLogo} alt="Voicebox" className="w-20 h-20 object-contain" />
|
||||||
|
</FadeIn>
|
||||||
|
|
||||||
|
<FadeIn delay={80}>
|
||||||
|
<div className="space-y-1.5">
|
||||||
|
<h1 className="text-lg font-semibold">Voicebox</h1>
|
||||||
|
<p className="text-xs text-muted-foreground/60 h-4">
|
||||||
|
{version ? `v${version}` : '\u00A0'}
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
</FadeIn>
|
||||||
|
|
||||||
|
<FadeIn delay={160}>
|
||||||
|
<p className="text-sm text-muted-foreground leading-relaxed max-w-sm">
|
||||||
|
The open-source voice synthesis studio. Clone voices, generate speech, apply effects,
|
||||||
|
and build voice-powered apps — all running locally on your machine.
|
||||||
|
</p>
|
||||||
|
</FadeIn>
|
||||||
|
|
||||||
|
<FadeIn delay={240}>
|
||||||
|
<div className="flex items-center gap-1.5 text-sm text-muted-foreground">
|
||||||
|
<span>Created by</span>
|
||||||
|
<a
|
||||||
|
href="https://github.com/jamiepine"
|
||||||
|
target="_blank"
|
||||||
|
rel="noopener noreferrer"
|
||||||
|
className="text-accent hover:underline"
|
||||||
|
>
|
||||||
|
Jamie Pine
|
||||||
|
</a>
|
||||||
|
</div>
|
||||||
|
</FadeIn>
|
||||||
|
|
||||||
|
<FadeIn delay={320}>
|
||||||
|
<div className="flex flex-wrap justify-center gap-3 pt-2">
|
||||||
|
<a
|
||||||
|
href="https://buymeacoffee.com/jamiepine"
|
||||||
|
target="_blank"
|
||||||
|
rel="noopener noreferrer"
|
||||||
|
className="group inline-flex items-center gap-2 rounded-lg border border-border/60 px-4 py-2 text-sm transition-colors hover:bg-muted/50"
|
||||||
|
>
|
||||||
|
<svg
|
||||||
|
className="h-4 w-4 text-[#FFDD00]"
|
||||||
|
viewBox="0 0 24 24"
|
||||||
|
fill="currentColor"
|
||||||
|
aria-hidden="true"
|
||||||
|
>
|
||||||
|
<path d="m20.216 6.415-.132-.666c-.119-.598-.388-1.163-1.001-1.379-.197-.069-.42-.098-.57-.241-.152-.143-.196-.366-.231-.572-.065-.378-.125-.756-.192-1.133-.057-.325-.102-.69-.25-.987-.195-.4-.597-.634-.996-.788a5.723 5.723 0 0 0-.626-.194c-1-.263-2.05-.36-3.077-.416a25.834 25.834 0 0 0-3.7.062c-.915.083-1.88.184-2.75.5-.318.116-.646.256-.888.501-.297.302-.393.77-.177 1.146.154.267.415.456.692.58.36.162.737.284 1.123.366 1.075.238 2.189.331 3.287.37 1.218.05 2.437.01 3.65-.118.299-.033.598-.073.896-.119.352-.054.578-.513.474-.834-.124-.383-.457-.531-.834-.473-.466.074-.96.108-1.382.146-1.177.08-2.358.082-3.536.006a22.228 22.228 0 0 1-1.157-.107c-.086-.01-.18-.025-.258-.036-.243-.036-.484-.08-.724-.13-.111-.027-.111-.185 0-.212h.005c.277-.06.557-.108.838-.147h.002c.131-.009.263-.032.394-.048a25.076 25.076 0 0 1 3.426-.12c.674.019 1.347.067 2.017.144l.228.031c.267.04.533.088.798.145.392.085.895.113 1.07.542.055.137.08.288.111.431l.319 1.484a.237.237 0 0 1-.199.284h-.003c-.037.006-.075.01-.112.015a36.704 36.704 0 0 1-4.743.295 37.059 37.059 0 0 1-4.699-.304c-.14-.017-.293-.042-.417-.06-.326-.048-.649-.108-.973-.161-.393-.065-.768-.032-1.123.161-.29.16-.527.404-.675.701-.154.316-.199.66-.267 1-.069.34-.176.707-.135 1.056.087.753.613 1.365 1.37 1.502a39.69 39.69 0 0 0 11.343.376.483.483 0 0 1 .535.53l-.071.697-1.018 9.907c-.041.41-.047.832-.125 1.237-.122.637-.553 1.028-1.182 1.171-.577.131-1.165.2-1.756.205-.656.004-1.31-.025-1.966-.022-.699.004-1.556-.06-2.095-.58-.475-.458-.54-1.174-.605-1.793l-.731-7.013-.322-3.094c-.037-.351-.286-.695-.678-.678-.336.015-.718.3-.678.679l.228 2.185.949 9.112c.147 1.344 1.174 2.068 2.446 2.272.742.12 1.503.144 2.257.156.966.016 1.942.053 2.892-.122 1.408-.258 2.465-1.198 2.616-2.657.34-3.332.683-6.663 1.024-9.995l.215-2.087a.484.484 0 0 1 .39-.426c.402-.078.787-.212 1.074-.518.455-.488.546-1.124.385-1.766zm-1.478.772c-.145.137-.363.201-.578.233-2.416.359-4.866.54-7.308.46-1.748-.06-3.477-.254-5.207-.498-.17-.024-.353-.055-.47-.18-.22-.236-.111-.71-.054-.995.052-.26.152-.609.463-.646.484-.057 1.046.148 1.526.22.577.088 1.156.159 1.737.212 2.48.226 5.002.19 7.472-.14.45-.06.899-.13 1.345-.21.399-.072.84-.206 1.08.206.166.281.188.657.162.974a.544.544 0 0 1-.169.364zm-6.159 3.9c-.862.37-1.84.788-3.109.788a5.884 5.884 0 0 1-1.569-.217l.877 9.004c.065.78.717 1.38 1.5 1.38 0 0 1.243.065 1.658.065.447 0 1.786-.065 1.786-.065.783 0 1.434-.6 1.499-1.38l.94-9.95a3.996 3.996 0 0 0-1.322-.238c-.826 0-1.491.284-2.26.613z" />
|
||||||
|
</svg>
|
||||||
|
Buy me a coffee
|
||||||
|
<ArrowUpRight className="h-3.5 w-3.5 text-muted-foreground/40 group-hover:text-muted-foreground transition-colors" />
|
||||||
|
</a>
|
||||||
|
<a
|
||||||
|
href="https://github.com/jamiepine/voicebox"
|
||||||
|
target="_blank"
|
||||||
|
rel="noopener noreferrer"
|
||||||
|
className="group inline-flex items-center gap-2 rounded-lg border border-border/60 px-4 py-2 text-sm transition-colors hover:bg-muted/50"
|
||||||
|
>
|
||||||
|
<svg
|
||||||
|
className="h-4 w-4 text-muted-foreground"
|
||||||
|
viewBox="0 0 24 24"
|
||||||
|
fill="currentColor"
|
||||||
|
aria-hidden="true"
|
||||||
|
>
|
||||||
|
<path d="M12 2C6.477 2 2 6.484 2 12.017c0 4.425 2.865 8.18 6.839 9.504.5.092.682-.217.682-.483 0-.237-.008-.868-.013-1.703-2.782.605-3.369-1.343-3.369-1.343-.454-1.158-1.11-1.466-1.11-1.466-.908-.62.069-.608.069-.608 1.003.07 1.531 1.032 1.531 1.032.892 1.53 2.341 1.088 2.91.832.092-.647.35-1.088.636-1.338-2.22-.253-4.555-1.113-4.555-4.951 0-1.093.39-1.988 1.029-2.688-.103-.253-.446-1.272.098-2.65 0 0 .84-.27 2.75 1.026A9.564 9.564 0 0112 6.844c.85.004 1.705.115 2.504.337 1.909-1.296 2.747-1.027 2.747-1.027.546 1.379.202 2.398.1 2.651.64.7 1.028 1.595 1.028 2.688 0 3.848-2.339 4.695-4.566 4.943.359.309.678.92.678 1.855 0 1.338-.012 2.419-.012 2.747 0 .268.18.58.688.482A10.019 10.019 0 0022 12.017C22 6.484 17.522 2 12 2z" />
|
||||||
|
</svg>
|
||||||
|
GitHub
|
||||||
|
<ArrowUpRight className="h-3.5 w-3.5 text-muted-foreground/40 group-hover:text-muted-foreground transition-colors" />
|
||||||
|
</a>
|
||||||
|
</div>
|
||||||
|
</FadeIn>
|
||||||
|
|
||||||
|
<FadeIn delay={400}>
|
||||||
|
<p className="text-xs text-muted-foreground/40 pt-4">
|
||||||
|
Licensed under{' '}
|
||||||
|
<a
|
||||||
|
href="https://github.com/jamiepine/voicebox/blob/main/LICENSE"
|
||||||
|
target="_blank"
|
||||||
|
rel="noopener noreferrer"
|
||||||
|
className="hover:text-muted-foreground/60 transition-colors"
|
||||||
|
>
|
||||||
|
MIT
|
||||||
|
</a>
|
||||||
|
</p>
|
||||||
|
</FadeIn>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,220 @@
|
|||||||
|
import changelogRaw from 'virtual:changelog';
|
||||||
|
import { useMemo, useState } from 'react';
|
||||||
|
import { Badge } from '@/components/ui/badge';
|
||||||
|
import { type ChangelogEntry, parseChangelog } from '@/lib/utils/parseChangelog';
|
||||||
|
|
||||||
|
function renderMarkdown(md: string): React.ReactNode[] {
|
||||||
|
const lines = md.split('\n');
|
||||||
|
const elements: React.ReactNode[] = [];
|
||||||
|
let i = 0;
|
||||||
|
|
||||||
|
while (i < lines.length) {
|
||||||
|
const line = lines[i];
|
||||||
|
|
||||||
|
// Skip empty lines
|
||||||
|
if (line.trim() === '') {
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Tables — collect all lines starting with |
|
||||||
|
if (line.trim().startsWith('|')) {
|
||||||
|
const tableLines: string[] = [];
|
||||||
|
while (i < lines.length && lines[i].trim().startsWith('|')) {
|
||||||
|
tableLines.push(lines[i]);
|
||||||
|
i++;
|
||||||
|
}
|
||||||
|
elements.push(renderTable(tableLines, elements.length));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Headings
|
||||||
|
if (line.startsWith('#### ')) {
|
||||||
|
elements.push(
|
||||||
|
<h5 key={elements.length} className="text-sm font-medium mt-5 mb-1">
|
||||||
|
{inlineMarkdown(line.slice(5))}
|
||||||
|
</h5>,
|
||||||
|
);
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (line.startsWith('### ')) {
|
||||||
|
elements.push(
|
||||||
|
<h4 key={elements.length} className="text-sm font-medium mt-6 mb-2">
|
||||||
|
{inlineMarkdown(line.slice(4))}
|
||||||
|
</h4>,
|
||||||
|
);
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// List items — collect consecutive
|
||||||
|
if (line.startsWith('- ')) {
|
||||||
|
const items: string[] = [];
|
||||||
|
while (i < lines.length && lines[i].startsWith('- ')) {
|
||||||
|
items.push(lines[i].slice(2));
|
||||||
|
i++;
|
||||||
|
}
|
||||||
|
elements.push(
|
||||||
|
<ul key={elements.length} className="space-y-1 my-2">
|
||||||
|
{items.map((item, idx) => (
|
||||||
|
<li key={idx} className="text-sm text-muted-foreground flex gap-2">
|
||||||
|
<span className="text-muted-foreground/50 select-none shrink-0">•</span>
|
||||||
|
<span>{inlineMarkdown(item)}</span>
|
||||||
|
</li>
|
||||||
|
))}
|
||||||
|
</ul>,
|
||||||
|
);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Paragraph
|
||||||
|
elements.push(
|
||||||
|
<p key={elements.length} className="text-sm text-muted-foreground my-2">
|
||||||
|
{inlineMarkdown(line)}
|
||||||
|
</p>,
|
||||||
|
);
|
||||||
|
i++;
|
||||||
|
}
|
||||||
|
|
||||||
|
return elements;
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderTable(tableLines: string[], keyBase: number): React.ReactNode {
|
||||||
|
const parseRow = (line: string) =>
|
||||||
|
line
|
||||||
|
.split('|')
|
||||||
|
.slice(1, -1)
|
||||||
|
.map((c) => c.trim());
|
||||||
|
|
||||||
|
const headers = parseRow(tableLines[0]);
|
||||||
|
// Skip separator line (index 1)
|
||||||
|
const rows = tableLines.slice(2).map(parseRow);
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div key={keyBase} className="overflow-x-auto my-3">
|
||||||
|
<table className="text-sm w-full">
|
||||||
|
<thead>
|
||||||
|
<tr className="border-b">
|
||||||
|
{headers.map((h, hIdx) => (
|
||||||
|
<th
|
||||||
|
key={hIdx}
|
||||||
|
className="text-left py-1.5 pr-4 text-muted-foreground font-medium text-xs"
|
||||||
|
>
|
||||||
|
{inlineMarkdown(h)}
|
||||||
|
</th>
|
||||||
|
))}
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
{rows.map((row, rowIdx) => (
|
||||||
|
<tr key={rowIdx} className="border-b border-border/50">
|
||||||
|
{row.map((cell, cellIdx) => (
|
||||||
|
<td key={cellIdx} className="py-1.5 pr-4 text-muted-foreground">
|
||||||
|
{inlineMarkdown(cell)}
|
||||||
|
</td>
|
||||||
|
))}
|
||||||
|
</tr>
|
||||||
|
))}
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function inlineMarkdown(text: string): React.ReactNode {
|
||||||
|
// Process inline markdown: bold, code, links
|
||||||
|
const parts: React.ReactNode[] = [];
|
||||||
|
// Regex matches: **bold**, `code`, [text](url)
|
||||||
|
const inlineRe = /\*\*(.+?)\*\*|`([^`]+)`|\[([^\]]+)\]\(([^)]+)\)/g;
|
||||||
|
let lastIndex = 0;
|
||||||
|
let match: RegExpExecArray | null = inlineRe.exec(text);
|
||||||
|
|
||||||
|
while (match !== null) {
|
||||||
|
if (match.index > lastIndex) {
|
||||||
|
parts.push(text.slice(lastIndex, match.index));
|
||||||
|
}
|
||||||
|
|
||||||
|
if (match[1] !== undefined) {
|
||||||
|
// Bold
|
||||||
|
parts.push(
|
||||||
|
<strong key={parts.length} className="font-medium text-foreground">
|
||||||
|
{match[1]}
|
||||||
|
</strong>,
|
||||||
|
);
|
||||||
|
} else if (match[2] !== undefined) {
|
||||||
|
// Code
|
||||||
|
parts.push(
|
||||||
|
<code key={parts.length} className="px-1 py-0.5 rounded bg-muted text-xs font-mono">
|
||||||
|
{match[2]}
|
||||||
|
</code>,
|
||||||
|
);
|
||||||
|
} else if (match[3] !== undefined && match[4] !== undefined) {
|
||||||
|
// Link
|
||||||
|
parts.push(
|
||||||
|
<a
|
||||||
|
key={parts.length}
|
||||||
|
href={match[4]}
|
||||||
|
target="_blank"
|
||||||
|
rel="noopener noreferrer"
|
||||||
|
className="text-accent hover:underline"
|
||||||
|
>
|
||||||
|
{match[3]}
|
||||||
|
</a>,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
lastIndex = match.index + match[0].length;
|
||||||
|
match = inlineRe.exec(text);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (lastIndex < text.length) {
|
||||||
|
parts.push(text.slice(lastIndex));
|
||||||
|
}
|
||||||
|
|
||||||
|
return parts.length === 1 ? parts[0] : parts;
|
||||||
|
}
|
||||||
|
|
||||||
|
function ChangelogEntryCard({ entry }: { entry: ChangelogEntry }) {
|
||||||
|
const [expanded, setExpanded] = useState(false);
|
||||||
|
const content = useMemo(() => renderMarkdown(entry.body), [entry.body]);
|
||||||
|
const isLong = entry.body.split('\n').length > 12;
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="border-b border-border/50 pb-6">
|
||||||
|
<div className="flex items-baseline gap-3 mb-1">
|
||||||
|
<h3 className="text-sm font-medium">{entry.version}</h3>
|
||||||
|
{entry.date && <span className="text-xs text-muted-foreground">{entry.date}</span>}
|
||||||
|
{entry.version === 'Unreleased' && <Badge variant="outline">dev</Badge>}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div className={isLong && !expanded ? 'max-h-48 overflow-hidden relative' : ''}>
|
||||||
|
{content}
|
||||||
|
{isLong && !expanded && (
|
||||||
|
<div className="absolute bottom-0 left-0 right-0 h-16 bg-gradient-to-t from-background to-transparent" />
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{isLong && (
|
||||||
|
<button
|
||||||
|
onClick={() => setExpanded(!expanded)}
|
||||||
|
className="text-xs text-accent hover:underline mt-2"
|
||||||
|
>
|
||||||
|
{expanded ? 'Show less' : 'Show more'}
|
||||||
|
</button>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function ChangelogPage() {
|
||||||
|
const entries = useMemo(() => parseChangelog(changelogRaw), []);
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="space-y-6 max-w-2xl">
|
||||||
|
{entries.map((entry) => (
|
||||||
|
<ChangelogEntryCard key={entry.version} entry={entry} />
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,379 @@
|
|||||||
|
import { zodResolver } from '@hookform/resolvers/zod';
|
||||||
|
import { AlertCircle, ArrowUpRight, Book, Download, Loader2, RefreshCw } from 'lucide-react';
|
||||||
|
import { useEffect, useState } from 'react';
|
||||||
|
import { useForm } from 'react-hook-form';
|
||||||
|
import * as z from 'zod';
|
||||||
|
import { Button } from '@/components/ui/button';
|
||||||
|
import { Form, FormControl, FormField, FormItem, FormMessage } from '@/components/ui/form';
|
||||||
|
import { Input } from '@/components/ui/input';
|
||||||
|
import { Progress } from '@/components/ui/progress';
|
||||||
|
import { Toggle } from '@/components/ui/toggle';
|
||||||
|
import { useToast } from '@/components/ui/use-toast';
|
||||||
|
import { useAutoUpdater } from '@/hooks/useAutoUpdater';
|
||||||
|
import { useServerHealth } from '@/lib/hooks/useServer';
|
||||||
|
import { usePlatform } from '@/platform/PlatformContext';
|
||||||
|
import { useServerStore } from '@/stores/serverStore';
|
||||||
|
import { SettingRow, SettingSection } from './SettingRow';
|
||||||
|
|
||||||
|
const connectionSchema = z.object({
|
||||||
|
serverUrl: z.string().url('Please enter a valid URL'),
|
||||||
|
});
|
||||||
|
|
||||||
|
type ConnectionFormValues = z.infer<typeof connectionSchema>;
|
||||||
|
|
||||||
|
export function GeneralPage() {
|
||||||
|
const platform = usePlatform();
|
||||||
|
const serverUrl = useServerStore((state) => state.serverUrl);
|
||||||
|
const setServerUrl = useServerStore((state) => state.setServerUrl);
|
||||||
|
const keepServerRunningOnClose = useServerStore((state) => state.keepServerRunningOnClose);
|
||||||
|
const setKeepServerRunningOnClose = useServerStore((state) => state.setKeepServerRunningOnClose);
|
||||||
|
const mode = useServerStore((state) => state.mode);
|
||||||
|
const setMode = useServerStore((state) => state.setMode);
|
||||||
|
const { toast } = useToast();
|
||||||
|
const { data: health, isLoading, error: healthError } = useServerHealth();
|
||||||
|
|
||||||
|
const form = useForm<ConnectionFormValues>({
|
||||||
|
resolver: zodResolver(connectionSchema),
|
||||||
|
defaultValues: { serverUrl },
|
||||||
|
});
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
form.reset({ serverUrl });
|
||||||
|
}, [serverUrl, form]);
|
||||||
|
|
||||||
|
const { isDirty } = form.formState;
|
||||||
|
|
||||||
|
function onSubmit(data: ConnectionFormValues) {
|
||||||
|
setServerUrl(data.serverUrl);
|
||||||
|
form.reset(data);
|
||||||
|
toast({
|
||||||
|
title: 'Server URL updated',
|
||||||
|
description: `Connected to ${data.serverUrl}`,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="space-y-8 max-w-2xl">
|
||||||
|
<div className="grid grid-cols-2 gap-3">
|
||||||
|
<a
|
||||||
|
href="https://docs.voicebox.sh"
|
||||||
|
target="_blank"
|
||||||
|
rel="noopener noreferrer"
|
||||||
|
className="group flex items-center gap-3 rounded-lg border border-border/60 p-4 transition-colors hover:bg-muted/50"
|
||||||
|
>
|
||||||
|
<Book className="h-5 w-5 shrink-0 text-accent" strokeWidth={2.5} />
|
||||||
|
<div className="min-w-0 flex-1">
|
||||||
|
<div className="text-sm font-medium">Read the Docs</div>
|
||||||
|
<div className="text-xs text-muted-foreground">docs.voicebox.sh</div>
|
||||||
|
</div>
|
||||||
|
<ArrowUpRight className="h-4 w-4 text-muted-foreground/40 group-hover:text-muted-foreground transition-colors" />
|
||||||
|
</a>
|
||||||
|
<a
|
||||||
|
href="https://discord.gg/StkzQasqPS"
|
||||||
|
target="_blank"
|
||||||
|
rel="noopener noreferrer"
|
||||||
|
className="group flex items-center gap-3 rounded-lg border border-border/60 p-4 transition-colors hover:bg-muted/50"
|
||||||
|
>
|
||||||
|
<svg
|
||||||
|
className="h-5 w-5 shrink-0 text-accent"
|
||||||
|
viewBox="0 0 24 24"
|
||||||
|
fill="currentColor"
|
||||||
|
aria-hidden="true"
|
||||||
|
>
|
||||||
|
<path d="M20.317 4.37a19.791 19.791 0 0 0-4.885-1.515.074.074 0 0 0-.079.037c-.21.375-.444.864-.608 1.25a18.27 18.27 0 0 0-5.487 0 12.64 12.64 0 0 0-.617-1.25.077.077 0 0 0-.079-.037A19.736 19.736 0 0 0 3.677 4.37a.07.07 0 0 0-.032.027C.533 9.046-.32 13.58.099 18.057a.082.082 0 0 0 .031.057 19.9 19.9 0 0 0 5.993 3.03.078.078 0 0 0 .084-.028c.462-.63.874-1.295 1.226-1.994a.076.076 0 0 0-.041-.106 13.107 13.107 0 0 1-1.872-.892.077.077 0 0 1-.008-.128 10.2 10.2 0 0 0 .372-.292.074.074 0 0 1 .077-.01c3.928 1.793 8.18 1.793 12.062 0a.074.074 0 0 1 .078.01c.12.098.246.198.373.292a.077.077 0 0 1-.006.127 12.299 12.299 0 0 1-1.873.892.077.077 0 0 0-.041.107c.36.698.772 1.362 1.225 1.993a.076.076 0 0 0 .084.028 19.839 19.839 0 0 0 6.002-3.03.077.077 0 0 0 .032-.054c.5-5.177-.838-9.674-3.549-13.66a.061.061 0 0 0-.031-.03zM8.02 15.33c-1.183 0-2.157-1.085-2.157-2.419 0-1.333.956-2.419 2.157-2.419 1.21 0 2.176 1.095 2.157 2.42 0 1.333-.956 2.418-2.157 2.418zm7.975 0c-1.183 0-2.157-1.085-2.157-2.419 0-1.333.955-2.419 2.157-2.419 1.21 0 2.176 1.095 2.157 2.42 0 1.333-.946 2.418-2.157 2.418z" />
|
||||||
|
</svg>
|
||||||
|
<div className="min-w-0 flex-1">
|
||||||
|
<div className="text-sm font-medium">Join the Discord</div>
|
||||||
|
<div className="text-xs text-muted-foreground">Get help & share voices</div>
|
||||||
|
</div>
|
||||||
|
<ArrowUpRight className="h-4 w-4 text-muted-foreground/40 group-hover:text-muted-foreground transition-colors" />
|
||||||
|
</a>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<SettingSection>
|
||||||
|
<SettingRow
|
||||||
|
title="Server URL"
|
||||||
|
description="The address of your voicebox backend server."
|
||||||
|
action={
|
||||||
|
<ConnectionStatus health={health} isLoading={isLoading} healthError={healthError} />
|
||||||
|
}
|
||||||
|
>
|
||||||
|
<Form {...form}>
|
||||||
|
<form onSubmit={form.handleSubmit(onSubmit)} className="flex gap-2">
|
||||||
|
<FormField
|
||||||
|
control={form.control}
|
||||||
|
name="serverUrl"
|
||||||
|
render={({ field }) => (
|
||||||
|
<FormItem className="flex-1">
|
||||||
|
<FormControl>
|
||||||
|
<Input placeholder="http://127.0.0.1:17493" {...field} />
|
||||||
|
</FormControl>
|
||||||
|
<FormMessage />
|
||||||
|
</FormItem>
|
||||||
|
)}
|
||||||
|
/>
|
||||||
|
{isDirty && (
|
||||||
|
<Button type="submit" size="sm">
|
||||||
|
Save
|
||||||
|
</Button>
|
||||||
|
)}
|
||||||
|
</form>
|
||||||
|
</Form>
|
||||||
|
</SettingRow>
|
||||||
|
|
||||||
|
<SettingRow
|
||||||
|
title="Keep server running when app closes"
|
||||||
|
description="The server will continue running in the background after closing the app."
|
||||||
|
htmlFor="keepServerRunning"
|
||||||
|
action={
|
||||||
|
<Toggle
|
||||||
|
id="keepServerRunning"
|
||||||
|
checked={keepServerRunningOnClose}
|
||||||
|
onCheckedChange={(checked: boolean) => {
|
||||||
|
setKeepServerRunningOnClose(checked);
|
||||||
|
platform.lifecycle.setKeepServerRunning(checked).catch((error) => {
|
||||||
|
console.error('Failed to sync setting to Rust:', error);
|
||||||
|
setKeepServerRunningOnClose(!checked);
|
||||||
|
toast({
|
||||||
|
title: 'Failed to update setting',
|
||||||
|
description: 'Could not sync setting to backend.',
|
||||||
|
variant: 'destructive',
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
});
|
||||||
|
toast({
|
||||||
|
title: 'Setting updated',
|
||||||
|
description: checked
|
||||||
|
? 'Server will continue running when app closes'
|
||||||
|
: 'Server will stop when app closes',
|
||||||
|
});
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
|
||||||
|
{platform.metadata.isTauri && (
|
||||||
|
<SettingRow
|
||||||
|
title="Allow network access"
|
||||||
|
description="Makes the server accessible from other devices on your network. Restart the app after changing."
|
||||||
|
htmlFor="allowNetworkAccess"
|
||||||
|
action={
|
||||||
|
<Toggle
|
||||||
|
id="allowNetworkAccess"
|
||||||
|
checked={mode === 'remote'}
|
||||||
|
onCheckedChange={(checked: boolean) => {
|
||||||
|
setMode(checked ? 'remote' : 'local');
|
||||||
|
toast({
|
||||||
|
title: 'Setting updated',
|
||||||
|
description: checked
|
||||||
|
? 'Network access enabled. Restart the app to apply.'
|
||||||
|
: 'Network access disabled. Restart the app to apply.',
|
||||||
|
});
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
</SettingSection>
|
||||||
|
|
||||||
|
<ApiReferenceCard serverUrl={serverUrl} />
|
||||||
|
|
||||||
|
{platform.metadata.isTauri && <UpdatesSection />}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function ConnectionStatus({
|
||||||
|
health,
|
||||||
|
isLoading,
|
||||||
|
healthError,
|
||||||
|
}: {
|
||||||
|
health: ReturnType<typeof useServerHealth>['data'];
|
||||||
|
isLoading: boolean;
|
||||||
|
healthError: ReturnType<typeof useServerHealth>['error'];
|
||||||
|
}) {
|
||||||
|
if (isLoading) {
|
||||||
|
return (
|
||||||
|
<div className="flex items-center gap-2 rounded-full border border-border/60 px-3 py-1">
|
||||||
|
<Loader2 className="h-3 w-3 animate-spin text-muted-foreground" />
|
||||||
|
<span className="text-xs text-muted-foreground">Connecting</span>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if (healthError) {
|
||||||
|
return (
|
||||||
|
<div className="flex items-center gap-2 rounded-full border border-destructive/30 px-3 py-1">
|
||||||
|
<span className="relative flex h-2 w-2">
|
||||||
|
<span className="absolute inline-flex h-full w-full rounded-full bg-destructive/40" />
|
||||||
|
<span className="relative inline-flex h-2 w-2 rounded-full bg-destructive" />
|
||||||
|
</span>
|
||||||
|
<span className="text-xs text-destructive">Offline</span>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if (health) {
|
||||||
|
return (
|
||||||
|
<div className="flex items-center gap-2 rounded-full border border-accent/30 px-3 py-1">
|
||||||
|
<span className="relative flex h-2 w-2">
|
||||||
|
<span className="absolute inline-flex h-full w-full animate-ping rounded-full bg-accent/60" />
|
||||||
|
<span className="relative inline-flex h-2 w-2 rounded-full bg-accent shadow-[0_0_6px_1px_hsl(var(--accent)/0.5)]" />
|
||||||
|
</span>
|
||||||
|
<span className="text-xs text-muted-foreground">Online</span>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function UpdatesSection() {
|
||||||
|
const platform = usePlatform();
|
||||||
|
const { status, checkForUpdates, downloadAndInstall, restartAndInstall } = useAutoUpdater(false);
|
||||||
|
const [currentVersion, setCurrentVersion] = useState<string>('');
|
||||||
|
const isDev = !import.meta.env?.PROD;
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
platform.metadata
|
||||||
|
.getVersion()
|
||||||
|
.then(setCurrentVersion)
|
||||||
|
.catch(() => setCurrentVersion('Unknown'));
|
||||||
|
}, [platform]);
|
||||||
|
|
||||||
|
return (
|
||||||
|
<SettingSection title="App Updates" description={`v${currentVersion}${isDev ? ' (dev)' : ''}`}>
|
||||||
|
{isDev ? (
|
||||||
|
<SettingRow
|
||||||
|
title="Development mode"
|
||||||
|
description="Auto-updates are disabled in development mode."
|
||||||
|
/>
|
||||||
|
) : (
|
||||||
|
<>
|
||||||
|
<SettingRow
|
||||||
|
title="Check for updates"
|
||||||
|
description={
|
||||||
|
status.available
|
||||||
|
? `Version ${status.version} available`
|
||||||
|
: status.checking
|
||||||
|
? 'Checking...'
|
||||||
|
: "You're up to date"
|
||||||
|
}
|
||||||
|
action={
|
||||||
|
<Button
|
||||||
|
onClick={checkForUpdates}
|
||||||
|
disabled={status.checking || status.downloading || status.readyToInstall}
|
||||||
|
variant="outline"
|
||||||
|
size="sm"
|
||||||
|
>
|
||||||
|
<RefreshCw
|
||||||
|
className={`h-3.5 w-3.5 mr-1.5 ${status.checking ? 'animate-spin' : ''}`}
|
||||||
|
/>
|
||||||
|
Check
|
||||||
|
</Button>
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
|
||||||
|
{status.error && (
|
||||||
|
<SettingRow title="Update error">
|
||||||
|
<div className="flex items-center gap-2 text-sm text-destructive">
|
||||||
|
<AlertCircle className="h-4 w-4" />
|
||||||
|
{status.error}
|
||||||
|
</div>
|
||||||
|
</SettingRow>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{status.available && !status.downloading && !status.readyToInstall && (
|
||||||
|
<SettingRow
|
||||||
|
title={`Update to ${status.version}`}
|
||||||
|
description="Download and install the latest version."
|
||||||
|
action={
|
||||||
|
<Button onClick={downloadAndInstall} size="sm">
|
||||||
|
<Download className="h-3.5 w-3.5 mr-1.5" />
|
||||||
|
Download
|
||||||
|
</Button>
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{status.downloading && (
|
||||||
|
<SettingRow title="Downloading update...">
|
||||||
|
<div className="space-y-1.5">
|
||||||
|
<Progress value={status.downloadProgress} />
|
||||||
|
<div className="flex items-center justify-between text-xs text-muted-foreground">
|
||||||
|
{status.downloadedBytes !== undefined &&
|
||||||
|
status.totalBytes !== undefined &&
|
||||||
|
status.totalBytes > 0 ? (
|
||||||
|
<span>
|
||||||
|
{(status.downloadedBytes / 1024 / 1024).toFixed(1)} MB /{' '}
|
||||||
|
{(status.totalBytes / 1024 / 1024).toFixed(1)} MB
|
||||||
|
</span>
|
||||||
|
) : (
|
||||||
|
<span />
|
||||||
|
)}
|
||||||
|
{status.downloadProgress !== undefined && <span>{status.downloadProgress}%</span>}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</SettingRow>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{status.readyToInstall && (
|
||||||
|
<SettingRow
|
||||||
|
title="Update ready to install"
|
||||||
|
description={`Version ${status.version} has been downloaded. Restart to complete.`}
|
||||||
|
action={
|
||||||
|
<Button onClick={restartAndInstall} size="sm">
|
||||||
|
<RefreshCw className="h-3.5 w-3.5 mr-1.5" />
|
||||||
|
Restart Now
|
||||||
|
</Button>
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
</SettingSection>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const API_ENDPOINTS = [
|
||||||
|
{ method: 'POST', path: '/generate', label: 'Generate speech' },
|
||||||
|
{ method: 'GET', path: '/health', label: 'Server status' },
|
||||||
|
{ method: 'GET', path: '/profiles', label: 'List voices' },
|
||||||
|
{ method: 'GET', path: '/history', label: 'Past generations' },
|
||||||
|
];
|
||||||
|
|
||||||
|
function ApiReferenceCard({ serverUrl }: { serverUrl: string }) {
|
||||||
|
return (
|
||||||
|
<div className="rounded-lg border border-border/60 p-4 space-y-3">
|
||||||
|
<div>
|
||||||
|
<h3 className="text-sm font-medium">API Access</h3>
|
||||||
|
<p className="text-sm text-muted-foreground">
|
||||||
|
Integrate Voicebox into your workflow via the REST API at{' '}
|
||||||
|
<code className="text-xs bg-muted px-1 py-0.5 rounded font-mono">{serverUrl}</code>
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<div className="space-y-1">
|
||||||
|
{API_ENDPOINTS.map((ep) => (
|
||||||
|
<div key={ep.path} className="flex items-center gap-2.5 py-1">
|
||||||
|
<span
|
||||||
|
className={`text-[10px] font-mono font-semibold w-9 text-center rounded px-1 py-px ${
|
||||||
|
ep.method === 'POST' ? 'bg-accent/10 text-accent' : 'bg-muted text-muted-foreground'
|
||||||
|
}`}
|
||||||
|
>
|
||||||
|
{ep.method}
|
||||||
|
</span>
|
||||||
|
<code className="text-xs font-mono text-muted-foreground">{ep.path}</code>
|
||||||
|
<span className="text-xs text-muted-foreground/50 ml-auto">{ep.label}</span>
|
||||||
|
</div>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
<p className="text-xs text-muted-foreground">
|
||||||
|
<a
|
||||||
|
href={`${serverUrl}/docs`}
|
||||||
|
target="_blank"
|
||||||
|
rel="noopener noreferrer"
|
||||||
|
className="text-accent hover:underline"
|
||||||
|
>
|
||||||
|
View the full API reference
|
||||||
|
</a>
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,138 @@
|
|||||||
|
import { FolderOpen } from 'lucide-react';
|
||||||
|
import { useCallback, useEffect, useState } from 'react';
|
||||||
|
import { Button } from '@/components/ui/button';
|
||||||
|
import { Slider } from '@/components/ui/slider';
|
||||||
|
import { Toggle } from '@/components/ui/toggle';
|
||||||
|
import { usePlatform } from '@/platform/PlatformContext';
|
||||||
|
import { useServerStore } from '@/stores/serverStore';
|
||||||
|
import { SettingRow, SettingSection } from './SettingRow';
|
||||||
|
|
||||||
|
export function GenerationPage() {
|
||||||
|
const platform = usePlatform();
|
||||||
|
const serverUrl = useServerStore((state) => state.serverUrl);
|
||||||
|
const maxChunkChars = useServerStore((state) => state.maxChunkChars);
|
||||||
|
const setMaxChunkChars = useServerStore((state) => state.setMaxChunkChars);
|
||||||
|
const crossfadeMs = useServerStore((state) => state.crossfadeMs);
|
||||||
|
const setCrossfadeMs = useServerStore((state) => state.setCrossfadeMs);
|
||||||
|
const normalizeAudio = useServerStore((state) => state.normalizeAudio);
|
||||||
|
const setNormalizeAudio = useServerStore((state) => state.setNormalizeAudio);
|
||||||
|
const autoplayOnGenerate = useServerStore((state) => state.autoplayOnGenerate);
|
||||||
|
const setAutoplayOnGenerate = useServerStore((state) => state.setAutoplayOnGenerate);
|
||||||
|
const [opening, setOpening] = useState(false);
|
||||||
|
const [generationsPath, setGenerationsPath] = useState<string | null>(null);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
fetch(`${serverUrl}/health/filesystem`)
|
||||||
|
.then((res) => res.json())
|
||||||
|
.then((data) => {
|
||||||
|
const genDir = data.directories?.find((d: { path: string }) =>
|
||||||
|
d.path.includes('generations'),
|
||||||
|
);
|
||||||
|
if (genDir?.path) setGenerationsPath(genDir.path);
|
||||||
|
})
|
||||||
|
.catch(() => {});
|
||||||
|
}, [serverUrl]);
|
||||||
|
|
||||||
|
const openGenerationsFolder = useCallback(async () => {
|
||||||
|
if (!generationsPath) return;
|
||||||
|
setOpening(true);
|
||||||
|
try {
|
||||||
|
await platform.filesystem.openPath(generationsPath);
|
||||||
|
} catch (e) {
|
||||||
|
console.error('Failed to open generations folder:', e);
|
||||||
|
} finally {
|
||||||
|
setOpening(false);
|
||||||
|
}
|
||||||
|
}, [platform, generationsPath]);
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="space-y-8 max-w-2xl">
|
||||||
|
<SettingSection
|
||||||
|
title="Generation"
|
||||||
|
description="Controls for long text generation. These settings apply to all engines."
|
||||||
|
>
|
||||||
|
<SettingRow
|
||||||
|
title="Auto-chunking limit"
|
||||||
|
description="Long text is split into chunks at sentence boundaries. Lower values can improve quality for long outputs."
|
||||||
|
action={
|
||||||
|
<span className="text-sm tabular-nums text-muted-foreground">
|
||||||
|
{maxChunkChars} chars
|
||||||
|
</span>
|
||||||
|
}
|
||||||
|
>
|
||||||
|
<Slider
|
||||||
|
id="maxChunkChars"
|
||||||
|
value={[maxChunkChars]}
|
||||||
|
onValueChange={([value]) => setMaxChunkChars(value)}
|
||||||
|
min={100}
|
||||||
|
max={5000}
|
||||||
|
step={50}
|
||||||
|
aria-label="Auto-chunking character limit"
|
||||||
|
/>
|
||||||
|
</SettingRow>
|
||||||
|
|
||||||
|
<SettingRow
|
||||||
|
title="Chunk crossfade"
|
||||||
|
description="Blends audio between chunks to smooth transitions. Set to 0 for a hard cut."
|
||||||
|
action={
|
||||||
|
<span className="text-sm tabular-nums text-muted-foreground">
|
||||||
|
{crossfadeMs === 0 ? 'Cut' : `${crossfadeMs}ms`}
|
||||||
|
</span>
|
||||||
|
}
|
||||||
|
>
|
||||||
|
<Slider
|
||||||
|
id="crossfadeMs"
|
||||||
|
value={[crossfadeMs]}
|
||||||
|
onValueChange={([value]) => setCrossfadeMs(value)}
|
||||||
|
min={0}
|
||||||
|
max={200}
|
||||||
|
step={10}
|
||||||
|
aria-label="Chunk crossfade duration"
|
||||||
|
/>
|
||||||
|
</SettingRow>
|
||||||
|
|
||||||
|
<SettingRow
|
||||||
|
title="Normalize audio"
|
||||||
|
description="Adjusts output volume to a consistent level across generations."
|
||||||
|
htmlFor="normalizeAudio"
|
||||||
|
action={
|
||||||
|
<Toggle
|
||||||
|
id="normalizeAudio"
|
||||||
|
checked={normalizeAudio}
|
||||||
|
onCheckedChange={setNormalizeAudio}
|
||||||
|
/>
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
|
||||||
|
<SettingRow
|
||||||
|
title="Autoplay on generate"
|
||||||
|
description="Automatically play audio when a generation completes."
|
||||||
|
htmlFor="autoplayOnGenerate"
|
||||||
|
action={
|
||||||
|
<Toggle
|
||||||
|
id="autoplayOnGenerate"
|
||||||
|
checked={autoplayOnGenerate}
|
||||||
|
onCheckedChange={setAutoplayOnGenerate}
|
||||||
|
/>
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
|
||||||
|
<SettingRow
|
||||||
|
title="Generations folder"
|
||||||
|
description={generationsPath ?? 'Where generated audio files are stored on disk.'}
|
||||||
|
action={
|
||||||
|
<Button
|
||||||
|
variant="outline"
|
||||||
|
size="sm"
|
||||||
|
onClick={openGenerationsFolder}
|
||||||
|
disabled={opening || !generationsPath}
|
||||||
|
>
|
||||||
|
<FolderOpen className="h-3.5 w-3.5 mr-1.5" />
|
||||||
|
Open
|
||||||
|
</Button>
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
</SettingSection>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,405 @@
|
|||||||
|
import { useQuery, useQueryClient } from '@tanstack/react-query';
|
||||||
|
import { AlertCircle, Cpu, Download, Loader2, RotateCw, Trash2 } from 'lucide-react';
|
||||||
|
import { useCallback, useEffect, useRef, useState } from 'react';
|
||||||
|
import { Button } from '@/components/ui/button';
|
||||||
|
import { Progress } from '@/components/ui/progress';
|
||||||
|
import { apiClient } from '@/lib/api/client';
|
||||||
|
import type { CudaDownloadProgress, HealthResponse } from '@/lib/api/types';
|
||||||
|
import { useServerHealth } from '@/lib/hooks/useServer';
|
||||||
|
import { usePlatform } from '@/platform/PlatformContext';
|
||||||
|
import { useServerStore } from '@/stores/serverStore';
|
||||||
|
import { SettingRow, SettingSection } from './SettingRow';
|
||||||
|
|
||||||
|
type RestartPhase = 'idle' | 'stopping' | 'waiting' | 'ready';
|
||||||
|
|
||||||
|
function AppleLogo({ className }: { className?: string }) {
|
||||||
|
return (
|
||||||
|
<svg className={className} viewBox="0 0 24 24" fill="currentColor" aria-hidden="true">
|
||||||
|
<path d="M18.71 19.5c-.83 1.24-1.71 2.45-3.05 2.47-1.34.03-1.77-.79-3.29-.79-1.53 0-2 .77-3.27.82-1.31.05-2.3-1.32-3.14-2.53C4.25 17 2.94 12.45 4.7 9.39c.87-1.52 2.43-2.48 4.12-2.51 1.28-.02 2.5.87 3.29.87.78 0 2.26-1.07 3.8-.91.65.03 2.47.26 3.64 1.98-.09.06-2.17 1.28-2.15 3.81.03 3.02 2.65 4.03 2.68 4.04-.03.07-.42 1.44-1.38 2.83M13 3.5c.73-.83 1.94-1.46 2.94-1.5.13 1.17-.34 2.35-1.04 3.19-.69.85-1.83 1.51-2.95 1.42-.15-1.15.41-2.35 1.05-3.11z" />
|
||||||
|
</svg>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function GpuIcon({ className }: { className?: string }) {
|
||||||
|
return (
|
||||||
|
<svg
|
||||||
|
className={className}
|
||||||
|
viewBox="0 0 24 24"
|
||||||
|
fill="none"
|
||||||
|
stroke="currentColor"
|
||||||
|
strokeWidth="1.5"
|
||||||
|
strokeLinecap="round"
|
||||||
|
strokeLinejoin="round"
|
||||||
|
aria-hidden="true"
|
||||||
|
>
|
||||||
|
<rect x="4" y="6" width="16" height="12" rx="2" />
|
||||||
|
<path d="M2 10h2M2 14h2M20 10h2M20 14h2" />
|
||||||
|
<path d="M9 10h6M9 14h4" />
|
||||||
|
</svg>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function GpuInfoCard({ health }: { health: HealthResponse }) {
|
||||||
|
const hasGpu = health.gpu_available && health.gpu_type;
|
||||||
|
|
||||||
|
// Parse GPU name from type string like "CUDA (NVIDIA RTX 4090)" or "MPS (Apple M2 Pro)"
|
||||||
|
const gpuName = hasGpu
|
||||||
|
? health.gpu_type!.replace(/^(CUDA|ROCm|MPS|Metal|XPU|DirectML)\s*\((.+)\)$/, '$2') ||
|
||||||
|
health.gpu_type!
|
||||||
|
: null;
|
||||||
|
const gpuBackend = hasGpu ? health.gpu_type!.replace(/\s*\(.+\)$/, '') : null;
|
||||||
|
const isApple = gpuBackend === 'MPS' || gpuBackend === 'Metal';
|
||||||
|
const showBackendVariant = health.backend_variant && health.backend_variant !== 'cpu';
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="rounded-lg border border-border/60 p-4">
|
||||||
|
<div className="flex items-center gap-3">
|
||||||
|
{hasGpu ? (
|
||||||
|
isApple ? (
|
||||||
|
<AppleLogo className="h-5 w-5 shrink-0 text-muted-foreground" />
|
||||||
|
) : (
|
||||||
|
<GpuIcon className="h-5 w-5 shrink-0 text-accent" />
|
||||||
|
)
|
||||||
|
) : (
|
||||||
|
<Cpu className="h-5 w-5 shrink-0 text-muted-foreground" />
|
||||||
|
)}
|
||||||
|
<div className="flex-1 min-w-0 space-y-0.5">
|
||||||
|
<div className="text-sm font-medium">{hasGpu ? gpuName : 'CPU Only'}</div>
|
||||||
|
<div className="flex flex-wrap items-center gap-x-3 gap-y-1 text-xs text-muted-foreground">
|
||||||
|
{hasGpu ? (
|
||||||
|
<>
|
||||||
|
<span>{gpuBackend}</span>
|
||||||
|
{showBackendVariant && (
|
||||||
|
<>
|
||||||
|
<span className="text-border">|</span>
|
||||||
|
<span className="uppercase">{health.backend_variant}</span>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
{health.vram_used_mb != null && health.vram_used_mb > 0 && (
|
||||||
|
<>
|
||||||
|
<span className="text-border">|</span>
|
||||||
|
<span>{health.vram_used_mb.toFixed(0)} MB VRAM</span>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
</>
|
||||||
|
) : (
|
||||||
|
<span>No GPU acceleration detected</span>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{hasGpu && (
|
||||||
|
<div className="flex items-center gap-2 rounded-full border border-accent/30 px-2.5 py-0.5">
|
||||||
|
<span className="relative flex h-1.5 w-1.5">
|
||||||
|
<span className="absolute inline-flex h-full w-full animate-ping rounded-full bg-accent/60" />
|
||||||
|
<span className="relative inline-flex h-1.5 w-1.5 rounded-full bg-accent shadow-[0_0_4px_1px_hsl(var(--accent)/0.4)]" />
|
||||||
|
</span>
|
||||||
|
<span className="text-[10px] font-medium text-muted-foreground">Active</span>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function GpuPage() {
|
||||||
|
const platform = usePlatform();
|
||||||
|
const queryClient = useQueryClient();
|
||||||
|
const serverUrl = useServerStore((state) => state.serverUrl);
|
||||||
|
const { data: health } = useServerHealth();
|
||||||
|
|
||||||
|
const [restartPhase, setRestartPhase] = useState<RestartPhase>('idle');
|
||||||
|
const [error, setError] = useState<string | null>(null);
|
||||||
|
const [downloadProgress, setDownloadProgress] = useState<CudaDownloadProgress | null>(null);
|
||||||
|
const healthPollRef = useRef<ReturnType<typeof setInterval> | null>(null);
|
||||||
|
|
||||||
|
const {
|
||||||
|
data: cudaStatus,
|
||||||
|
isLoading: _cudaStatusLoading,
|
||||||
|
refetch: refetchCudaStatus,
|
||||||
|
} = useQuery({
|
||||||
|
queryKey: ['cuda-status', serverUrl],
|
||||||
|
queryFn: () => apiClient.getCudaStatus(),
|
||||||
|
refetchInterval: (query) => (query.state.status === 'pending' ? false : 10000),
|
||||||
|
retry: 1,
|
||||||
|
enabled: !!health,
|
||||||
|
});
|
||||||
|
|
||||||
|
const isCurrentlyCuda = health?.backend_variant === 'cuda';
|
||||||
|
const cudaAvailable = cudaStatus?.available ?? false;
|
||||||
|
const cudaDownloading = cudaStatus?.downloading ?? false;
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
return () => {
|
||||||
|
if (healthPollRef.current) {
|
||||||
|
clearInterval(healthPollRef.current);
|
||||||
|
healthPollRef.current = null;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (!cudaDownloading || !serverUrl) return;
|
||||||
|
|
||||||
|
const eventSource = new EventSource(`${serverUrl}/backend/cuda-progress`);
|
||||||
|
|
||||||
|
eventSource.onmessage = (event) => {
|
||||||
|
try {
|
||||||
|
const data = JSON.parse(event.data) as CudaDownloadProgress;
|
||||||
|
setDownloadProgress(data);
|
||||||
|
|
||||||
|
if (data.status === 'complete') {
|
||||||
|
eventSource.close();
|
||||||
|
setDownloadProgress(null);
|
||||||
|
refetchCudaStatus();
|
||||||
|
} else if (data.status === 'error') {
|
||||||
|
eventSource.close();
|
||||||
|
setError(data.error || 'Download failed');
|
||||||
|
setDownloadProgress(null);
|
||||||
|
refetchCudaStatus();
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
console.error('Error parsing CUDA progress event:', e);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
eventSource.onerror = () => {
|
||||||
|
eventSource.close();
|
||||||
|
};
|
||||||
|
|
||||||
|
return () => {
|
||||||
|
eventSource.close();
|
||||||
|
};
|
||||||
|
}, [cudaDownloading, serverUrl, refetchCudaStatus]);
|
||||||
|
|
||||||
|
const clearHealthPolling = useCallback(() => {
|
||||||
|
if (healthPollRef.current) {
|
||||||
|
clearInterval(healthPollRef.current);
|
||||||
|
healthPollRef.current = null;
|
||||||
|
}
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
const startHealthPolling = useCallback(() => {
|
||||||
|
clearHealthPolling();
|
||||||
|
|
||||||
|
healthPollRef.current = setInterval(async () => {
|
||||||
|
try {
|
||||||
|
const result = await apiClient.getHealth();
|
||||||
|
if (result.status === 'healthy') {
|
||||||
|
clearHealthPolling();
|
||||||
|
setRestartPhase('ready');
|
||||||
|
queryClient.invalidateQueries();
|
||||||
|
setTimeout(() => setRestartPhase('idle'), 2000);
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
// Server still down, keep polling
|
||||||
|
}
|
||||||
|
}, 1000);
|
||||||
|
}, [queryClient, clearHealthPolling]);
|
||||||
|
|
||||||
|
const restartServerWithPolling = useCallback(
|
||||||
|
async (errorMessage: string) => {
|
||||||
|
setRestartPhase('stopping');
|
||||||
|
try {
|
||||||
|
await platform.lifecycle.restartServer();
|
||||||
|
setRestartPhase('waiting');
|
||||||
|
startHealthPolling();
|
||||||
|
} catch (e: unknown) {
|
||||||
|
clearHealthPolling();
|
||||||
|
setRestartPhase('idle');
|
||||||
|
throw new Error(e instanceof Error ? e.message : errorMessage);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
[platform, startHealthPolling, clearHealthPolling],
|
||||||
|
);
|
||||||
|
|
||||||
|
const handleDownload = async () => {
|
||||||
|
setError(null);
|
||||||
|
try {
|
||||||
|
await apiClient.downloadCudaBackend();
|
||||||
|
refetchCudaStatus();
|
||||||
|
} catch (e: unknown) {
|
||||||
|
const msg = e instanceof Error ? e.message : 'Failed to start download';
|
||||||
|
if (msg.includes('already downloaded')) {
|
||||||
|
refetchCudaStatus();
|
||||||
|
} else {
|
||||||
|
setError(msg);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleRestart = async () => {
|
||||||
|
setError(null);
|
||||||
|
try {
|
||||||
|
await restartServerWithPolling('Restart failed');
|
||||||
|
} catch (e: unknown) {
|
||||||
|
setError(e instanceof Error ? e.message : 'Restart failed');
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleSwitchToCpu = async () => {
|
||||||
|
setError(null);
|
||||||
|
setRestartPhase('stopping');
|
||||||
|
try {
|
||||||
|
await apiClient.deleteCudaBackend();
|
||||||
|
await restartServerWithPolling('Failed to switch to CPU');
|
||||||
|
} catch (e: unknown) {
|
||||||
|
setError(e instanceof Error ? e.message : 'Failed to switch to CPU');
|
||||||
|
refetchCudaStatus();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleDelete = async () => {
|
||||||
|
setError(null);
|
||||||
|
try {
|
||||||
|
await apiClient.deleteCudaBackend();
|
||||||
|
refetchCudaStatus();
|
||||||
|
} catch (e: unknown) {
|
||||||
|
setError(e instanceof Error ? e.message : 'Failed to delete CUDA backend');
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const formatBytes = (bytes: number): string => {
|
||||||
|
if (bytes === 0) return '0 B';
|
||||||
|
const k = 1024;
|
||||||
|
const sizes = ['B', 'KB', 'MB', 'GB'];
|
||||||
|
const i = Math.floor(Math.log(bytes) / Math.log(k));
|
||||||
|
return `${(bytes / k ** i).toFixed(1)} ${sizes[i]}`;
|
||||||
|
};
|
||||||
|
|
||||||
|
if (!health) return null;
|
||||||
|
|
||||||
|
const hasNativeGpu =
|
||||||
|
health.gpu_available &&
|
||||||
|
!isCurrentlyCuda &&
|
||||||
|
health.gpu_type &&
|
||||||
|
!health.gpu_type.includes('CUDA');
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="space-y-8 max-w-2xl">
|
||||||
|
<GpuInfoCard health={health} />
|
||||||
|
|
||||||
|
{/* CUDA section — only when no native GPU and not already on CUDA */}
|
||||||
|
{!hasNativeGpu && !isCurrentlyCuda && (
|
||||||
|
<SettingSection
|
||||||
|
title="CUDA Backend"
|
||||||
|
description="NVIDIA GPU acceleration via a downloadable CUDA backend."
|
||||||
|
>
|
||||||
|
{/* Download progress */}
|
||||||
|
{cudaDownloading && downloadProgress && (
|
||||||
|
<SettingRow title="Downloading CUDA backend...">
|
||||||
|
<div className="space-y-1.5">
|
||||||
|
<Progress value={downloadProgress.progress} className="h-2" />
|
||||||
|
<div className="flex items-center justify-between text-xs text-muted-foreground">
|
||||||
|
<span>
|
||||||
|
{downloadProgress.filename ||
|
||||||
|
(cudaAvailable ? 'Updating...' : 'Downloading...')}
|
||||||
|
</span>
|
||||||
|
<span>
|
||||||
|
{downloadProgress.total > 0
|
||||||
|
? `${formatBytes(downloadProgress.current)} / ${formatBytes(downloadProgress.total)}`
|
||||||
|
: `${downloadProgress.progress.toFixed(1)}%`}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</SettingRow>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Restart in progress */}
|
||||||
|
{restartPhase !== 'idle' && (
|
||||||
|
<SettingRow
|
||||||
|
title={
|
||||||
|
restartPhase === 'ready'
|
||||||
|
? 'Server restarted successfully'
|
||||||
|
: restartPhase === 'waiting'
|
||||||
|
? 'Restarting server...'
|
||||||
|
: 'Stopping server...'
|
||||||
|
}
|
||||||
|
action={<Loader2 className="h-4 w-4 animate-spin text-muted-foreground" />}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Error */}
|
||||||
|
{error && (
|
||||||
|
<SettingRow title="Error">
|
||||||
|
<div className="flex items-center gap-2 text-sm text-destructive">
|
||||||
|
<AlertCircle className="h-4 w-4 shrink-0" />
|
||||||
|
<span>{error}</span>
|
||||||
|
</div>
|
||||||
|
</SettingRow>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Actions */}
|
||||||
|
{restartPhase === 'idle' && !cudaDownloading && (
|
||||||
|
<>
|
||||||
|
{!cudaAvailable && !isCurrentlyCuda && (
|
||||||
|
<SettingRow
|
||||||
|
title="Download CUDA backend"
|
||||||
|
description="~2.4 GB download. Requires an NVIDIA GPU with CUDA support."
|
||||||
|
action={
|
||||||
|
<Button onClick={handleDownload} size="sm">
|
||||||
|
<Download className="h-3.5 w-3.5 mr-1.5" />
|
||||||
|
Download
|
||||||
|
</Button>
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{cudaAvailable && !isCurrentlyCuda && platform.metadata.isTauri && (
|
||||||
|
<SettingRow
|
||||||
|
title="Switch to CUDA backend"
|
||||||
|
description="CUDA backend is downloaded and ready. Restart to enable."
|
||||||
|
action={
|
||||||
|
<Button onClick={handleRestart} size="sm">
|
||||||
|
<RotateCw className="h-3.5 w-3.5 mr-1.5" />
|
||||||
|
Restart
|
||||||
|
</Button>
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{isCurrentlyCuda && platform.metadata.isTauri && (
|
||||||
|
<SettingRow
|
||||||
|
title="Switch to CPU backend"
|
||||||
|
description="Disable GPU acceleration. You can re-download CUDA later."
|
||||||
|
action={
|
||||||
|
<Button onClick={handleSwitchToCpu} variant="outline" size="sm">
|
||||||
|
<RotateCw className="h-3.5 w-3.5 mr-1.5" />
|
||||||
|
Switch
|
||||||
|
</Button>
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{cudaAvailable && !isCurrentlyCuda && (
|
||||||
|
<SettingRow
|
||||||
|
title="Remove CUDA backend"
|
||||||
|
description="Delete the downloaded CUDA binary to free disk space."
|
||||||
|
action={
|
||||||
|
<Button
|
||||||
|
onClick={handleDelete}
|
||||||
|
variant="ghost"
|
||||||
|
size="sm"
|
||||||
|
className="text-muted-foreground hover:text-destructive"
|
||||||
|
>
|
||||||
|
<Trash2 className="h-3.5 w-3.5 mr-1.5" />
|
||||||
|
Remove
|
||||||
|
</Button>
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
</SettingSection>
|
||||||
|
)}
|
||||||
|
|
||||||
|
<p className="text-xs text-muted-foreground/60 leading-relaxed">
|
||||||
|
Voicebox automatically detects and uses the best available GPU on your system. On Apple
|
||||||
|
Silicon Macs, the MLX backend runs natively on the Neural Engine and GPU via Metal
|
||||||
|
Performance Shaders (MPS), with no additional setup required. On Windows and Linux with
|
||||||
|
NVIDIA GPUs, you can download an optional CUDA backend for hardware-accelerated inference.
|
||||||
|
AMD ROCm, Intel XPU, and DirectML are also supported where available through PyTorch. When
|
||||||
|
no GPU is detected, Voicebox falls back to CPU — all engines still work, just slower.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,104 @@
|
|||||||
|
import { useEffect, useRef, useState } from 'react';
|
||||||
|
import { Button } from '@/components/ui/button';
|
||||||
|
import { cn } from '@/lib/utils/cn';
|
||||||
|
import { type LogEntry, useLogStore } from '@/stores/logStore';
|
||||||
|
|
||||||
|
function formatTime(timestamp: number): string {
|
||||||
|
const d = new Date(timestamp);
|
||||||
|
return d.toLocaleTimeString(undefined, {
|
||||||
|
hour: '2-digit',
|
||||||
|
minute: '2-digit',
|
||||||
|
second: '2-digit',
|
||||||
|
hour12: false,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function LogLine({ entry }: { entry: LogEntry }) {
|
||||||
|
return (
|
||||||
|
<div className="flex gap-3 font-mono text-xs leading-5 hover:bg-muted/30">
|
||||||
|
<span className="text-muted-foreground/50 select-none shrink-0">
|
||||||
|
{formatTime(entry.timestamp)}
|
||||||
|
</span>
|
||||||
|
<span
|
||||||
|
className={cn(
|
||||||
|
'whitespace-pre-wrap break-all',
|
||||||
|
entry.stream === 'stderr' ? 'text-orange-400/80' : 'text-muted-foreground',
|
||||||
|
)}
|
||||||
|
>
|
||||||
|
{entry.line}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function LogsPage() {
|
||||||
|
const entries = useLogStore((s) => s.entries);
|
||||||
|
const clear = useLogStore((s) => s.clear);
|
||||||
|
const containerRef = useRef<HTMLDivElement>(null);
|
||||||
|
const [autoScroll, setAutoScroll] = useState(true);
|
||||||
|
|
||||||
|
// Auto-scroll to bottom when new entries arrive
|
||||||
|
useEffect(() => {
|
||||||
|
if (autoScroll && containerRef.current) {
|
||||||
|
containerRef.current.scrollTop = containerRef.current.scrollHeight;
|
||||||
|
}
|
||||||
|
}, [entries.length, autoScroll]);
|
||||||
|
|
||||||
|
// Detect manual scroll to disable auto-scroll
|
||||||
|
const handleScroll = () => {
|
||||||
|
const el = containerRef.current;
|
||||||
|
if (!el) return;
|
||||||
|
const atBottom = el.scrollHeight - el.scrollTop - el.clientHeight < 40;
|
||||||
|
setAutoScroll(atBottom);
|
||||||
|
};
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="flex flex-col h-full min-h-0">
|
||||||
|
<div className="flex items-center justify-between mb-3">
|
||||||
|
<div>
|
||||||
|
<h3 className="text-sm font-medium">Server Logs</h3>
|
||||||
|
<p className="text-sm text-muted-foreground">
|
||||||
|
{entries.length} {entries.length === 1 ? 'line' : 'lines'}
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<div className="flex items-center gap-2">
|
||||||
|
{!autoScroll && (
|
||||||
|
<Button
|
||||||
|
variant="outline"
|
||||||
|
size="sm"
|
||||||
|
onClick={() => {
|
||||||
|
setAutoScroll(true);
|
||||||
|
containerRef.current?.scrollTo({ top: containerRef.current.scrollHeight });
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
Scroll to bottom
|
||||||
|
</Button>
|
||||||
|
)}
|
||||||
|
<Button variant="outline" size="sm" onClick={clear}>
|
||||||
|
Clear
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div
|
||||||
|
ref={containerRef}
|
||||||
|
onScroll={handleScroll}
|
||||||
|
className="flex-1 min-h-0 overflow-y-auto rounded-md border bg-black/20 p-3"
|
||||||
|
>
|
||||||
|
{entries.length === 0 ? (
|
||||||
|
<div className="text-sm text-muted-foreground/50 font-mono space-y-1">
|
||||||
|
<p>No log output yet.</p>
|
||||||
|
{!import.meta.env?.PROD && (
|
||||||
|
<p>
|
||||||
|
Server logs are only captured when the app manages the server process (production
|
||||||
|
builds).
|
||||||
|
</p>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
) : (
|
||||||
|
entries.map((entry) => <LogLine key={entry.id} entry={entry} />)
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -1,35 +1,70 @@
|
|||||||
import { ConnectionForm } from '@/components/ServerSettings/ConnectionForm';
|
import { Link, Outlet, useMatchRoute } from '@tanstack/react-router';
|
||||||
import { GenerationSettings } from '@/components/ServerSettings/GenerationSettings';
|
|
||||||
import { GpuAcceleration } from '@/components/ServerSettings/GpuAcceleration';
|
|
||||||
import { UpdateStatus } from '@/components/ServerSettings/UpdateStatus';
|
|
||||||
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
|
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
|
||||||
import { cn } from '@/lib/utils/cn';
|
import { cn } from '@/lib/utils/cn';
|
||||||
import { usePlatform } from '@/platform/PlatformContext';
|
import { usePlatform } from '@/platform/PlatformContext';
|
||||||
import { usePlayerStore } from '@/stores/playerStore';
|
import { usePlayerStore } from '@/stores/playerStore';
|
||||||
|
|
||||||
export function ServerTab() {
|
interface SettingsTab {
|
||||||
|
label: string;
|
||||||
|
path:
|
||||||
|
| '/settings'
|
||||||
|
| '/settings/generation'
|
||||||
|
| '/settings/gpu'
|
||||||
|
| '/settings/logs'
|
||||||
|
| '/settings/changelog'
|
||||||
|
| '/settings/about';
|
||||||
|
tauriOnly?: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
const tabs: SettingsTab[] = [
|
||||||
|
{ label: 'General', path: '/settings' },
|
||||||
|
{ label: 'Generation', path: '/settings/generation' },
|
||||||
|
{ label: 'GPU', path: '/settings/gpu', tauriOnly: true },
|
||||||
|
{ label: 'Logs', path: '/settings/logs', tauriOnly: true },
|
||||||
|
{ label: 'Changelog', path: '/settings/changelog' },
|
||||||
|
{ label: 'About', path: '/settings/about' },
|
||||||
|
];
|
||||||
|
|
||||||
|
export function SettingsLayout() {
|
||||||
const platform = usePlatform();
|
const platform = usePlatform();
|
||||||
const isPlayerVisible = !!usePlayerStore((state) => state.audioUrl);
|
const isPlayerVisible = !!usePlayerStore((state) => state.audioUrl);
|
||||||
|
const matchRoute = useMatchRoute();
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div
|
<div className="flex flex-col h-full min-h-0">
|
||||||
className={cn('overflow-y-auto flex flex-col', isPlayerVisible && BOTTOM_SAFE_AREA_PADDING)}
|
<nav className="flex gap-1 border-b shrink-0">
|
||||||
>
|
{tabs.map((tab) => {
|
||||||
<div className="grid gap-4 md:grid-cols-2">
|
if (tab.tauriOnly && !platform.metadata.isTauri) return null;
|
||||||
<ConnectionForm />
|
|
||||||
<GenerationSettings />
|
const isActive =
|
||||||
{platform.metadata.isTauri && <GpuAcceleration />}
|
tab.path === '/settings'
|
||||||
{platform.metadata.isTauri && <UpdateStatus />}
|
? matchRoute({ to: tab.path, fuzzy: false })
|
||||||
</div>
|
: matchRoute({ to: tab.path });
|
||||||
<div className="py-8 text-center text-sm text-muted-foreground">
|
|
||||||
Created by{' '}
|
return (
|
||||||
<a
|
<Link
|
||||||
href="https://github.com/jamiepine"
|
key={tab.path}
|
||||||
target="_blank"
|
to={tab.path}
|
||||||
rel="noopener noreferrer"
|
className={cn(
|
||||||
className="text-accent hover:underline"
|
'px-4 py-2 text-sm font-medium transition-colors border-b-2 -mb-px',
|
||||||
>
|
isActive
|
||||||
Jamie Pine
|
? 'border-accent text-foreground'
|
||||||
</a>
|
: 'border-transparent text-muted-foreground hover:text-foreground hover:border-muted-foreground/30',
|
||||||
|
)}
|
||||||
|
>
|
||||||
|
{tab.label}
|
||||||
|
</Link>
|
||||||
|
);
|
||||||
|
})}
|
||||||
|
</nav>
|
||||||
|
|
||||||
|
<div
|
||||||
|
className={cn(
|
||||||
|
'flex-1 overflow-y-auto pt-6 pb-6 px-2 -mx-2',
|
||||||
|
isPlayerVisible && BOTTOM_SAFE_AREA_PADDING,
|
||||||
|
)}
|
||||||
|
>
|
||||||
|
<Outlet />
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -0,0 +1,62 @@
|
|||||||
|
import type { ReactNode } from 'react';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A section header with title and optional description, separated by a border.
|
||||||
|
*/
|
||||||
|
export function SettingSection({
|
||||||
|
title,
|
||||||
|
description,
|
||||||
|
children,
|
||||||
|
}: {
|
||||||
|
title?: string;
|
||||||
|
description?: string;
|
||||||
|
children: ReactNode;
|
||||||
|
}) {
|
||||||
|
return (
|
||||||
|
<div className="space-y-1">
|
||||||
|
{title && <h3 className="text-sm font-medium">{title}</h3>}
|
||||||
|
{description && <p className="text-sm text-muted-foreground">{description}</p>}
|
||||||
|
<div className={`${title || description ? 'pt-3' : ''} space-y-0 divide-y divide-border/60`}>
|
||||||
|
{children}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A single settings row: label+description on the left, action on the right.
|
||||||
|
* Use for toggles, inputs, buttons, badges — any control type.
|
||||||
|
*/
|
||||||
|
export function SettingRow({
|
||||||
|
title,
|
||||||
|
description,
|
||||||
|
htmlFor,
|
||||||
|
action,
|
||||||
|
children,
|
||||||
|
}: {
|
||||||
|
title: string;
|
||||||
|
description?: string;
|
||||||
|
htmlFor?: string;
|
||||||
|
/** Right-aligned control (checkbox, button, badge, etc.) */
|
||||||
|
action?: ReactNode;
|
||||||
|
/** Full-width content rendered below the label row (for sliders, inputs, etc.) */
|
||||||
|
children?: ReactNode;
|
||||||
|
}) {
|
||||||
|
return (
|
||||||
|
<div className="py-3">
|
||||||
|
<div className="flex items-center justify-between gap-8">
|
||||||
|
<div className="min-w-0">
|
||||||
|
<label
|
||||||
|
htmlFor={htmlFor}
|
||||||
|
className={`text-sm font-medium leading-none select-none ${htmlFor ? 'cursor-pointer' : ''}`}
|
||||||
|
>
|
||||||
|
{title}
|
||||||
|
</label>
|
||||||
|
{description && <p className="text-sm text-muted-foreground mt-0.5">{description}</p>}
|
||||||
|
</div>
|
||||||
|
{action && <div className="shrink-0">{action}</div>}
|
||||||
|
</div>
|
||||||
|
{children && <div className="mt-3">{children}</div>}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
import { Link, useMatchRoute } from '@tanstack/react-router';
|
import { Link, useMatchRoute } from '@tanstack/react-router';
|
||||||
import { AudioLines, Box, Mic, Server, Speaker, Volume2, Wand2 } from 'lucide-react';
|
import { AudioLines, Box, Mic, Settings, Speaker, Volume2, Wand2 } from 'lucide-react';
|
||||||
import { useEffect, useState } from 'react';
|
import { useEffect, useState } from 'react';
|
||||||
import voiceboxLogo from '@/assets/voicebox-logo.png';
|
import voiceboxLogo from '@/assets/voicebox-logo.png';
|
||||||
import { cn } from '@/lib/utils/cn';
|
import { cn } from '@/lib/utils/cn';
|
||||||
@@ -19,7 +19,7 @@ const tabs = [
|
|||||||
{ id: 'effects', path: '/effects', icon: Wand2, label: 'Effects' },
|
{ id: 'effects', path: '/effects', icon: Wand2, label: 'Effects' },
|
||||||
{ id: 'audio', path: '/audio', icon: Speaker, label: 'Audio' },
|
{ id: 'audio', path: '/audio', icon: Speaker, label: 'Audio' },
|
||||||
{ id: 'models', path: '/models', icon: Box, label: 'Models' },
|
{ id: 'models', path: '/models', icon: Box, label: 'Models' },
|
||||||
{ id: 'server', path: '/server', icon: Server, label: 'Server' },
|
{ id: 'settings', path: '/settings', icon: Settings, label: 'Settings' },
|
||||||
];
|
];
|
||||||
|
|
||||||
export function Sidebar({ isMacOS }: SidebarProps) {
|
export function Sidebar({ isMacOS }: SidebarProps) {
|
||||||
@@ -54,9 +54,10 @@ export function Sidebar({ isMacOS }: SidebarProps) {
|
|||||||
<div className="flex flex-col gap-3">
|
<div className="flex flex-col gap-3">
|
||||||
{tabs.map((tab, index) => {
|
{tabs.map((tab, index) => {
|
||||||
const Icon = tab.icon;
|
const Icon = tab.icon;
|
||||||
// For index route, use exact match; for others, use default matching
|
|
||||||
const isActive =
|
const isActive =
|
||||||
tab.path === '/' ? matchRoute({ to: '/', exact: true }) : matchRoute({ to: tab.path });
|
tab.path === '/'
|
||||||
|
? matchRoute({ to: '/', fuzzy: false })
|
||||||
|
: matchRoute({ to: tab.path, fuzzy: true });
|
||||||
|
|
||||||
// Accent fades as buttons get further from the logo
|
// Accent fades as buttons get further from the logo
|
||||||
const accentOpacity = Math.max(0.08, 0.5 - index * 0.07);
|
const accentOpacity = Math.max(0.08, 0.5 - index * 0.07);
|
||||||
@@ -98,7 +99,7 @@ export function Sidebar({ isMacOS }: SidebarProps) {
|
|||||||
<span className="text-[10px] text-muted-foreground/50">v{version}</span>
|
<span className="text-[10px] text-muted-foreground/50">v{version}</span>
|
||||||
{updateStatus.available && (
|
{updateStatus.available && (
|
||||||
<Link
|
<Link
|
||||||
to="/server"
|
to="/settings"
|
||||||
className="text-[9px] font-semibold tracking-wide uppercase px-2 py-0.5 rounded-full bg-accent/15 text-accent hover:bg-accent/25 transition-colors"
|
className="text-[9px] font-semibold tracking-wide uppercase px-2 py-0.5 rounded-full bg-accent/15 text-accent hover:bg-accent/25 transition-colors"
|
||||||
>
|
>
|
||||||
Update
|
Update
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
import * as React from 'react';
|
|
||||||
import * as SliderPrimitive from '@radix-ui/react-slider';
|
import * as SliderPrimitive from '@radix-ui/react-slider';
|
||||||
|
import * as React from 'react';
|
||||||
import { cn } from '@/lib/utils/cn';
|
import { cn } from '@/lib/utils/cn';
|
||||||
|
|
||||||
const Slider = React.forwardRef<
|
const Slider = React.forwardRef<
|
||||||
@@ -14,7 +14,7 @@ const Slider = React.forwardRef<
|
|||||||
<SliderPrimitive.Track className="relative h-2 w-full grow overflow-hidden rounded-full bg-secondary">
|
<SliderPrimitive.Track className="relative h-2 w-full grow overflow-hidden rounded-full bg-secondary">
|
||||||
<SliderPrimitive.Range className="absolute h-full bg-primary" />
|
<SliderPrimitive.Range className="absolute h-full bg-primary" />
|
||||||
</SliderPrimitive.Track>
|
</SliderPrimitive.Track>
|
||||||
<SliderPrimitive.Thumb className="block h-5 w-5 rounded-full border-2 border-primary bg-background ring-offset-background transition-colors focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:pointer-events-none disabled:opacity-50 translate-x-0.5" />
|
<SliderPrimitive.Thumb className="block h-0 w-0 outline-none disabled:pointer-events-none disabled:opacity-50 after:block after:h-5 after:w-5 after:rounded-full after:border-2 after:border-primary after:bg-background after:ring-offset-background after:transition-colors after:absolute after:top-1/2 after:left-1/2 after:-translate-x-1/2 after:-translate-y-1/2 focus-visible:after:ring-2 focus-visible:after:ring-ring focus-visible:after:ring-offset-2" />
|
||||||
</SliderPrimitive.Root>
|
</SliderPrimitive.Root>
|
||||||
));
|
));
|
||||||
Slider.displayName = SliderPrimitive.Root.displayName;
|
Slider.displayName = SliderPrimitive.Root.displayName;
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { usePlayerStore } from '@/stores/playerStore';
|
||||||
import {
|
import {
|
||||||
Toast,
|
Toast,
|
||||||
ToastClose,
|
ToastClose,
|
||||||
@@ -10,6 +11,7 @@ import { useToast } from './use-toast';
|
|||||||
|
|
||||||
export function Toaster() {
|
export function Toaster() {
|
||||||
const { toasts } = useToast();
|
const { toasts } = useToast();
|
||||||
|
const isPlayerOpen = !!usePlayerStore((s) => s.audioUrl);
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<ToastProvider>
|
<ToastProvider>
|
||||||
@@ -23,7 +25,7 @@ export function Toaster() {
|
|||||||
<ToastClose />
|
<ToastClose />
|
||||||
</Toast>
|
</Toast>
|
||||||
))}
|
))}
|
||||||
<ToastViewport />
|
<ToastViewport className={isPlayerOpen ? 'sm:bottom-44' : ''} />
|
||||||
</ToastProvider>
|
</ToastProvider>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,48 @@
|
|||||||
|
import * as React from 'react';
|
||||||
|
import { cn } from '@/lib/utils/cn';
|
||||||
|
|
||||||
|
export interface ToggleProps {
|
||||||
|
checked?: boolean;
|
||||||
|
onCheckedChange?: (checked: boolean) => void;
|
||||||
|
disabled?: boolean;
|
||||||
|
className?: string;
|
||||||
|
id?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const Toggle = React.forwardRef<HTMLButtonElement, ToggleProps>(
|
||||||
|
({ checked = false, onCheckedChange, disabled = false, className, id, ...props }, ref) => {
|
||||||
|
return (
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
ref={ref}
|
||||||
|
id={id}
|
||||||
|
role="switch"
|
||||||
|
aria-checked={checked}
|
||||||
|
disabled={disabled}
|
||||||
|
onClick={() => {
|
||||||
|
if (!disabled && onCheckedChange) {
|
||||||
|
onCheckedChange(!checked);
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
className={cn(
|
||||||
|
'relative inline-flex h-5 w-9 shrink-0 items-center rounded-full transition-colors',
|
||||||
|
'focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2',
|
||||||
|
checked ? 'bg-accent' : 'bg-muted-foreground/25',
|
||||||
|
disabled ? 'opacity-50 cursor-not-allowed' : 'cursor-pointer',
|
||||||
|
className,
|
||||||
|
)}
|
||||||
|
{...props}
|
||||||
|
>
|
||||||
|
<span
|
||||||
|
className={cn(
|
||||||
|
'pointer-events-none block h-4 w-4 rounded-full bg-white shadow-sm transition-transform',
|
||||||
|
checked ? 'translate-x-[18px]' : 'translate-x-[2px]',
|
||||||
|
)}
|
||||||
|
/>
|
||||||
|
</button>
|
||||||
|
);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
Toggle.displayName = 'Toggle';
|
||||||
|
|
||||||
|
export { Toggle };
|
||||||
Vendored
+5
@@ -1,3 +1,8 @@
|
|||||||
interface Window {
|
interface Window {
|
||||||
__voiceboxServerStartedByApp?: boolean;
|
__voiceboxServerStartedByApp?: boolean;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
declare module 'virtual:changelog' {
|
||||||
|
const raw: string;
|
||||||
|
export default raw;
|
||||||
|
}
|
||||||
|
|||||||
+35
-12
@@ -32,8 +32,24 @@ import type {
|
|||||||
TranscriptionResponse,
|
TranscriptionResponse,
|
||||||
VoiceProfileCreate,
|
VoiceProfileCreate,
|
||||||
VoiceProfileResponse,
|
VoiceProfileResponse,
|
||||||
|
WhisperModelSize,
|
||||||
} from './types';
|
} from './types';
|
||||||
|
|
||||||
|
function formatErrorDetail(detail: unknown, fallback: string): string {
|
||||||
|
if (typeof detail === 'string') return detail;
|
||||||
|
if (Array.isArray(detail)) {
|
||||||
|
return detail
|
||||||
|
.map((e: Record<string, unknown>) => e.msg || e.message || JSON.stringify(e))
|
||||||
|
.join('; ');
|
||||||
|
}
|
||||||
|
if (detail && typeof detail === 'object') {
|
||||||
|
const obj = detail as Record<string, unknown>;
|
||||||
|
if (typeof obj.message === 'string') return obj.message;
|
||||||
|
return JSON.stringify(detail);
|
||||||
|
}
|
||||||
|
return fallback;
|
||||||
|
}
|
||||||
|
|
||||||
class ApiClient {
|
class ApiClient {
|
||||||
private getBaseUrl(): string {
|
private getBaseUrl(): string {
|
||||||
const serverUrl = useServerStore.getState().serverUrl;
|
const serverUrl = useServerStore.getState().serverUrl;
|
||||||
@@ -54,7 +70,7 @@ class ApiClient {
|
|||||||
const error = await response.json().catch(() => ({
|
const error = await response.json().catch(() => ({
|
||||||
detail: response.statusText,
|
detail: response.statusText,
|
||||||
}));
|
}));
|
||||||
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
|
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
|
||||||
}
|
}
|
||||||
|
|
||||||
return response.json();
|
return response.json();
|
||||||
@@ -113,7 +129,7 @@ class ApiClient {
|
|||||||
const error = await response.json().catch(() => ({
|
const error = await response.json().catch(() => ({
|
||||||
detail: response.statusText,
|
detail: response.statusText,
|
||||||
}));
|
}));
|
||||||
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
|
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
|
||||||
}
|
}
|
||||||
|
|
||||||
return response.json();
|
return response.json();
|
||||||
@@ -147,7 +163,7 @@ class ApiClient {
|
|||||||
const error = await response.json().catch(() => ({
|
const error = await response.json().catch(() => ({
|
||||||
detail: response.statusText,
|
detail: response.statusText,
|
||||||
}));
|
}));
|
||||||
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
|
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
|
||||||
}
|
}
|
||||||
|
|
||||||
return response.blob();
|
return response.blob();
|
||||||
@@ -167,7 +183,7 @@ class ApiClient {
|
|||||||
const error = await response.json().catch(() => ({
|
const error = await response.json().catch(() => ({
|
||||||
detail: response.statusText,
|
detail: response.statusText,
|
||||||
}));
|
}));
|
||||||
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
|
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
|
||||||
}
|
}
|
||||||
|
|
||||||
return response.json();
|
return response.json();
|
||||||
@@ -187,7 +203,7 @@ class ApiClient {
|
|||||||
const error = await response.json().catch(() => ({
|
const error = await response.json().catch(() => ({
|
||||||
detail: response.statusText,
|
detail: response.statusText,
|
||||||
}));
|
}));
|
||||||
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
|
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
|
||||||
}
|
}
|
||||||
|
|
||||||
return response.json();
|
return response.json();
|
||||||
@@ -257,7 +273,7 @@ class ApiClient {
|
|||||||
const error = await response.json().catch(() => ({
|
const error = await response.json().catch(() => ({
|
||||||
detail: response.statusText,
|
detail: response.statusText,
|
||||||
}));
|
}));
|
||||||
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
|
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
|
||||||
}
|
}
|
||||||
|
|
||||||
return response.blob();
|
return response.blob();
|
||||||
@@ -271,7 +287,7 @@ class ApiClient {
|
|||||||
const error = await response.json().catch(() => ({
|
const error = await response.json().catch(() => ({
|
||||||
detail: response.statusText,
|
detail: response.statusText,
|
||||||
}));
|
}));
|
||||||
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
|
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
|
||||||
}
|
}
|
||||||
|
|
||||||
return response.blob();
|
return response.blob();
|
||||||
@@ -297,7 +313,7 @@ class ApiClient {
|
|||||||
const error = await response.json().catch(() => ({
|
const error = await response.json().catch(() => ({
|
||||||
detail: response.statusText,
|
detail: response.statusText,
|
||||||
}));
|
}));
|
||||||
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
|
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
|
||||||
}
|
}
|
||||||
|
|
||||||
return response.json();
|
return response.json();
|
||||||
@@ -318,12 +334,19 @@ class ApiClient {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Transcription
|
// Transcription
|
||||||
async transcribeAudio(file: File, language?: LanguageCode): Promise<TranscriptionResponse> {
|
async transcribeAudio(
|
||||||
|
file: File,
|
||||||
|
language?: LanguageCode,
|
||||||
|
model?: WhisperModelSize,
|
||||||
|
): Promise<TranscriptionResponse> {
|
||||||
const formData = new FormData();
|
const formData = new FormData();
|
||||||
formData.append('file', file);
|
formData.append('file', file);
|
||||||
if (language) {
|
if (language) {
|
||||||
formData.append('language', language);
|
formData.append('language', language);
|
||||||
}
|
}
|
||||||
|
if (model) {
|
||||||
|
formData.append('model', model);
|
||||||
|
}
|
||||||
|
|
||||||
const url = `${this.getBaseUrl()}/transcribe`;
|
const url = `${this.getBaseUrl()}/transcribe`;
|
||||||
const response = await fetch(url, {
|
const response = await fetch(url, {
|
||||||
@@ -335,7 +358,7 @@ class ApiClient {
|
|||||||
const error = await response.json().catch(() => ({
|
const error = await response.json().catch(() => ({
|
||||||
detail: response.statusText,
|
detail: response.statusText,
|
||||||
}));
|
}));
|
||||||
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
|
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
|
||||||
}
|
}
|
||||||
|
|
||||||
return response.json();
|
return response.json();
|
||||||
@@ -608,7 +631,7 @@ class ApiClient {
|
|||||||
const error = await response.json().catch(() => ({
|
const error = await response.json().catch(() => ({
|
||||||
detail: response.statusText,
|
detail: response.statusText,
|
||||||
}));
|
}));
|
||||||
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
|
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
|
||||||
}
|
}
|
||||||
|
|
||||||
return response.blob();
|
return response.blob();
|
||||||
@@ -705,7 +728,7 @@ class ApiClient {
|
|||||||
const error = await response.json().catch(() => ({
|
const error = await response.json().catch(() => ({
|
||||||
detail: response.statusText,
|
detail: response.statusText,
|
||||||
}));
|
}));
|
||||||
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
|
throw new Error(formatErrorDetail(error.detail, `HTTP error! status: ${response.status}`));
|
||||||
}
|
}
|
||||||
|
|
||||||
return response.blob();
|
return response.blob();
|
||||||
|
|||||||
@@ -42,8 +42,8 @@ export interface GenerationRequest {
|
|||||||
text: string;
|
text: string;
|
||||||
language: LanguageCode;
|
language: LanguageCode;
|
||||||
seed?: number;
|
seed?: number;
|
||||||
model_size?: '1.7B' | '0.6B';
|
model_size?: '1.7B' | '0.6B' | '1B' | '3B';
|
||||||
engine?: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo';
|
engine?: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo' | 'tada';
|
||||||
instruct?: string;
|
instruct?: string;
|
||||||
max_chunk_chars?: number;
|
max_chunk_chars?: number;
|
||||||
crossfade_ms?: number;
|
crossfade_ms?: number;
|
||||||
@@ -99,8 +99,11 @@ export interface HistoryListResponse {
|
|||||||
total: number;
|
total: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export type WhisperModelSize = 'base' | 'small' | 'medium' | 'large' | 'turbo';
|
||||||
|
|
||||||
export interface TranscriptionRequest {
|
export interface TranscriptionRequest {
|
||||||
language?: LanguageCode;
|
language?: LanguageCode;
|
||||||
|
model?: WhisperModelSize;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface TranscriptionResponse {
|
export interface TranscriptionResponse {
|
||||||
|
|||||||
@@ -66,6 +66,7 @@ export const ENGINE_LANGUAGES: Record<string, readonly LanguageCode[]> = {
|
|||||||
'zh',
|
'zh',
|
||||||
],
|
],
|
||||||
chatterbox_turbo: ['en'],
|
chatterbox_turbo: ['en'],
|
||||||
|
tada: ['en', 'ar', 'zh', 'de', 'es', 'fr', 'it', 'ja', 'pl', 'pt'],
|
||||||
} as const;
|
} as const;
|
||||||
|
|
||||||
/** Helper: get language options for a given engine. */
|
/** Helper: get language options for a given engine. */
|
||||||
|
|||||||
@@ -15,9 +15,9 @@ const generationSchema = z.object({
|
|||||||
text: z.string().min(1, '').max(50000),
|
text: z.string().min(1, '').max(50000),
|
||||||
language: z.enum(LANGUAGE_CODES as [LanguageCode, ...LanguageCode[]]),
|
language: z.enum(LANGUAGE_CODES as [LanguageCode, ...LanguageCode[]]),
|
||||||
seed: z.number().int().optional(),
|
seed: z.number().int().optional(),
|
||||||
modelSize: z.enum(['1.7B', '0.6B']).optional(),
|
modelSize: z.enum(['1.7B', '0.6B', '1B', '3B']).optional(),
|
||||||
instruct: z.string().max(500).optional(),
|
instruct: z.string().max(500).optional(),
|
||||||
engine: z.enum(['qwen', 'luxtts', 'chatterbox', 'chatterbox_turbo']).optional(),
|
engine: z.enum(['qwen', 'luxtts', 'chatterbox', 'chatterbox_turbo', 'tada']).optional(),
|
||||||
});
|
});
|
||||||
|
|
||||||
export type GenerationFormValues = z.infer<typeof generationSchema>;
|
export type GenerationFormValues = z.infer<typeof generationSchema>;
|
||||||
@@ -79,7 +79,11 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
|
|||||||
? 'chatterbox-tts'
|
? 'chatterbox-tts'
|
||||||
: engine === 'chatterbox_turbo'
|
: engine === 'chatterbox_turbo'
|
||||||
? 'chatterbox-turbo'
|
? 'chatterbox-turbo'
|
||||||
: `qwen-tts-${data.modelSize}`;
|
: engine === 'tada'
|
||||||
|
? data.modelSize === '3B'
|
||||||
|
? 'tada-3b-ml'
|
||||||
|
: 'tada-1b'
|
||||||
|
: `qwen-tts-${data.modelSize}`;
|
||||||
const displayName =
|
const displayName =
|
||||||
engine === 'luxtts'
|
engine === 'luxtts'
|
||||||
? 'LuxTTS'
|
? 'LuxTTS'
|
||||||
@@ -87,9 +91,13 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
|
|||||||
? 'Chatterbox TTS'
|
? 'Chatterbox TTS'
|
||||||
: engine === 'chatterbox_turbo'
|
: engine === 'chatterbox_turbo'
|
||||||
? 'Chatterbox Turbo'
|
? 'Chatterbox Turbo'
|
||||||
: data.modelSize === '1.7B'
|
: engine === 'tada'
|
||||||
? 'Qwen TTS 1.7B'
|
? data.modelSize === '3B'
|
||||||
: 'Qwen TTS 0.6B';
|
? 'TADA 3B Multilingual'
|
||||||
|
: 'TADA 1B'
|
||||||
|
: data.modelSize === '1.7B'
|
||||||
|
? 'Qwen TTS 1.7B'
|
||||||
|
: 'Qwen TTS 0.6B';
|
||||||
|
|
||||||
// Check if model needs downloading
|
// Check if model needs downloading
|
||||||
try {
|
try {
|
||||||
@@ -104,7 +112,7 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
|
|||||||
console.error('Failed to check model status:', error);
|
console.error('Failed to check model status:', error);
|
||||||
}
|
}
|
||||||
|
|
||||||
const isQwen = engine === 'qwen';
|
const hasModelSizes = engine === 'qwen' || engine === 'tada';
|
||||||
const effectsChain = options.getEffectsChain?.();
|
const effectsChain = options.getEffectsChain?.();
|
||||||
// This now returns immediately with status="generating"
|
// This now returns immediately with status="generating"
|
||||||
const result = await generation.mutateAsync({
|
const result = await generation.mutateAsync({
|
||||||
@@ -112,9 +120,9 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
|
|||||||
text: data.text,
|
text: data.text,
|
||||||
language: data.language,
|
language: data.language,
|
||||||
seed: data.seed,
|
seed: data.seed,
|
||||||
model_size: isQwen ? data.modelSize : undefined,
|
model_size: hasModelSizes ? data.modelSize : undefined,
|
||||||
engine,
|
engine,
|
||||||
instruct: isQwen ? data.instruct || undefined : undefined,
|
instruct: engine === 'qwen' ? data.instruct || undefined : undefined,
|
||||||
max_chunk_chars: maxChunkChars,
|
max_chunk_chars: maxChunkChars,
|
||||||
crossfade_ms: crossfadeMs,
|
crossfade_ms: crossfadeMs,
|
||||||
normalize: normalizeAudio,
|
normalize: normalizeAudio,
|
||||||
|
|||||||
@@ -75,8 +75,8 @@ export function useGenerationProgress() {
|
|||||||
currentSources.delete(id);
|
currentSources.delete(id);
|
||||||
removePendingGeneration(id);
|
removePendingGeneration(id);
|
||||||
|
|
||||||
// Refresh history to pick up the completed generation
|
// Refetch history to pick up the completed generation
|
||||||
queryClient.invalidateQueries({ queryKey: ['history'] });
|
queryClient.refetchQueries({ queryKey: ['history'] });
|
||||||
|
|
||||||
// If this generation was queued for a story, add it now
|
// If this generation was queued for a story, add it now
|
||||||
const storyId = removePendingStoryAdd(id);
|
const storyId = removePendingStoryAdd(id);
|
||||||
@@ -120,7 +120,7 @@ export function useGenerationProgress() {
|
|||||||
removePendingGeneration(id);
|
removePendingGeneration(id);
|
||||||
removePendingStoryAdd(id);
|
removePendingStoryAdd(id);
|
||||||
|
|
||||||
queryClient.invalidateQueries({ queryKey: ['history'] });
|
queryClient.refetchQueries({ queryKey: ['history'] });
|
||||||
|
|
||||||
toast({
|
toast({
|
||||||
title: data.status === 'not_found' ? 'Generation not found' : 'Generation failed',
|
title: data.status === 'not_found' ? 'Generation not found' : 'Generation failed',
|
||||||
@@ -134,11 +134,12 @@ export function useGenerationProgress() {
|
|||||||
};
|
};
|
||||||
|
|
||||||
source.onerror = () => {
|
source.onerror = () => {
|
||||||
// EventSource auto-reconnects, but if we get repeated errors
|
// SSE connection dropped — clean up and refresh history so any
|
||||||
// just clean up
|
// completed/failed generation still appears in the list
|
||||||
source.close();
|
source.close();
|
||||||
currentSources.delete(id);
|
currentSources.delete(id);
|
||||||
removePendingGeneration(id);
|
removePendingGeneration(id);
|
||||||
|
queryClient.refetchQueries({ queryKey: ['history'] });
|
||||||
};
|
};
|
||||||
|
|
||||||
currentSources.set(id, source);
|
currentSources.set(id, source);
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
import { useState, useRef, useCallback, useEffect } from 'react';
|
import { useCallback, useEffect, useRef, useState } from 'react';
|
||||||
import { usePlatform } from '@/platform/PlatformContext';
|
import { usePlatform } from '@/platform/PlatformContext';
|
||||||
|
|
||||||
interface UseSystemAudioCaptureOptions {
|
interface UseSystemAudioCaptureOptions {
|
||||||
@@ -100,9 +100,7 @@ export function useSystemAudioCapture({
|
|||||||
onRecordingComplete?.(blob, recordedDuration);
|
onRecordingComplete?.(blob, recordedDuration);
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
const errorMessage =
|
const errorMessage =
|
||||||
err instanceof Error
|
err instanceof Error ? err.message : 'Failed to stop system audio capture.';
|
||||||
? err.message
|
|
||||||
: 'Failed to stop system audio capture.';
|
|
||||||
setError(errorMessage);
|
setError(errorMessage);
|
||||||
}
|
}
|
||||||
}, [isRecording, onRecordingComplete, platform]);
|
}, [isRecording, onRecordingComplete, platform]);
|
||||||
|
|||||||
@@ -1,10 +1,18 @@
|
|||||||
import { useMutation } from '@tanstack/react-query';
|
import { useMutation } from '@tanstack/react-query';
|
||||||
import { apiClient } from '@/lib/api/client';
|
import { apiClient } from '@/lib/api/client';
|
||||||
|
import type { WhisperModelSize } from '@/lib/api/types';
|
||||||
import type { LanguageCode } from '@/lib/constants/languages';
|
import type { LanguageCode } from '@/lib/constants/languages';
|
||||||
|
|
||||||
export function useTranscription() {
|
export function useTranscription() {
|
||||||
return useMutation({
|
return useMutation({
|
||||||
mutationFn: ({ file, language }: { file: File; language?: LanguageCode }) =>
|
mutationFn: ({
|
||||||
apiClient.transcribeAudio(file, language),
|
file,
|
||||||
|
language,
|
||||||
|
model,
|
||||||
|
}: {
|
||||||
|
file: File;
|
||||||
|
language?: LanguageCode;
|
||||||
|
model?: WhisperModelSize;
|
||||||
|
}) => apiClient.transcribeAudio(file, language, model),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,37 @@
|
|||||||
|
export interface ChangelogEntry {
|
||||||
|
version: string;
|
||||||
|
date: string | null;
|
||||||
|
body: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parses a Keep-a-Changelog style markdown string into structured entries.
|
||||||
|
*
|
||||||
|
* Splits on `## [version]` headings and extracts the version + date from each.
|
||||||
|
* The body is the raw markdown between headings (trimmed), with the leading
|
||||||
|
* `# Changelog` title and trailing link references stripped.
|
||||||
|
*/
|
||||||
|
export function parseChangelog(raw: string): ChangelogEntry[] {
|
||||||
|
const entries: ChangelogEntry[] = [];
|
||||||
|
|
||||||
|
// Strip trailing link reference definitions (e.g. [0.1.0]: https://...)
|
||||||
|
const cleaned = raw.replace(/^\[[\w.]+\]:.*$/gm, '').trimEnd();
|
||||||
|
|
||||||
|
// Match `## [version]` or `## [version] - date`
|
||||||
|
const headingRe = /^## \[(.+?)\](?:\s*-\s*(.+))?$/gm;
|
||||||
|
const matches = [...cleaned.matchAll(headingRe)];
|
||||||
|
|
||||||
|
for (let i = 0; i < matches.length; i++) {
|
||||||
|
const match = matches[i];
|
||||||
|
const version = match[1];
|
||||||
|
const date = match[2]?.trim() || null;
|
||||||
|
|
||||||
|
const start = match.index! + match[0].length;
|
||||||
|
const end = i + 1 < matches.length ? matches[i + 1].index! : cleaned.length;
|
||||||
|
const body = cleaned.slice(start, end).trim();
|
||||||
|
|
||||||
|
entries.push({ version, date, body });
|
||||||
|
}
|
||||||
|
|
||||||
|
return entries;
|
||||||
|
}
|
||||||
@@ -50,12 +50,18 @@ export interface PlatformAudio {
|
|||||||
stopPlayback(): void;
|
stopPlayback(): void;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export interface ServerLogEntry {
|
||||||
|
stream: 'stdout' | 'stderr';
|
||||||
|
line: string;
|
||||||
|
}
|
||||||
|
|
||||||
export interface PlatformLifecycle {
|
export interface PlatformLifecycle {
|
||||||
startServer(remote?: boolean, modelsDir?: string | null): Promise<string>;
|
startServer(remote?: boolean, modelsDir?: string | null): Promise<string>;
|
||||||
stopServer(): Promise<void>;
|
stopServer(): Promise<void>;
|
||||||
restartServer(modelsDir?: string | null): Promise<string>;
|
restartServer(modelsDir?: string | null): Promise<string>;
|
||||||
setKeepServerRunning(keep: boolean): Promise<void>;
|
setKeepServerRunning(keep: boolean): Promise<void>;
|
||||||
setupWindowCloseHandler(): Promise<void>;
|
setupWindowCloseHandler(): Promise<void>;
|
||||||
|
subscribeToServerLogs(callback: (entry: ServerLogEntry) => void): () => void;
|
||||||
onServerReady?: () => void;
|
onServerReady?: () => void;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+72
-6
@@ -1,10 +1,22 @@
|
|||||||
import { createRootRoute, createRoute, createRouter, Outlet } from '@tanstack/react-router';
|
import {
|
||||||
|
createRootRoute,
|
||||||
|
createRoute,
|
||||||
|
createRouter,
|
||||||
|
Outlet,
|
||||||
|
redirect,
|
||||||
|
} from '@tanstack/react-router';
|
||||||
import { AppFrame } from '@/components/AppFrame/AppFrame';
|
import { AppFrame } from '@/components/AppFrame/AppFrame';
|
||||||
import { AudioTab } from '@/components/AudioTab/AudioTab';
|
import { AudioTab } from '@/components/AudioTab/AudioTab';
|
||||||
import { EffectsTab } from '@/components/EffectsTab/EffectsTab';
|
import { EffectsTab } from '@/components/EffectsTab/EffectsTab';
|
||||||
import { MainEditor } from '@/components/MainEditor/MainEditor';
|
import { MainEditor } from '@/components/MainEditor/MainEditor';
|
||||||
import { ModelsTab } from '@/components/ModelsTab/ModelsTab';
|
import { ModelsTab } from '@/components/ModelsTab/ModelsTab';
|
||||||
import { ServerTab } from '@/components/ServerTab/ServerTab';
|
import { AboutPage } from '@/components/ServerTab/AboutPage';
|
||||||
|
import { ChangelogPage } from '@/components/ServerTab/ChangelogPage';
|
||||||
|
import { GeneralPage } from '@/components/ServerTab/GeneralPage';
|
||||||
|
import { GenerationPage } from '@/components/ServerTab/GenerationPage';
|
||||||
|
import { GpuPage } from '@/components/ServerTab/GpuPage';
|
||||||
|
import { LogsPage } from '@/components/ServerTab/LogsPage';
|
||||||
|
import { SettingsLayout } from '@/components/ServerTab/ServerTab';
|
||||||
import { Sidebar } from '@/components/Sidebar';
|
import { Sidebar } from '@/components/Sidebar';
|
||||||
import { StoriesTab } from '@/components/StoriesTab/StoriesTab';
|
import { StoriesTab } from '@/components/StoriesTab/StoriesTab';
|
||||||
import { Toaster } from '@/components/ui/toaster';
|
import { Toaster } from '@/components/ui/toaster';
|
||||||
@@ -120,11 +132,57 @@ const modelsRoute = createRoute({
|
|||||||
component: ModelsTab,
|
component: ModelsTab,
|
||||||
});
|
});
|
||||||
|
|
||||||
// Server route
|
// Settings layout route (parent for sub-tabs)
|
||||||
const serverRoute = createRoute({
|
const settingsRoute = createRoute({
|
||||||
|
getParentRoute: () => rootRoute,
|
||||||
|
path: '/settings',
|
||||||
|
component: SettingsLayout,
|
||||||
|
});
|
||||||
|
|
||||||
|
// Settings sub-routes
|
||||||
|
const settingsGeneralRoute = createRoute({
|
||||||
|
getParentRoute: () => settingsRoute,
|
||||||
|
path: '/',
|
||||||
|
component: GeneralPage,
|
||||||
|
});
|
||||||
|
|
||||||
|
const settingsGenerationRoute = createRoute({
|
||||||
|
getParentRoute: () => settingsRoute,
|
||||||
|
path: '/generation',
|
||||||
|
component: GenerationPage,
|
||||||
|
});
|
||||||
|
|
||||||
|
const settingsGpuRoute = createRoute({
|
||||||
|
getParentRoute: () => settingsRoute,
|
||||||
|
path: '/gpu',
|
||||||
|
component: GpuPage,
|
||||||
|
});
|
||||||
|
|
||||||
|
const settingsChangelogRoute = createRoute({
|
||||||
|
getParentRoute: () => settingsRoute,
|
||||||
|
path: '/changelog',
|
||||||
|
component: ChangelogPage,
|
||||||
|
});
|
||||||
|
|
||||||
|
const settingsLogsRoute = createRoute({
|
||||||
|
getParentRoute: () => settingsRoute,
|
||||||
|
path: '/logs',
|
||||||
|
component: LogsPage,
|
||||||
|
});
|
||||||
|
|
||||||
|
const settingsAboutRoute = createRoute({
|
||||||
|
getParentRoute: () => settingsRoute,
|
||||||
|
path: '/about',
|
||||||
|
component: AboutPage,
|
||||||
|
});
|
||||||
|
|
||||||
|
// Redirect old /server path to /settings
|
||||||
|
const serverRedirectRoute = createRoute({
|
||||||
getParentRoute: () => rootRoute,
|
getParentRoute: () => rootRoute,
|
||||||
path: '/server',
|
path: '/server',
|
||||||
component: ServerTab,
|
beforeLoad: () => {
|
||||||
|
throw redirect({ to: '/settings' });
|
||||||
|
},
|
||||||
});
|
});
|
||||||
|
|
||||||
// Route tree
|
// Route tree
|
||||||
@@ -135,7 +193,15 @@ const routeTree = rootRoute.addChildren([
|
|||||||
audioRoute,
|
audioRoute,
|
||||||
effectsRoute,
|
effectsRoute,
|
||||||
modelsRoute,
|
modelsRoute,
|
||||||
serverRoute,
|
settingsRoute.addChildren([
|
||||||
|
settingsGeneralRoute,
|
||||||
|
settingsGenerationRoute,
|
||||||
|
settingsGpuRoute,
|
||||||
|
settingsLogsRoute,
|
||||||
|
settingsChangelogRoute,
|
||||||
|
settingsAboutRoute,
|
||||||
|
]),
|
||||||
|
serverRedirectRoute,
|
||||||
]);
|
]);
|
||||||
|
|
||||||
// Create router
|
// Create router
|
||||||
|
|||||||
@@ -0,0 +1,31 @@
|
|||||||
|
import { create } from 'zustand';
|
||||||
|
import type { ServerLogEntry } from '@/platform/types';
|
||||||
|
|
||||||
|
const MAX_LOG_ENTRIES = 2000;
|
||||||
|
|
||||||
|
let nextLogEntryId = 0;
|
||||||
|
|
||||||
|
export interface LogEntry extends ServerLogEntry {
|
||||||
|
id: number;
|
||||||
|
timestamp: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface LogStore {
|
||||||
|
entries: LogEntry[];
|
||||||
|
addEntry: (entry: ServerLogEntry) => void;
|
||||||
|
clear: () => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const useLogStore = create<LogStore>((set) => ({
|
||||||
|
entries: [],
|
||||||
|
addEntry: (entry) =>
|
||||||
|
set((state) => {
|
||||||
|
const newEntry: LogEntry = { ...entry, id: nextLogEntryId++, timestamp: Date.now() };
|
||||||
|
const entries = [...state.entries, newEntry];
|
||||||
|
if (entries.length > MAX_LOG_ENTRIES) {
|
||||||
|
return { entries: entries.slice(entries.length - MAX_LOG_ENTRIES) };
|
||||||
|
}
|
||||||
|
return { entries };
|
||||||
|
}),
|
||||||
|
clear: () => set({ entries: [] }),
|
||||||
|
}));
|
||||||
@@ -6,5 +6,5 @@
|
|||||||
"moduleResolution": "bundler",
|
"moduleResolution": "bundler",
|
||||||
"allowSyntheticDefaultImports": true
|
"allowSyntheticDefaultImports": true
|
||||||
},
|
},
|
||||||
"include": ["vite.config.ts"]
|
"include": ["vite.config.ts", "plugins/**/*.ts"]
|
||||||
}
|
}
|
||||||
|
|||||||
+2
-1
@@ -2,9 +2,10 @@ import path from 'node:path';
|
|||||||
import tailwindcss from '@tailwindcss/vite';
|
import tailwindcss from '@tailwindcss/vite';
|
||||||
import react from '@vitejs/plugin-react';
|
import react from '@vitejs/plugin-react';
|
||||||
import { defineConfig } from 'vite';
|
import { defineConfig } from 'vite';
|
||||||
|
import { changelogPlugin } from './plugins/changelog';
|
||||||
|
|
||||||
export default defineConfig({
|
export default defineConfig({
|
||||||
plugins: [tailwindcss(), react()],
|
plugins: [tailwindcss(), react(), changelogPlugin(path.resolve(__dirname, '..'))],
|
||||||
resolve: {
|
resolve: {
|
||||||
alias: {
|
alias: {
|
||||||
'@': path.resolve(__dirname, './src'),
|
'@': path.resolve(__dirname, './src'),
|
||||||
|
|||||||
+1
-1
@@ -1,3 +1,3 @@
|
|||||||
# Backend package
|
# Backend package
|
||||||
|
|
||||||
__version__ = "0.2.3"
|
__version__ = "0.3.1"
|
||||||
|
|||||||
@@ -77,6 +77,7 @@ def create_app() -> FastAPI:
|
|||||||
_configure_cors(application)
|
_configure_cors(application)
|
||||||
register_routers(application)
|
register_routers(application)
|
||||||
_register_lifecycle(application)
|
_register_lifecycle(application)
|
||||||
|
_mount_frontend(application)
|
||||||
|
|
||||||
return application
|
return application
|
||||||
|
|
||||||
@@ -104,6 +105,43 @@ def _configure_cors(application: FastAPI) -> None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _mount_frontend(application: FastAPI) -> None:
|
||||||
|
"""Serve the built web frontend when present (Docker / web deployment).
|
||||||
|
|
||||||
|
The Dockerfile copies the Vite build output to ``/app/frontend/``. When
|
||||||
|
that directory exists we mount static assets and add a catch-all route so
|
||||||
|
the React SPA handles client-side routing. In dev or API-only mode the
|
||||||
|
directory is absent and this function is a no-op.
|
||||||
|
"""
|
||||||
|
frontend_dir = Path(__file__).resolve().parent.parent / "frontend"
|
||||||
|
if not frontend_dir.is_dir():
|
||||||
|
return
|
||||||
|
|
||||||
|
from fastapi.staticfiles import StaticFiles
|
||||||
|
from fastapi.responses import FileResponse
|
||||||
|
|
||||||
|
# Mount hashed assets (JS, CSS, images) that Vite places under /assets
|
||||||
|
assets_dir = frontend_dir / "assets"
|
||||||
|
if assets_dir.is_dir():
|
||||||
|
application.mount(
|
||||||
|
"/assets",
|
||||||
|
StaticFiles(directory=str(assets_dir)),
|
||||||
|
name="frontend-assets",
|
||||||
|
)
|
||||||
|
|
||||||
|
# SPA catch-all: serve files if they exist, otherwise index.html for
|
||||||
|
# client-side routes like /voices, /stories, /models, etc.
|
||||||
|
@application.get("/{full_path:path}")
|
||||||
|
async def serve_spa(full_path: str):
|
||||||
|
file_path = (frontend_dir / full_path).resolve()
|
||||||
|
# Guard against path traversal — only serve files inside frontend_dir
|
||||||
|
if full_path and file_path.is_file() and str(file_path).startswith(str(frontend_dir)):
|
||||||
|
return FileResponse(file_path)
|
||||||
|
return FileResponse(frontend_dir / "index.html", media_type="text/html")
|
||||||
|
|
||||||
|
logger.info("Frontend: serving SPA from %s", frontend_dir)
|
||||||
|
|
||||||
|
|
||||||
def _get_gpu_status() -> str:
|
def _get_gpu_status() -> str:
|
||||||
"""Return a human-readable string describing GPU availability."""
|
"""Return a human-readable string describing GPU availability."""
|
||||||
backend_type = get_backend_type()
|
backend_type = get_backend_type()
|
||||||
|
|||||||
@@ -134,6 +134,7 @@ class STTBackend(Protocol):
|
|||||||
self,
|
self,
|
||||||
audio_path: str,
|
audio_path: str,
|
||||||
language: Optional[str] = None,
|
language: Optional[str] = None,
|
||||||
|
model_size: Optional[str] = None,
|
||||||
) -> str:
|
) -> str:
|
||||||
"""
|
"""
|
||||||
Transcribe audio to text.
|
Transcribe audio to text.
|
||||||
@@ -165,6 +166,7 @@ TTS_ENGINES = {
|
|||||||
"luxtts": "LuxTTS",
|
"luxtts": "LuxTTS",
|
||||||
"chatterbox": "Chatterbox TTS",
|
"chatterbox": "Chatterbox TTS",
|
||||||
"chatterbox_turbo": "Chatterbox Turbo",
|
"chatterbox_turbo": "Chatterbox Turbo",
|
||||||
|
"tada": "TADA",
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -258,6 +260,24 @@ def _get_non_qwen_tts_configs() -> list[ModelConfig]:
|
|||||||
needs_trim=True,
|
needs_trim=True,
|
||||||
languages=["en"],
|
languages=["en"],
|
||||||
),
|
),
|
||||||
|
ModelConfig(
|
||||||
|
model_name="tada-1b",
|
||||||
|
display_name="TADA 1B (English)",
|
||||||
|
engine="tada",
|
||||||
|
hf_repo_id="HumeAI/tada-1b",
|
||||||
|
model_size="1B",
|
||||||
|
size_mb=4000,
|
||||||
|
languages=["en"],
|
||||||
|
),
|
||||||
|
ModelConfig(
|
||||||
|
model_name="tada-3b-ml",
|
||||||
|
display_name="TADA 3B Multilingual",
|
||||||
|
engine="tada",
|
||||||
|
hf_repo_id="HumeAI/tada-3b-ml",
|
||||||
|
model_size="3B",
|
||||||
|
size_mb=8000,
|
||||||
|
languages=["en", "ar", "zh", "de", "es", "fr", "it", "ja", "pl", "pt"],
|
||||||
|
),
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
@@ -338,10 +358,12 @@ def engine_has_model_sizes(engine: str) -> bool:
|
|||||||
|
|
||||||
|
|
||||||
async def load_engine_model(engine: str, model_size: str = "default") -> None:
|
async def load_engine_model(engine: str, model_size: str = "default") -> None:
|
||||||
"""Load a model for the given engine, handling the Qwen model_size special case."""
|
"""Load a model for the given engine, handling engines with multiple model sizes."""
|
||||||
backend = get_tts_backend_for_engine(engine)
|
backend = get_tts_backend_for_engine(engine)
|
||||||
if engine == "qwen":
|
if engine == "qwen":
|
||||||
await backend.load_model_async(model_size)
|
await backend.load_model_async(model_size)
|
||||||
|
elif engine == "tada":
|
||||||
|
await backend.load_model(model_size)
|
||||||
else:
|
else:
|
||||||
await backend.load_model()
|
await backend.load_model()
|
||||||
|
|
||||||
@@ -357,7 +379,7 @@ async def ensure_model_cached_or_raise(engine: str, model_size: str = "default")
|
|||||||
cfg = c
|
cfg = c
|
||||||
break
|
break
|
||||||
|
|
||||||
if engine == "qwen":
|
if engine in ("qwen", "tada"):
|
||||||
if not backend._is_model_cached(model_size):
|
if not backend._is_model_cached(model_size):
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status_code=400,
|
status_code=400,
|
||||||
@@ -489,6 +511,10 @@ def get_tts_backend_for_engine(engine: str) -> TTSBackend:
|
|||||||
from .chatterbox_turbo_backend import ChatterboxTurboTTSBackend
|
from .chatterbox_turbo_backend import ChatterboxTurboTTSBackend
|
||||||
|
|
||||||
backend = ChatterboxTurboTTSBackend()
|
backend = ChatterboxTurboTTSBackend()
|
||||||
|
elif engine == "tada":
|
||||||
|
from .hume_backend import HumeTadaBackend
|
||||||
|
|
||||||
|
backend = HumeTadaBackend()
|
||||||
else:
|
else:
|
||||||
raise ValueError(f"Unknown TTS engine: {engine}. Supported: {list(TTS_ENGINES.keys())}")
|
raise ValueError(f"Unknown TTS engine: {engine}. Supported: {list(TTS_ENGINES.keys())}")
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,347 @@
|
|||||||
|
"""
|
||||||
|
HumeAI TADA TTS backend implementation.
|
||||||
|
|
||||||
|
Wraps HumeAI's TADA (Text-Acoustic Dual Alignment) model for
|
||||||
|
high-quality voice cloning. Two model variants:
|
||||||
|
- tada-1b: English-only, ~2B params (Llama 3.2 1B base)
|
||||||
|
- tada-3b-ml: Multilingual, ~4B params (Llama 3.2 3B base)
|
||||||
|
|
||||||
|
Both use a shared encoder/codec (HumeAI/tada-codec). The encoder
|
||||||
|
produces 1:1 aligned token embeddings from reference audio, and the
|
||||||
|
causal LM generates speech via flow-matching diffusion.
|
||||||
|
|
||||||
|
24kHz output, bf16 inference on CUDA, fp32 on CPU.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
import threading
|
||||||
|
from typing import ClassVar, List, Optional, Tuple
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
from . import TTSBackend
|
||||||
|
from .base import (
|
||||||
|
is_model_cached,
|
||||||
|
get_torch_device,
|
||||||
|
combine_voice_prompts as _combine_voice_prompts,
|
||||||
|
model_load_progress,
|
||||||
|
)
|
||||||
|
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# HuggingFace repos
|
||||||
|
TADA_CODEC_REPO = "HumeAI/tada-codec"
|
||||||
|
TADA_1B_REPO = "HumeAI/tada-1b"
|
||||||
|
TADA_3B_ML_REPO = "HumeAI/tada-3b-ml"
|
||||||
|
|
||||||
|
TADA_MODEL_REPOS = {
|
||||||
|
"1B": TADA_1B_REPO,
|
||||||
|
"3B": TADA_3B_ML_REPO,
|
||||||
|
}
|
||||||
|
|
||||||
|
# Key weight files for cache detection
|
||||||
|
_TADA_MODEL_WEIGHT_FILES = [
|
||||||
|
"model.safetensors",
|
||||||
|
]
|
||||||
|
|
||||||
|
_TADA_CODEC_WEIGHT_FILES = [
|
||||||
|
"encoder/model.safetensors",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class HumeTadaBackend:
|
||||||
|
"""HumeAI TADA TTS backend for high-quality voice cloning."""
|
||||||
|
|
||||||
|
_load_lock: ClassVar[threading.Lock] = threading.Lock()
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self.model = None
|
||||||
|
self.encoder = None
|
||||||
|
self.model_size = "1B" # default to 1B
|
||||||
|
self._device = None
|
||||||
|
self._model_load_lock = asyncio.Lock()
|
||||||
|
|
||||||
|
def _get_device(self) -> str:
|
||||||
|
# Force CPU on macOS — MPS has issues with flow matching
|
||||||
|
# and large vocab lm_head (>65536 output channels)
|
||||||
|
return get_torch_device(force_cpu_on_mac=True)
|
||||||
|
|
||||||
|
def is_loaded(self) -> bool:
|
||||||
|
return self.model is not None
|
||||||
|
|
||||||
|
def _get_model_path(self, model_size: str = "1B") -> str:
|
||||||
|
return TADA_MODEL_REPOS.get(model_size, TADA_1B_REPO)
|
||||||
|
|
||||||
|
def _is_model_cached(self, model_size: str = "1B") -> bool:
|
||||||
|
repo = TADA_MODEL_REPOS.get(model_size, TADA_1B_REPO)
|
||||||
|
model_cached = is_model_cached(repo, required_files=_TADA_MODEL_WEIGHT_FILES)
|
||||||
|
codec_cached = is_model_cached(TADA_CODEC_REPO, required_files=_TADA_CODEC_WEIGHT_FILES)
|
||||||
|
return model_cached and codec_cached
|
||||||
|
|
||||||
|
async def load_model(self, model_size: str = "1B") -> None:
|
||||||
|
"""Load the TADA model and encoder."""
|
||||||
|
if self.model is not None and self.model_size == model_size:
|
||||||
|
return
|
||||||
|
async with self._model_load_lock:
|
||||||
|
if self.model is not None and self.model_size == model_size:
|
||||||
|
return
|
||||||
|
# Unload existing model if switching sizes
|
||||||
|
if self.model is not None:
|
||||||
|
self.unload_model()
|
||||||
|
self.model_size = model_size
|
||||||
|
await asyncio.to_thread(self._load_model_sync, model_size)
|
||||||
|
|
||||||
|
def _load_model_sync(self, model_size: str = "1B"):
|
||||||
|
"""Synchronous model loading with progress tracking."""
|
||||||
|
model_name = f"tada-{model_size.lower()}"
|
||||||
|
is_cached = self._is_model_cached(model_size)
|
||||||
|
repo = TADA_MODEL_REPOS.get(model_size, TADA_1B_REPO)
|
||||||
|
|
||||||
|
with model_load_progress(model_name, is_cached):
|
||||||
|
# Install DAC shim before importing tada — tada's encoder/decoder
|
||||||
|
# import dac.nn.layers.Snake1d which requires the descript-audio-codec
|
||||||
|
# package. The real package pulls in onnx/tensorboard/matplotlib via
|
||||||
|
# descript-audiotools, so we use a lightweight shim instead.
|
||||||
|
from ..utils.dac_shim import install_dac_shim
|
||||||
|
install_dac_shim()
|
||||||
|
|
||||||
|
import torch
|
||||||
|
from huggingface_hub import snapshot_download
|
||||||
|
|
||||||
|
device = self._get_device()
|
||||||
|
self._device = device
|
||||||
|
logger.info(f"Loading HumeAI TADA {model_size} on {device}...")
|
||||||
|
|
||||||
|
# Download codec (encoder + decoder) if not cached
|
||||||
|
logger.info("Downloading TADA codec...")
|
||||||
|
snapshot_download(
|
||||||
|
repo_id=TADA_CODEC_REPO,
|
||||||
|
token=None,
|
||||||
|
allow_patterns=["*.safetensors", "*.json", "*.txt", "*.bin"],
|
||||||
|
)
|
||||||
|
|
||||||
|
# Download model weights if not cached
|
||||||
|
logger.info(f"Downloading TADA {model_size} model...")
|
||||||
|
snapshot_download(
|
||||||
|
repo_id=repo,
|
||||||
|
token=None,
|
||||||
|
allow_patterns=["*.safetensors", "*.json", "*.txt", "*.bin", "*.model"],
|
||||||
|
)
|
||||||
|
|
||||||
|
# TADA hardcodes "meta-llama/Llama-3.2-1B" as the tokenizer
|
||||||
|
# source in its Aligner and TadaForCausalLM.from_pretrained().
|
||||||
|
# That repo is gated (requires Meta license acceptance).
|
||||||
|
# Download the tokenizer from an ungated mirror and get its
|
||||||
|
# local cache path so we can point TADA at it directly.
|
||||||
|
logger.info("Downloading Llama tokenizer (ungated mirror)...")
|
||||||
|
tokenizer_path = snapshot_download(
|
||||||
|
repo_id="unsloth/Llama-3.2-1B",
|
||||||
|
token=None,
|
||||||
|
allow_patterns=["tokenizer*", "special_tokens*"],
|
||||||
|
)
|
||||||
|
|
||||||
|
# Determine dtype — use bf16 on CUDA for ~50% memory savings
|
||||||
|
if device == "cuda" and torch.cuda.is_bf16_supported():
|
||||||
|
model_dtype = torch.bfloat16
|
||||||
|
else:
|
||||||
|
model_dtype = torch.float32
|
||||||
|
|
||||||
|
# Patch the Aligner config class to use the local tokenizer
|
||||||
|
# path instead of the gated "meta-llama/Llama-3.2-1B" default.
|
||||||
|
# This avoids monkey-patching AutoTokenizer.from_pretrained
|
||||||
|
# which corrupts the classmethod descriptor for other engines.
|
||||||
|
from tada.modules.aligner import AlignerConfig
|
||||||
|
AlignerConfig.tokenizer_name = tokenizer_path
|
||||||
|
|
||||||
|
# Load encoder (only needed for voice prompt encoding)
|
||||||
|
from tada.modules.encoder import Encoder
|
||||||
|
logger.info("Loading TADA encoder...")
|
||||||
|
self.encoder = Encoder.from_pretrained(
|
||||||
|
TADA_CODEC_REPO, subfolder="encoder"
|
||||||
|
).to(device)
|
||||||
|
self.encoder.eval()
|
||||||
|
|
||||||
|
# Load the causal LM (includes decoder for wav generation).
|
||||||
|
# TadaForCausalLM.from_pretrained() calls
|
||||||
|
# getattr(config, "tokenizer_name", "meta-llama/Llama-3.2-1B")
|
||||||
|
# which hits the gated repo. Pre-load the config from HF,
|
||||||
|
# inject the local tokenizer path, then pass it in.
|
||||||
|
from tada.modules.tada import TadaForCausalLM, TadaConfig
|
||||||
|
logger.info(f"Loading TADA {model_size} model...")
|
||||||
|
config = TadaConfig.from_pretrained(repo)
|
||||||
|
config.tokenizer_name = tokenizer_path
|
||||||
|
self.model = TadaForCausalLM.from_pretrained(
|
||||||
|
repo, config=config, torch_dtype=model_dtype
|
||||||
|
).to(device)
|
||||||
|
self.model.eval()
|
||||||
|
|
||||||
|
logger.info(f"HumeAI TADA {model_size} loaded successfully on {device}")
|
||||||
|
|
||||||
|
def unload_model(self) -> None:
|
||||||
|
"""Unload model and encoder to free memory."""
|
||||||
|
if self.model is not None:
|
||||||
|
del self.model
|
||||||
|
self.model = None
|
||||||
|
if self.encoder is not None:
|
||||||
|
del self.encoder
|
||||||
|
self.encoder = None
|
||||||
|
|
||||||
|
self._device = None
|
||||||
|
|
||||||
|
import torch
|
||||||
|
if torch.cuda.is_available():
|
||||||
|
torch.cuda.empty_cache()
|
||||||
|
|
||||||
|
logger.info("HumeAI TADA unloaded")
|
||||||
|
|
||||||
|
async def create_voice_prompt(
|
||||||
|
self,
|
||||||
|
audio_path: str,
|
||||||
|
reference_text: str,
|
||||||
|
use_cache: bool = True,
|
||||||
|
) -> Tuple[dict, bool]:
|
||||||
|
"""
|
||||||
|
Create voice prompt from reference audio using TADA's encoder.
|
||||||
|
|
||||||
|
TADA's encoder performs forced alignment between audio and text tokens,
|
||||||
|
producing an EncoderOutput with 1:1 token-audio alignment. If no
|
||||||
|
reference_text is provided, the encoder uses built-in ASR (English only).
|
||||||
|
|
||||||
|
We serialize the EncoderOutput to a dict for caching.
|
||||||
|
"""
|
||||||
|
await self.load_model(self.model_size)
|
||||||
|
|
||||||
|
cache_key = (
|
||||||
|
"tada_" + get_cache_key(audio_path, reference_text)
|
||||||
|
) if use_cache else None
|
||||||
|
|
||||||
|
if cache_key:
|
||||||
|
cached = get_cached_voice_prompt(cache_key)
|
||||||
|
if cached is not None and isinstance(cached, dict):
|
||||||
|
return cached, True
|
||||||
|
|
||||||
|
def _encode_sync():
|
||||||
|
import torch
|
||||||
|
import soundfile as sf
|
||||||
|
|
||||||
|
device = self._device
|
||||||
|
|
||||||
|
# Load audio with soundfile (torchaudio 2.10+ requires torchcodec)
|
||||||
|
audio_np, sr = sf.read(str(audio_path), dtype="float32")
|
||||||
|
audio = torch.from_numpy(audio_np).float()
|
||||||
|
if audio.ndim == 1:
|
||||||
|
audio = audio.unsqueeze(0) # (samples,) -> (1, samples)
|
||||||
|
else:
|
||||||
|
audio = audio.T # (samples, channels) -> (channels, samples)
|
||||||
|
audio = audio.to(device)
|
||||||
|
|
||||||
|
# Encode with forced alignment
|
||||||
|
text_arg = [reference_text] if reference_text else None
|
||||||
|
prompt = self.encoder(
|
||||||
|
audio, text=text_arg, sample_rate=sr
|
||||||
|
)
|
||||||
|
|
||||||
|
# Serialize EncoderOutput to a dict of CPU tensors for caching
|
||||||
|
prompt_dict = {}
|
||||||
|
for field_name in prompt.__dataclass_fields__:
|
||||||
|
val = getattr(prompt, field_name)
|
||||||
|
if isinstance(val, torch.Tensor):
|
||||||
|
prompt_dict[field_name] = val.detach().cpu()
|
||||||
|
elif isinstance(val, list):
|
||||||
|
prompt_dict[field_name] = val
|
||||||
|
elif isinstance(val, (int, float)):
|
||||||
|
prompt_dict[field_name] = val
|
||||||
|
else:
|
||||||
|
prompt_dict[field_name] = val
|
||||||
|
return prompt_dict
|
||||||
|
|
||||||
|
encoded = await asyncio.to_thread(_encode_sync)
|
||||||
|
|
||||||
|
if cache_key:
|
||||||
|
cache_voice_prompt(cache_key, encoded)
|
||||||
|
|
||||||
|
return encoded, False
|
||||||
|
|
||||||
|
async def combine_voice_prompts(
|
||||||
|
self,
|
||||||
|
audio_paths: List[str],
|
||||||
|
reference_texts: List[str],
|
||||||
|
) -> Tuple[np.ndarray, str]:
|
||||||
|
return await _combine_voice_prompts(audio_paths, reference_texts, sample_rate=24000)
|
||||||
|
|
||||||
|
async def generate(
|
||||||
|
self,
|
||||||
|
text: str,
|
||||||
|
voice_prompt: dict,
|
||||||
|
language: str = "en",
|
||||||
|
seed: Optional[int] = None,
|
||||||
|
instruct: Optional[str] = None,
|
||||||
|
) -> Tuple[np.ndarray, int]:
|
||||||
|
"""
|
||||||
|
Generate audio from text using HumeAI TADA.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
text: Text to synthesize
|
||||||
|
voice_prompt: Serialized EncoderOutput dict from create_voice_prompt()
|
||||||
|
language: Language code (en, ar, de, es, fr, it, ja, pl, pt, zh)
|
||||||
|
seed: Random seed for reproducibility
|
||||||
|
instruct: Not supported by TADA (ignored)
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Tuple of (audio_array, sample_rate=24000)
|
||||||
|
"""
|
||||||
|
await self.load_model(self.model_size)
|
||||||
|
|
||||||
|
def _generate_sync():
|
||||||
|
import torch
|
||||||
|
from tada.modules.encoder import EncoderOutput
|
||||||
|
|
||||||
|
if seed is not None:
|
||||||
|
torch.manual_seed(seed)
|
||||||
|
if torch.cuda.is_available():
|
||||||
|
torch.cuda.manual_seed(seed)
|
||||||
|
|
||||||
|
device = self._device
|
||||||
|
|
||||||
|
# Reconstruct EncoderOutput from the cached dict
|
||||||
|
restored = {}
|
||||||
|
for k, v in voice_prompt.items():
|
||||||
|
if isinstance(v, torch.Tensor):
|
||||||
|
# Move to device and match model dtype for float tensors
|
||||||
|
if v.is_floating_point():
|
||||||
|
model_dtype = next(self.model.parameters()).dtype
|
||||||
|
restored[k] = v.to(device=device, dtype=model_dtype)
|
||||||
|
else:
|
||||||
|
restored[k] = v.to(device=device)
|
||||||
|
else:
|
||||||
|
restored[k] = v
|
||||||
|
|
||||||
|
prompt = EncoderOutput(**restored)
|
||||||
|
|
||||||
|
# For non-English with the 3B-ML model, we could reload the
|
||||||
|
# encoder with the language-specific aligner. However, the
|
||||||
|
# generation itself is language-agnostic — only the encoder's
|
||||||
|
# aligner changes. Since we encode at create_voice_prompt time,
|
||||||
|
# the language is already baked in. For simplicity, we don't
|
||||||
|
# reload the encoder here.
|
||||||
|
|
||||||
|
logger.info(f"[TADA] Generating ({language}), text length: {len(text)}")
|
||||||
|
|
||||||
|
output = self.model.generate(
|
||||||
|
prompt=prompt,
|
||||||
|
text=text,
|
||||||
|
)
|
||||||
|
|
||||||
|
# output.audio is a list of tensors (one per batch item)
|
||||||
|
if output.audio and output.audio[0] is not None:
|
||||||
|
audio_tensor = output.audio[0]
|
||||||
|
audio = audio_tensor.detach().cpu().numpy().squeeze().astype(np.float32)
|
||||||
|
else:
|
||||||
|
logger.warning("[TADA] Generation produced no audio")
|
||||||
|
audio = np.zeros(24000, dtype=np.float32)
|
||||||
|
|
||||||
|
return audio, 24000
|
||||||
|
|
||||||
|
return await asyncio.to_thread(_generate_sync)
|
||||||
@@ -6,7 +6,6 @@ from typing import Optional, List, Tuple
|
|||||||
import asyncio
|
import asyncio
|
||||||
import logging
|
import logging
|
||||||
import numpy as np
|
import numpy as np
|
||||||
import os
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -21,6 +20,7 @@ ensure_original_qwen_config_cached()
|
|||||||
from . import TTSBackend, STTBackend, LANGUAGE_CODE_TO_NAME, WHISPER_HF_REPOS
|
from . import TTSBackend, STTBackend, LANGUAGE_CODE_TO_NAME, WHISPER_HF_REPOS
|
||||||
from .base import is_model_cached, combine_voice_prompts as _combine_voice_prompts, model_load_progress
|
from .base import is_model_cached, combine_voice_prompts as _combine_voice_prompts, model_load_progress
|
||||||
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
|
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
|
||||||
|
from ..utils.hf_offline_patch import force_offline_if_cached
|
||||||
|
|
||||||
|
|
||||||
class MLXTTSBackend:
|
class MLXTTSBackend:
|
||||||
@@ -96,32 +96,13 @@ class MLXTTSBackend:
|
|||||||
model_name = f"qwen-tts-{model_size}"
|
model_name = f"qwen-tts-{model_size}"
|
||||||
is_cached = self._is_model_cached(model_size)
|
is_cached = self._is_model_cached(model_size)
|
||||||
|
|
||||||
# Force offline mode when cached to avoid network requests
|
with model_load_progress(model_name, is_cached):
|
||||||
original_hf_hub_offline = os.environ.get("HF_HUB_OFFLINE")
|
from mlx_audio.tts import load
|
||||||
if is_cached:
|
|
||||||
os.environ["HF_HUB_OFFLINE"] = "1"
|
|
||||||
logger.info("[PATCH] Model %s is cached, forcing HF_HUB_OFFLINE=1 to avoid network requests", model_size)
|
|
||||||
|
|
||||||
try:
|
logger.info("Loading MLX TTS model %s...", model_size)
|
||||||
with model_load_progress(model_name, is_cached):
|
|
||||||
from mlx_audio.tts import load
|
|
||||||
|
|
||||||
logger.info("Loading MLX TTS model %s...", model_size)
|
with force_offline_if_cached(is_cached, model_name):
|
||||||
|
self.model = load(model_path)
|
||||||
try:
|
|
||||||
self.model = load(model_path)
|
|
||||||
except Exception as load_error:
|
|
||||||
if is_cached and "offline" in str(load_error).lower():
|
|
||||||
logger.warning("[PATCH] Offline load failed, trying with network: %s", load_error)
|
|
||||||
os.environ.pop("HF_HUB_OFFLINE", None)
|
|
||||||
self.model = load(model_path)
|
|
||||||
else:
|
|
||||||
raise
|
|
||||||
finally:
|
|
||||||
if original_hf_hub_offline is not None:
|
|
||||||
os.environ["HF_HUB_OFFLINE"] = original_hf_hub_offline
|
|
||||||
else:
|
|
||||||
os.environ.pop("HF_HUB_OFFLINE", None)
|
|
||||||
|
|
||||||
self._current_model_size = model_size
|
self._current_model_size = model_size
|
||||||
self.model_size = model_size
|
self.model_size = model_size
|
||||||
@@ -329,7 +310,9 @@ class MLXSTTBackend:
|
|||||||
|
|
||||||
model_name = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
|
model_name = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
|
||||||
logger.info("Loading MLX Whisper model %s...", model_size)
|
logger.info("Loading MLX Whisper model %s...", model_size)
|
||||||
self.model = load(model_name)
|
|
||||||
|
with force_offline_if_cached(is_cached, progress_model_name):
|
||||||
|
self.model = load(model_name)
|
||||||
|
|
||||||
self.model_size = model_size
|
self.model_size = model_size
|
||||||
logger.info("MLX Whisper model %s loaded successfully", model_size)
|
logger.info("MLX Whisper model %s loaded successfully", model_size)
|
||||||
@@ -345,18 +328,20 @@ class MLXSTTBackend:
|
|||||||
self,
|
self,
|
||||||
audio_path: str,
|
audio_path: str,
|
||||||
language: Optional[str] = None,
|
language: Optional[str] = None,
|
||||||
|
model_size: Optional[str] = None,
|
||||||
) -> str:
|
) -> str:
|
||||||
"""
|
"""
|
||||||
Transcribe audio to text.
|
Transcribe audio to text.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
audio_path: Path to audio file
|
audio_path: Path to audio file
|
||||||
language: Optional language hint (en or zh)
|
language: Optional language hint
|
||||||
|
model_size: Optional model size override
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Transcribed text
|
Transcribed text
|
||||||
"""
|
"""
|
||||||
await self.load_model_async(None)
|
await self.load_model_async(model_size)
|
||||||
|
|
||||||
def _transcribe_sync():
|
def _transcribe_sync():
|
||||||
"""Run synchronous transcription in thread pool."""
|
"""Run synchronous transcription in thread pool."""
|
||||||
|
|||||||
@@ -19,6 +19,7 @@ from .base import (
|
|||||||
)
|
)
|
||||||
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
|
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
|
||||||
from ..utils.audio import load_audio
|
from ..utils.audio import load_audio
|
||||||
|
from ..utils.hf_offline_patch import force_offline_if_cached
|
||||||
|
|
||||||
|
|
||||||
class PyTorchTTSBackend:
|
class PyTorchTTSBackend:
|
||||||
@@ -96,18 +97,19 @@ class PyTorchTTSBackend:
|
|||||||
model_path = self._get_model_path(model_size)
|
model_path = self._get_model_path(model_size)
|
||||||
logger.info("Loading TTS model %s on %s...", model_size, self.device)
|
logger.info("Loading TTS model %s on %s...", model_size, self.device)
|
||||||
|
|
||||||
if self.device == "cpu":
|
with force_offline_if_cached(is_cached, model_name):
|
||||||
self.model = Qwen3TTSModel.from_pretrained(
|
if self.device == "cpu":
|
||||||
model_path,
|
self.model = Qwen3TTSModel.from_pretrained(
|
||||||
torch_dtype=torch.float32,
|
model_path,
|
||||||
low_cpu_mem_usage=False,
|
torch_dtype=torch.float32,
|
||||||
)
|
low_cpu_mem_usage=False,
|
||||||
else:
|
)
|
||||||
self.model = Qwen3TTSModel.from_pretrained(
|
else:
|
||||||
model_path,
|
self.model = Qwen3TTSModel.from_pretrained(
|
||||||
device_map=self.device,
|
model_path,
|
||||||
torch_dtype=torch.bfloat16,
|
device_map=self.device,
|
||||||
)
|
torch_dtype=torch.bfloat16,
|
||||||
|
)
|
||||||
|
|
||||||
self._current_model_size = model_size
|
self._current_model_size = model_size
|
||||||
self.model_size = model_size
|
self.model_size = model_size
|
||||||
@@ -282,8 +284,9 @@ class PyTorchSTTBackend:
|
|||||||
model_name = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
|
model_name = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
|
||||||
logger.info("Loading Whisper model %s on %s...", model_size, self.device)
|
logger.info("Loading Whisper model %s on %s...", model_size, self.device)
|
||||||
|
|
||||||
self.processor = WhisperProcessor.from_pretrained(model_name)
|
with force_offline_if_cached(is_cached, progress_model_name):
|
||||||
self.model = WhisperForConditionalGeneration.from_pretrained(model_name)
|
self.processor = WhisperProcessor.from_pretrained(model_name)
|
||||||
|
self.model = WhisperForConditionalGeneration.from_pretrained(model_name)
|
||||||
|
|
||||||
self.model.to(self.device)
|
self.model.to(self.device)
|
||||||
self.model_size = model_size
|
self.model_size = model_size
|
||||||
@@ -306,18 +309,20 @@ class PyTorchSTTBackend:
|
|||||||
self,
|
self,
|
||||||
audio_path: str,
|
audio_path: str,
|
||||||
language: Optional[str] = None,
|
language: Optional[str] = None,
|
||||||
|
model_size: Optional[str] = None,
|
||||||
) -> str:
|
) -> str:
|
||||||
"""
|
"""
|
||||||
Transcribe audio to text.
|
Transcribe audio to text.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
audio_path: Path to audio file
|
audio_path: Path to audio file
|
||||||
language: Optional language hint (en or zh)
|
language: Optional language hint
|
||||||
|
model_size: Optional model size override
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Transcribed text
|
Transcribed text
|
||||||
"""
|
"""
|
||||||
await self.load_model_async(None)
|
await self.load_model_async(model_size)
|
||||||
|
|
||||||
def _transcribe_sync():
|
def _transcribe_sync():
|
||||||
"""Run synchronous transcription in thread pool."""
|
"""Run synchronous transcription in thread pool."""
|
||||||
|
|||||||
+54
-6
@@ -34,9 +34,15 @@ def build_server(cuda=False):
|
|||||||
binary_name = "voicebox-server-cuda" if cuda else "voicebox-server"
|
binary_name = "voicebox-server-cuda" if cuda else "voicebox-server"
|
||||||
|
|
||||||
# PyInstaller arguments
|
# PyInstaller arguments
|
||||||
|
# CUDA builds use --onedir so we can split the output into two archives:
|
||||||
|
# 1. Server core (~200-400MB) — versioned with the app
|
||||||
|
# 2. CUDA libs (~2GB) — versioned independently (only redownloaded on
|
||||||
|
# CUDA toolkit / torch major version changes)
|
||||||
|
# CPU builds remain --onefile for simplicity.
|
||||||
|
pack_mode = "--onedir" if cuda else "--onefile"
|
||||||
args = [
|
args = [
|
||||||
"server.py", # Use server.py as entry point instead of main.py
|
"server.py", # Use server.py as entry point instead of main.py
|
||||||
"--onefile",
|
pack_mode,
|
||||||
"--name",
|
"--name",
|
||||||
binary_name,
|
binary_name,
|
||||||
]
|
]
|
||||||
@@ -127,7 +133,13 @@ def build_server(cuda=False):
|
|||||||
"uvicorn",
|
"uvicorn",
|
||||||
"--hidden-import",
|
"--hidden-import",
|
||||||
"sqlalchemy",
|
"sqlalchemy",
|
||||||
"--hidden-import",
|
# librosa uses lazy_loader which generates .pyi stub files at
|
||||||
|
# install time and reads them at runtime to discover submodules.
|
||||||
|
# --hidden-import alone doesn't bundle the stubs, causing
|
||||||
|
# "Cannot load imports from non-existent stub" at runtime.
|
||||||
|
"--collect-all",
|
||||||
|
"lazy_loader",
|
||||||
|
"--collect-all",
|
||||||
"librosa",
|
"librosa",
|
||||||
"--hidden-import",
|
"--hidden-import",
|
||||||
"soundfile",
|
"soundfile",
|
||||||
@@ -159,9 +171,9 @@ def build_server(cuda=False):
|
|||||||
"tqdm",
|
"tqdm",
|
||||||
"--hidden-import",
|
"--hidden-import",
|
||||||
"requests",
|
"requests",
|
||||||
"--collect-submodules",
|
# qwen_tts uses inspect.getsource() at runtime to locate
|
||||||
"qwen_tts",
|
# modeling_qwen3_tts.py — needs physical .py source files bundled
|
||||||
"--collect-data",
|
"--collect-all",
|
||||||
"qwen_tts",
|
"qwen_tts",
|
||||||
# Fix for pkg_resources and jaraco namespace packages
|
# Fix for pkg_resources and jaraco namespace packages
|
||||||
"--hidden-import",
|
"--hidden-import",
|
||||||
@@ -180,6 +192,42 @@ def build_server(cuda=False):
|
|||||||
# needed by LuxTTS for text-to-phoneme conversion
|
# needed by LuxTTS for text-to-phoneme conversion
|
||||||
"--collect-all",
|
"--collect-all",
|
||||||
"piper_phonemize",
|
"piper_phonemize",
|
||||||
|
# HumeAI TADA — speech-language model using Llama + flow matching
|
||||||
|
"--hidden-import",
|
||||||
|
"backend.backends.hume_backend",
|
||||||
|
"--hidden-import",
|
||||||
|
"tada",
|
||||||
|
"--hidden-import",
|
||||||
|
"tada.modules",
|
||||||
|
"--hidden-import",
|
||||||
|
"tada.modules.tada",
|
||||||
|
"--hidden-import",
|
||||||
|
"tada.modules.encoder",
|
||||||
|
"--hidden-import",
|
||||||
|
"tada.modules.decoder",
|
||||||
|
"--hidden-import",
|
||||||
|
"tada.modules.aligner",
|
||||||
|
"--hidden-import",
|
||||||
|
"tada.modules.acoustic_spkr_verf",
|
||||||
|
"--hidden-import",
|
||||||
|
"tada.nn",
|
||||||
|
"--hidden-import",
|
||||||
|
"tada.nn.vibevoice",
|
||||||
|
"--hidden-import",
|
||||||
|
"tada.utils",
|
||||||
|
"--hidden-import",
|
||||||
|
"tada.utils.gray_code",
|
||||||
|
"--hidden-import",
|
||||||
|
"tada.utils.text",
|
||||||
|
# DAC shim — provides dac.nn.layers.Snake1d without the real
|
||||||
|
# descript-audio-codec package (which pulls onnx/tensorboard via
|
||||||
|
# descript-audiotools). The shim is in backend/utils/dac_shim.py.
|
||||||
|
"--hidden-import",
|
||||||
|
"backend.utils.dac_shim",
|
||||||
|
"--hidden-import",
|
||||||
|
"torchaudio",
|
||||||
|
"--collect-submodules",
|
||||||
|
"tada",
|
||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -322,7 +370,7 @@ def build_server(cuda=False):
|
|||||||
"torchvision",
|
"torchvision",
|
||||||
"torchaudio",
|
"torchaudio",
|
||||||
"--index-url",
|
"--index-url",
|
||||||
"https://download.pytorch.org/whl/cu126",
|
"https://download.pytorch.org/whl/cu128",
|
||||||
"--force-reinstall",
|
"--force-reinstall",
|
||||||
"-q",
|
"-q",
|
||||||
],
|
],
|
||||||
|
|||||||
+4
-3
@@ -66,9 +66,9 @@ class GenerationRequest(BaseModel):
|
|||||||
text: str = Field(..., min_length=1, max_length=50000)
|
text: str = Field(..., min_length=1, max_length=50000)
|
||||||
language: str = Field(default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it|he|ar|da|el|fi|hi|ms|nl|no|pl|sv|sw|tr)$")
|
language: str = Field(default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it|he|ar|da|el|fi|hi|ms|nl|no|pl|sv|sw|tr)$")
|
||||||
seed: Optional[int] = Field(None, ge=0)
|
seed: Optional[int] = Field(None, ge=0)
|
||||||
model_size: Optional[str] = Field(default="1.7B", pattern="^(1\\.7B|0\\.6B)$")
|
model_size: Optional[str] = Field(default="1.7B", pattern="^(1\\.7B|0\\.6B|1B|3B)$")
|
||||||
instruct: Optional[str] = Field(None, max_length=500)
|
instruct: Optional[str] = Field(None, max_length=500)
|
||||||
engine: Optional[str] = Field(default="qwen", pattern="^(qwen|luxtts|chatterbox|chatterbox_turbo)$")
|
engine: Optional[str] = Field(default="qwen", pattern="^(qwen|luxtts|chatterbox|chatterbox_turbo|tada)$")
|
||||||
max_chunk_chars: int = Field(
|
max_chunk_chars: int = Field(
|
||||||
default=800, ge=100, le=5000, description="Max characters per chunk for long text splitting"
|
default=800, ge=100, le=5000, description="Max characters per chunk for long text splitting"
|
||||||
)
|
)
|
||||||
@@ -149,7 +149,8 @@ class HistoryListResponse(BaseModel):
|
|||||||
class TranscriptionRequest(BaseModel):
|
class TranscriptionRequest(BaseModel):
|
||||||
"""Request model for audio transcription."""
|
"""Request model for audio transcription."""
|
||||||
|
|
||||||
language: Optional[str] = Field(None, pattern="^(en|zh)$")
|
language: Optional[str] = Field(None, pattern="^(en|zh|ja|ko|de|fr|ru|pt|es|it)$")
|
||||||
|
model: Optional[str] = Field(None, pattern="^(base|small|medium|large|turbo)$")
|
||||||
|
|
||||||
|
|
||||||
class TranscriptionResponse(BaseModel):
|
class TranscriptionResponse(BaseModel):
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ sqlalchemy>=2.0.0
|
|||||||
alembic>=1.13.0
|
alembic>=1.13.0
|
||||||
|
|
||||||
# ML models
|
# ML models
|
||||||
torch>=2.1.0
|
torch>=2.7.0
|
||||||
transformers>=4.36.0,<=4.57.6
|
transformers>=4.36.0,<=4.57.6
|
||||||
accelerate>=0.26.0
|
accelerate>=0.26.0
|
||||||
huggingface_hub>=0.20.0
|
huggingface_hub>=0.20.0
|
||||||
@@ -33,6 +33,13 @@ s3tokenizer
|
|||||||
spacy-pkuseg
|
spacy-pkuseg
|
||||||
pyloudnorm
|
pyloudnorm
|
||||||
|
|
||||||
|
# HumeAI TADA sub-dependencies (hume-tada itself is installed
|
||||||
|
# --no-deps in the setup script because it pins torch>=2.7,<2.8.
|
||||||
|
# descript-audio-codec is NOT installed — it pulls onnx/tensorboard
|
||||||
|
# via descript-audiotools. A lightweight shim in utils/dac_shim.py
|
||||||
|
# provides the only class TADA uses: Snake1d.)
|
||||||
|
torchaudio
|
||||||
|
|
||||||
# Audio processing
|
# Audio processing
|
||||||
librosa>=0.10.0
|
librosa>=0.10.0
|
||||||
soundfile>=0.12.0
|
soundfile>=0.12.0
|
||||||
|
|||||||
@@ -1,12 +1,15 @@
|
|||||||
"""TTS generation endpoints."""
|
"""TTS generation endpoints."""
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
|
import logging
|
||||||
import uuid
|
import uuid
|
||||||
|
|
||||||
from fastapi import APIRouter, Depends, HTTPException
|
from fastapi import APIRouter, Depends, HTTPException
|
||||||
from fastapi.responses import StreamingResponse
|
from fastapi.responses import StreamingResponse
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
from .. import models
|
from .. import models
|
||||||
from ..services import history, profiles, tts
|
from ..services import history, profiles, tts
|
||||||
from ..database import Generation as DBGeneration, VoiceProfile as DBVoiceProfile, get_db
|
from ..database import Generation as DBGeneration, VoiceProfile as DBVoiceProfile, get_db
|
||||||
@@ -181,25 +184,28 @@ async def get_generation_status(generation_id: str, db: Session = Depends(get_db
|
|||||||
import json
|
import json
|
||||||
|
|
||||||
async def event_stream():
|
async def event_stream():
|
||||||
while True:
|
try:
|
||||||
db.expire_all()
|
while True:
|
||||||
gen = db.query(DBGeneration).filter_by(id=generation_id).first()
|
db.expire_all()
|
||||||
if not gen:
|
gen = db.query(DBGeneration).filter_by(id=generation_id).first()
|
||||||
yield f"data: {json.dumps({'status': 'not_found', 'id': generation_id})}\n\n"
|
if not gen:
|
||||||
return
|
yield f"data: {json.dumps({'status': 'not_found', 'id': generation_id})}\n\n"
|
||||||
|
return
|
||||||
|
|
||||||
payload = {
|
payload = {
|
||||||
"id": gen.id,
|
"id": gen.id,
|
||||||
"status": gen.status or "completed",
|
"status": gen.status or "completed",
|
||||||
"duration": gen.duration,
|
"duration": gen.duration,
|
||||||
"error": gen.error,
|
"error": gen.error,
|
||||||
}
|
}
|
||||||
yield f"data: {json.dumps(payload)}\n\n"
|
yield f"data: {json.dumps(payload)}\n\n"
|
||||||
|
|
||||||
if (gen.status or "completed") in ("completed", "failed"):
|
if (gen.status or "completed") in ("completed", "failed"):
|
||||||
return
|
return
|
||||||
|
|
||||||
await asyncio.sleep(1)
|
await asyncio.sleep(1)
|
||||||
|
except (BrokenPipeError, ConnectionResetError, asyncio.CancelledError):
|
||||||
|
logger.debug("SSE client disconnected for generation %s", generation_id)
|
||||||
|
|
||||||
return StreamingResponse(
|
return StreamingResponse(
|
||||||
event_stream(),
|
event_stream(),
|
||||||
@@ -265,9 +271,12 @@ async def stream_speech(
|
|||||||
wav_bytes = tts.audio_to_wav_bytes(audio, sample_rate)
|
wav_bytes = tts.audio_to_wav_bytes(audio, sample_rate)
|
||||||
|
|
||||||
async def _wav_stream():
|
async def _wav_stream():
|
||||||
chunk_size = 64 * 1024
|
try:
|
||||||
for i in range(0, len(wav_bytes), chunk_size):
|
chunk_size = 64 * 1024
|
||||||
yield wav_bytes[i : i + chunk_size]
|
for i in range(0, len(wav_bytes), chunk_size):
|
||||||
|
yield wav_bytes[i : i + chunk_size]
|
||||||
|
except (BrokenPipeError, ConnectionResetError, asyncio.CancelledError):
|
||||||
|
logger.debug("Client disconnected during audio stream")
|
||||||
|
|
||||||
return StreamingResponse(
|
return StreamingResponse(
|
||||||
_wav_stream(),
|
_wav_stream(),
|
||||||
|
|||||||
@@ -3,9 +3,11 @@
|
|||||||
import asyncio
|
import asyncio
|
||||||
import os
|
import os
|
||||||
import signal
|
import signal
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
import torch
|
import torch
|
||||||
from fastapi import APIRouter, Depends
|
from fastapi import APIRouter, Depends
|
||||||
|
from fastapi.responses import FileResponse
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from .. import config, models
|
from .. import config, models
|
||||||
@@ -15,12 +17,18 @@ from ..utils.platform_detect import get_backend_type
|
|||||||
|
|
||||||
router = APIRouter()
|
router = APIRouter()
|
||||||
|
|
||||||
|
# Frontend build directory — present in Docker, absent in dev/API-only mode
|
||||||
|
_frontend_dir = Path(__file__).resolve().parent.parent.parent / "frontend"
|
||||||
|
|
||||||
|
|
||||||
@router.get("/")
|
@router.get("/")
|
||||||
async def root():
|
async def root():
|
||||||
"""Root endpoint."""
|
"""Root endpoint — serves SPA index.html in Docker, JSON otherwise."""
|
||||||
from .. import __version__
|
from .. import __version__
|
||||||
|
|
||||||
|
index = _frontend_dir / "index.html"
|
||||||
|
if index.is_file():
|
||||||
|
return FileResponse(index, media_type="text/html")
|
||||||
return {"message": "voicebox API", "version": __version__}
|
return {"message": "voicebox API", "version": __version__}
|
||||||
|
|
||||||
|
|
||||||
@@ -199,7 +207,7 @@ async def filesystem_health():
|
|||||||
|
|
||||||
checks.append(
|
checks.append(
|
||||||
models.DirectoryCheck(
|
models.DirectoryCheck(
|
||||||
path=str(dir_path),
|
path=str(dir_path.resolve()),
|
||||||
exists=exists,
|
exists=exists,
|
||||||
writable=writable,
|
writable=writable,
|
||||||
error=error,
|
error=error,
|
||||||
|
|||||||
@@ -102,6 +102,10 @@ async def delete_profile(
|
|||||||
return {"message": "Profile deleted successfully"}
|
return {"message": "Profile deleted successfully"}
|
||||||
|
|
||||||
|
|
||||||
|
SAMPLE_MAX_FILE_SIZE = 50 * 1024 * 1024 # 50 MB
|
||||||
|
SAMPLE_UPLOAD_CHUNK_SIZE = 1024 * 1024 # 1 MB
|
||||||
|
|
||||||
|
|
||||||
@router.post("/profiles/{profile_id}/samples", response_model=models.ProfileSampleResponse)
|
@router.post("/profiles/{profile_id}/samples", response_model=models.ProfileSampleResponse)
|
||||||
async def add_profile_sample(
|
async def add_profile_sample(
|
||||||
profile_id: str,
|
profile_id: str,
|
||||||
@@ -115,8 +119,16 @@ async def add_profile_sample(
|
|||||||
file_suffix = _uploaded_ext if _uploaded_ext in _allowed_audio_exts else ".wav"
|
file_suffix = _uploaded_ext if _uploaded_ext in _allowed_audio_exts else ".wav"
|
||||||
|
|
||||||
with tempfile.NamedTemporaryFile(suffix=file_suffix, delete=False) as tmp:
|
with tempfile.NamedTemporaryFile(suffix=file_suffix, delete=False) as tmp:
|
||||||
content = await file.read()
|
total_size = 0
|
||||||
tmp.write(content)
|
while chunk := await file.read(SAMPLE_UPLOAD_CHUNK_SIZE):
|
||||||
|
total_size += len(chunk)
|
||||||
|
if total_size > SAMPLE_MAX_FILE_SIZE:
|
||||||
|
Path(tmp.name).unlink(missing_ok=True)
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=413,
|
||||||
|
detail=f"File too large (max {SAMPLE_MAX_FILE_SIZE // (1024 * 1024)} MB)",
|
||||||
|
)
|
||||||
|
tmp.write(chunk)
|
||||||
tmp_path = tmp.name
|
tmp_path = tmp.name
|
||||||
|
|
||||||
try:
|
try:
|
||||||
|
|||||||
@@ -20,6 +20,7 @@ UPLOAD_CHUNK_SIZE = 1024 * 1024 # 1MB
|
|||||||
async def transcribe_audio(
|
async def transcribe_audio(
|
||||||
file: UploadFile = File(...),
|
file: UploadFile = File(...),
|
||||||
language: str | None = Form(None),
|
language: str | None = Form(None),
|
||||||
|
model: str | None = Form(None),
|
||||||
):
|
):
|
||||||
"""Transcribe audio file to text."""
|
"""Transcribe audio file to text."""
|
||||||
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
|
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
|
||||||
@@ -29,14 +30,23 @@ async def transcribe_audio(
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
from ..utils.audio import load_audio
|
from ..utils.audio import load_audio
|
||||||
|
from ..backends import WHISPER_HF_REPOS
|
||||||
|
|
||||||
audio, sr = await asyncio.to_thread(load_audio, tmp_path)
|
audio, sr = await asyncio.to_thread(load_audio, tmp_path)
|
||||||
duration = len(audio) / sr
|
duration = len(audio) / sr
|
||||||
|
|
||||||
whisper_model = transcribe.get_whisper_model()
|
whisper_model = transcribe.get_whisper_model()
|
||||||
model_size = whisper_model.model_size
|
model_size = model if model else whisper_model.model_size
|
||||||
|
|
||||||
if not whisper_model.is_loaded() and not whisper_model._is_model_cached(model_size):
|
valid_sizes = list(WHISPER_HF_REPOS.keys())
|
||||||
|
if model_size not in valid_sizes:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=400,
|
||||||
|
detail=f"Invalid model size '{model_size}'. Must be one of: {', '.join(valid_sizes)}",
|
||||||
|
)
|
||||||
|
|
||||||
|
already_loaded = whisper_model.is_loaded() and whisper_model.model_size == model_size
|
||||||
|
if not already_loaded and not whisper_model._is_model_cached(model_size):
|
||||||
progress_model_name = f"whisper-{model_size}"
|
progress_model_name = f"whisper-{model_size}"
|
||||||
task_manager = get_task_manager()
|
task_manager = get_task_manager()
|
||||||
|
|
||||||
@@ -59,7 +69,7 @@ async def transcribe_audio(
|
|||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
text = await whisper_model.transcribe(tmp_path, language)
|
text = await whisper_model.transcribe(tmp_path, language, model_size)
|
||||||
|
|
||||||
return models.TranscriptionResponse(
|
return models.TranscriptionResponse(
|
||||||
text=text,
|
text=text,
|
||||||
|
|||||||
+266
-119
@@ -1,16 +1,22 @@
|
|||||||
"""
|
"""
|
||||||
CUDA backend binary download, assembly, and verification.
|
CUDA backend download, assembly, and verification.
|
||||||
|
|
||||||
Downloads split parts of the CUDA-enabled voicebox-server binary from
|
Downloads two archives from GitHub Releases:
|
||||||
GitHub Releases, reassembles them, verifies integrity via SHA-256,
|
1. Server core (voicebox-server-cuda.tar.gz) — the exe + non-NVIDIA deps,
|
||||||
and places the binary in the app's data directory for use on next
|
versioned with the app.
|
||||||
backend restart.
|
2. CUDA libs (cuda-libs-{version}.tar.gz) — NVIDIA runtime libraries,
|
||||||
|
versioned independently (only redownloaded on CUDA toolkit bump).
|
||||||
|
|
||||||
|
Both archives are extracted into {data_dir}/backends/cuda/ which forms the
|
||||||
|
complete PyInstaller --onedir directory structure that torch expects.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import hashlib
|
import hashlib
|
||||||
|
import json
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
import sys
|
import sys
|
||||||
|
import tarfile
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Optional
|
from typing import Optional
|
||||||
|
|
||||||
@@ -24,6 +30,10 @@ GITHUB_RELEASES_URL = "https://github.com/jamiepine/voicebox/releases/download"
|
|||||||
|
|
||||||
PROGRESS_KEY = "cuda-backend"
|
PROGRESS_KEY = "cuda-backend"
|
||||||
|
|
||||||
|
# The current expected CUDA libs version. Bump this when we change the
|
||||||
|
# CUDA toolkit version or torch's CUDA dependency changes (e.g. cu126 -> cu128).
|
||||||
|
CUDA_LIBS_VERSION = "cu128-v1"
|
||||||
|
|
||||||
|
|
||||||
def get_backends_dir() -> Path:
|
def get_backends_dir() -> Path:
|
||||||
"""Directory where downloaded backend binaries are stored."""
|
"""Directory where downloaded backend binaries are stored."""
|
||||||
@@ -32,21 +42,46 @@ def get_backends_dir() -> Path:
|
|||||||
return d
|
return d
|
||||||
|
|
||||||
|
|
||||||
def get_cuda_binary_name() -> str:
|
def get_cuda_dir() -> Path:
|
||||||
"""Platform-specific CUDA binary filename."""
|
"""Directory where the CUDA backend (onedir) is extracted."""
|
||||||
|
d = get_backends_dir() / "cuda"
|
||||||
|
d.mkdir(parents=True, exist_ok=True)
|
||||||
|
return d
|
||||||
|
|
||||||
|
|
||||||
|
def get_cuda_exe_name() -> str:
|
||||||
|
"""Platform-specific CUDA executable filename."""
|
||||||
if sys.platform == "win32":
|
if sys.platform == "win32":
|
||||||
return "voicebox-server-cuda.exe"
|
return "voicebox-server-cuda.exe"
|
||||||
return "voicebox-server-cuda"
|
return "voicebox-server-cuda"
|
||||||
|
|
||||||
|
|
||||||
def get_cuda_binary_path() -> Optional[Path]:
|
def get_cuda_binary_path() -> Optional[Path]:
|
||||||
"""Return path to CUDA binary if it exists."""
|
"""Return path to the CUDA executable if it exists inside the onedir."""
|
||||||
p = get_backends_dir() / get_cuda_binary_name()
|
p = get_cuda_dir() / get_cuda_exe_name()
|
||||||
if p.exists():
|
if p.exists():
|
||||||
return p
|
return p
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def get_cuda_libs_manifest_path() -> Path:
|
||||||
|
"""Path to the cuda-libs.json manifest inside the CUDA dir."""
|
||||||
|
return get_cuda_dir() / "cuda-libs.json"
|
||||||
|
|
||||||
|
|
||||||
|
def get_installed_cuda_libs_version() -> Optional[str]:
|
||||||
|
"""Read the installed CUDA libs version from cuda-libs.json, or None."""
|
||||||
|
manifest_path = get_cuda_libs_manifest_path()
|
||||||
|
if not manifest_path.exists():
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
data = json.loads(manifest_path.read_text())
|
||||||
|
return data.get("version")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Could not read cuda-libs.json: {e}")
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
def is_cuda_active() -> bool:
|
def is_cuda_active() -> bool:
|
||||||
"""Check if the current process is the CUDA binary.
|
"""Check if the current process is the CUDA binary.
|
||||||
|
|
||||||
@@ -60,25 +95,151 @@ def get_cuda_status() -> dict:
|
|||||||
progress_manager = get_progress_manager()
|
progress_manager = get_progress_manager()
|
||||||
cuda_path = get_cuda_binary_path()
|
cuda_path = get_cuda_binary_path()
|
||||||
progress = progress_manager.get_progress(PROGRESS_KEY)
|
progress = progress_manager.get_progress(PROGRESS_KEY)
|
||||||
|
cuda_libs_version = get_installed_cuda_libs_version()
|
||||||
|
|
||||||
return {
|
return {
|
||||||
"available": cuda_path is not None,
|
"available": cuda_path is not None,
|
||||||
"active": is_cuda_active(),
|
"active": is_cuda_active(),
|
||||||
"binary_path": str(cuda_path) if cuda_path else None,
|
"binary_path": str(cuda_path) if cuda_path else None,
|
||||||
|
"cuda_libs_version": cuda_libs_version,
|
||||||
"downloading": progress is not None and progress.get("status") == "downloading",
|
"downloading": progress is not None and progress.get("status") == "downloading",
|
||||||
"download_progress": progress,
|
"download_progress": progress,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
async def download_cuda_binary(version: Optional[str] = None):
|
def _needs_server_download(version: Optional[str] = None) -> bool:
|
||||||
"""Download the CUDA backend binary from GitHub Releases.
|
"""Check if the server core archive needs to be (re)downloaded."""
|
||||||
|
cuda_path = get_cuda_binary_path()
|
||||||
|
if not cuda_path:
|
||||||
|
return True
|
||||||
|
# Check if the binary version matches the expected app version
|
||||||
|
installed = get_cuda_binary_version()
|
||||||
|
expected = version or __version__
|
||||||
|
if expected.startswith("v"):
|
||||||
|
expected = expected[1:]
|
||||||
|
return installed != expected
|
||||||
|
|
||||||
Downloads split parts listed in a manifest file, concatenates them,
|
|
||||||
and verifies the SHA-256 checksum for integrity. Atomic write
|
def _needs_cuda_libs_download() -> bool:
|
||||||
(temp file -> rename).
|
"""Check if the CUDA libs archive needs to be (re)downloaded."""
|
||||||
|
installed = get_installed_cuda_libs_version()
|
||||||
|
if installed is None:
|
||||||
|
return True
|
||||||
|
return installed != CUDA_LIBS_VERSION
|
||||||
|
|
||||||
|
|
||||||
|
async def _download_and_extract_archive(
|
||||||
|
client,
|
||||||
|
url: str,
|
||||||
|
sha256_url: Optional[str],
|
||||||
|
dest_dir: Path,
|
||||||
|
label: str,
|
||||||
|
progress_offset: int,
|
||||||
|
total_size: int,
|
||||||
|
):
|
||||||
|
"""Download a .tar.gz archive and extract it into dest_dir.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
version: Version tag (e.g. "v0.2.0"). Defaults to current app version.
|
client: httpx.AsyncClient
|
||||||
|
url: URL of the .tar.gz archive
|
||||||
|
sha256_url: URL of the .sha256 checksum file (optional)
|
||||||
|
dest_dir: Directory to extract into
|
||||||
|
label: Human-readable label for progress updates
|
||||||
|
progress_offset: Byte offset for progress reporting (when downloading
|
||||||
|
multiple archives sequentially)
|
||||||
|
total_size: Total bytes across all downloads (for progress bar)
|
||||||
|
"""
|
||||||
|
progress = get_progress_manager()
|
||||||
|
temp_path = dest_dir / f".download-{label.replace(' ', '-')}.tmp"
|
||||||
|
|
||||||
|
# Clean up leftover partial download
|
||||||
|
if temp_path.exists():
|
||||||
|
temp_path.unlink()
|
||||||
|
|
||||||
|
# Fetch expected checksum (fail-fast: never extract an unverified archive)
|
||||||
|
expected_sha = None
|
||||||
|
if sha256_url:
|
||||||
|
try:
|
||||||
|
sha_resp = await client.get(sha256_url)
|
||||||
|
sha_resp.raise_for_status()
|
||||||
|
expected_sha = sha_resp.text.strip().split()[0]
|
||||||
|
logger.info(f"{label}: expected SHA-256: {expected_sha[:16]}...")
|
||||||
|
except Exception as e:
|
||||||
|
raise RuntimeError(f"{label}: failed to fetch checksum from {sha256_url}") from e
|
||||||
|
|
||||||
|
# Stream download, verify, and extract — always clean up temp file
|
||||||
|
downloaded = 0
|
||||||
|
try:
|
||||||
|
async with client.stream("GET", url) as response:
|
||||||
|
response.raise_for_status()
|
||||||
|
with open(temp_path, "wb") as f:
|
||||||
|
async for chunk in response.aiter_bytes(chunk_size=1024 * 1024):
|
||||||
|
f.write(chunk)
|
||||||
|
downloaded += len(chunk)
|
||||||
|
progress.update_progress(
|
||||||
|
PROGRESS_KEY,
|
||||||
|
current=progress_offset + downloaded,
|
||||||
|
total=total_size,
|
||||||
|
filename=f"Downloading {label}",
|
||||||
|
status="downloading",
|
||||||
|
)
|
||||||
|
|
||||||
|
# Verify integrity
|
||||||
|
if expected_sha:
|
||||||
|
progress.update_progress(
|
||||||
|
PROGRESS_KEY,
|
||||||
|
current=progress_offset + downloaded,
|
||||||
|
total=total_size,
|
||||||
|
filename=f"Verifying {label}...",
|
||||||
|
status="downloading",
|
||||||
|
)
|
||||||
|
sha256 = hashlib.sha256()
|
||||||
|
with open(temp_path, "rb") as f:
|
||||||
|
while True:
|
||||||
|
data = f.read(1024 * 1024)
|
||||||
|
if not data:
|
||||||
|
break
|
||||||
|
sha256.update(data)
|
||||||
|
actual = sha256.hexdigest()
|
||||||
|
if actual != expected_sha:
|
||||||
|
raise ValueError(
|
||||||
|
f"{label} integrity check failed: expected {expected_sha[:16]}..., got {actual[:16]}..."
|
||||||
|
)
|
||||||
|
logger.info(f"{label}: integrity verified")
|
||||||
|
|
||||||
|
# Extract (use data filter for path traversal protection on Python 3.12+)
|
||||||
|
progress.update_progress(
|
||||||
|
PROGRESS_KEY,
|
||||||
|
current=progress_offset + downloaded,
|
||||||
|
total=total_size,
|
||||||
|
filename=f"Extracting {label}...",
|
||||||
|
status="downloading",
|
||||||
|
)
|
||||||
|
with tarfile.open(temp_path, "r:gz") as tar:
|
||||||
|
if sys.version_info >= (3, 12):
|
||||||
|
tar.extractall(path=dest_dir, filter="data")
|
||||||
|
else:
|
||||||
|
tar.extractall(path=dest_dir)
|
||||||
|
|
||||||
|
logger.info(f"{label}: extracted to {dest_dir}")
|
||||||
|
finally:
|
||||||
|
if temp_path.exists():
|
||||||
|
temp_path.unlink()
|
||||||
|
return downloaded
|
||||||
|
|
||||||
|
|
||||||
|
async def download_cuda_binary(version: Optional[str] = None):
|
||||||
|
"""Download the CUDA backend (server core + CUDA libs if needed).
|
||||||
|
|
||||||
|
Downloads both archives from GitHub Releases, extracts them into
|
||||||
|
{data_dir}/backends/cuda/, and writes the cuda-libs.json manifest.
|
||||||
|
|
||||||
|
Only downloads what's needed:
|
||||||
|
- Server core: always redownloaded (versioned with app)
|
||||||
|
- CUDA libs: only if missing or version mismatch
|
||||||
|
|
||||||
|
Args:
|
||||||
|
version: Version tag (e.g. "v0.3.0"). Defaults to current app version.
|
||||||
"""
|
"""
|
||||||
import httpx
|
import httpx
|
||||||
|
|
||||||
@@ -86,114 +247,91 @@ async def download_cuda_binary(version: Optional[str] = None):
|
|||||||
version = f"v{__version__}"
|
version = f"v{__version__}"
|
||||||
|
|
||||||
progress = get_progress_manager()
|
progress = get_progress_manager()
|
||||||
binary_name = get_cuda_binary_name()
|
cuda_dir = get_cuda_dir()
|
||||||
dest_dir = get_backends_dir()
|
|
||||||
final_path = dest_dir / binary_name
|
|
||||||
temp_path = dest_dir / f"{binary_name}.download"
|
|
||||||
|
|
||||||
# Clean up any leftover partial download
|
need_server = _needs_server_download(version)
|
||||||
if temp_path.exists():
|
need_libs = _needs_cuda_libs_download()
|
||||||
temp_path.unlink()
|
|
||||||
|
|
||||||
logger.info(f"Starting CUDA backend download for {version}")
|
if not need_server and not need_libs:
|
||||||
|
logger.info("CUDA backend is up to date, nothing to download")
|
||||||
|
return
|
||||||
|
|
||||||
|
logger.info(
|
||||||
|
f"Starting CUDA backend download for {version} "
|
||||||
|
f"(server={'yes' if need_server else 'cached'}, "
|
||||||
|
f"libs={'yes' if need_libs else 'cached'})"
|
||||||
|
)
|
||||||
progress.update_progress(
|
progress.update_progress(
|
||||||
PROGRESS_KEY, current=0, total=0,
|
PROGRESS_KEY,
|
||||||
filename="Fetching manifest...", status="downloading",
|
current=0,
|
||||||
|
total=0,
|
||||||
|
filename="Preparing download...",
|
||||||
|
status="downloading",
|
||||||
)
|
)
|
||||||
|
|
||||||
base_url = f"{GITHUB_RELEASES_URL}/{version}"
|
base_url = f"{GITHUB_RELEASES_URL}/{version}"
|
||||||
stem = Path(binary_name).stem # voicebox-server-cuda
|
server_archive = "voicebox-server-cuda.tar.gz"
|
||||||
|
libs_archive = f"cuda-libs-{CUDA_LIBS_VERSION}.tar.gz"
|
||||||
|
|
||||||
try:
|
try:
|
||||||
async with httpx.AsyncClient(follow_redirects=True, timeout=30.0) as client:
|
async with httpx.AsyncClient(follow_redirects=True, timeout=30.0) as client:
|
||||||
# Fetch the manifest (list of split part filenames)
|
# Estimate total download size
|
||||||
manifest_url = f"{base_url}/{stem}.manifest"
|
|
||||||
manifest_resp = await client.get(manifest_url)
|
|
||||||
manifest_resp.raise_for_status()
|
|
||||||
parts = [p.strip() for p in manifest_resp.text.strip().splitlines() if p.strip()]
|
|
||||||
|
|
||||||
if not parts:
|
|
||||||
raise ValueError("Empty manifest — no split parts found")
|
|
||||||
|
|
||||||
logger.info(f"Found {len(parts)} split parts to download")
|
|
||||||
|
|
||||||
# Fetch expected checksum (optional — for integrity verification)
|
|
||||||
expected_sha = None
|
|
||||||
try:
|
|
||||||
sha_url = f"{base_url}/{stem}.sha256"
|
|
||||||
sha_resp = await client.get(sha_url)
|
|
||||||
if sha_resp.status_code == 200:
|
|
||||||
# Format: "sha256hex filename\n"
|
|
||||||
expected_sha = sha_resp.text.strip().split()[0]
|
|
||||||
logger.info(f"Expected SHA-256: {expected_sha[:16]}...")
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning(f"Could not fetch checksum file — skipping verification: {e}")
|
|
||||||
|
|
||||||
# Get total size across all parts by issuing HEAD requests
|
|
||||||
total_size = 0
|
total_size = 0
|
||||||
for part_name in parts:
|
if need_server:
|
||||||
try:
|
try:
|
||||||
head_resp = await client.head(f"{base_url}/{part_name}")
|
head = await client.head(f"{base_url}/{server_archive}")
|
||||||
content_length = int(head_resp.headers.get("content-length", 0))
|
total_size += int(head.headers.get("content-length", 0))
|
||||||
total_size += content_length
|
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
if need_libs:
|
||||||
|
try:
|
||||||
|
head = await client.head(f"{base_url}/{libs_archive}")
|
||||||
|
total_size += int(head.headers.get("content-length", 0))
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
logger.info(f"Total download size: {total_size / 1024 / 1024:.1f} MB")
|
logger.info(f"Total download size: {total_size / 1024 / 1024:.1f} MB")
|
||||||
|
|
||||||
# Download and concatenate parts
|
offset = 0
|
||||||
total_downloaded = 0
|
|
||||||
with open(temp_path, "wb") as f:
|
|
||||||
for i, part_name in enumerate(parts):
|
|
||||||
part_url = f"{base_url}/{part_name}"
|
|
||||||
logger.info(f"Downloading part {i + 1}/{len(parts)}: {part_name}")
|
|
||||||
|
|
||||||
async with client.stream("GET", part_url) as response:
|
# Download server core
|
||||||
response.raise_for_status()
|
if need_server:
|
||||||
async for chunk in response.aiter_bytes(chunk_size=1024 * 1024):
|
server_downloaded = await _download_and_extract_archive(
|
||||||
f.write(chunk)
|
client,
|
||||||
total_downloaded += len(chunk)
|
url=f"{base_url}/{server_archive}",
|
||||||
progress.update_progress(
|
sha256_url=f"{base_url}/{server_archive}.sha256",
|
||||||
PROGRESS_KEY, current=total_downloaded, total=total_size,
|
dest_dir=cuda_dir,
|
||||||
filename=f"Downloading CUDA backend ({i + 1}/{len(parts)})",
|
label="CUDA server",
|
||||||
status="downloading",
|
progress_offset=offset,
|
||||||
)
|
total_size=total_size,
|
||||||
|
|
||||||
# Verify integrity if checksum was available
|
|
||||||
if expected_sha:
|
|
||||||
progress.update_progress(
|
|
||||||
PROGRESS_KEY, current=total_downloaded, total=total_downloaded,
|
|
||||||
filename="Verifying integrity...", status="downloading",
|
|
||||||
)
|
|
||||||
sha256 = hashlib.sha256()
|
|
||||||
with open(temp_path, "rb") as f:
|
|
||||||
while True:
|
|
||||||
chunk = f.read(1024 * 1024)
|
|
||||||
if not chunk:
|
|
||||||
break
|
|
||||||
sha256.update(chunk)
|
|
||||||
|
|
||||||
actual = sha256.hexdigest()
|
|
||||||
if actual != expected_sha:
|
|
||||||
raise ValueError(
|
|
||||||
f"Integrity check failed: expected {expected_sha[:16]}..., "
|
|
||||||
f"got {actual[:16]}..."
|
|
||||||
)
|
)
|
||||||
logger.info(f"Integrity verified: {actual[:16]}...")
|
offset += server_downloaded
|
||||||
|
|
||||||
# Atomic move into place (replace handles existing target on all platforms)
|
# Make executable on Unix
|
||||||
temp_path.replace(final_path)
|
exe_path = cuda_dir / get_cuda_exe_name()
|
||||||
|
if sys.platform != "win32" and exe_path.exists():
|
||||||
|
exe_path.chmod(0o755)
|
||||||
|
|
||||||
# Make executable on Unix
|
# Download CUDA libs
|
||||||
if sys.platform != "win32":
|
if need_libs:
|
||||||
final_path.chmod(0o755)
|
await _download_and_extract_archive(
|
||||||
|
client,
|
||||||
|
url=f"{base_url}/{libs_archive}",
|
||||||
|
sha256_url=f"{base_url}/{libs_archive}.sha256",
|
||||||
|
dest_dir=cuda_dir,
|
||||||
|
label="CUDA libraries",
|
||||||
|
progress_offset=offset,
|
||||||
|
total_size=total_size,
|
||||||
|
)
|
||||||
|
|
||||||
logger.info(f"CUDA backend downloaded to {final_path}")
|
# Write local cuda-libs.json manifest
|
||||||
|
manifest = {"version": CUDA_LIBS_VERSION}
|
||||||
|
get_cuda_libs_manifest_path().write_text(json.dumps(manifest, indent=2) + "\n")
|
||||||
|
|
||||||
|
logger.info(f"CUDA backend ready at {cuda_dir}")
|
||||||
progress.mark_complete(PROGRESS_KEY)
|
progress.mark_complete(PROGRESS_KEY)
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
# Clean up on failure
|
|
||||||
if temp_path.exists():
|
|
||||||
temp_path.unlink()
|
|
||||||
logger.error(f"CUDA backend download failed: {e}")
|
logger.error(f"CUDA backend download failed: {e}")
|
||||||
progress.mark_error(PROGRESS_KEY, str(e))
|
progress.mark_error(PROGRESS_KEY, str(e))
|
||||||
raise
|
raise
|
||||||
@@ -202,15 +340,19 @@ async def download_cuda_binary(version: Optional[str] = None):
|
|||||||
def get_cuda_binary_version() -> Optional[str]:
|
def get_cuda_binary_version() -> Optional[str]:
|
||||||
"""Get the version of the installed CUDA binary, or None if not installed."""
|
"""Get the version of the installed CUDA binary, or None if not installed."""
|
||||||
import subprocess
|
import subprocess
|
||||||
|
|
||||||
cuda_path = get_cuda_binary_path()
|
cuda_path = get_cuda_binary_path()
|
||||||
if not cuda_path:
|
if not cuda_path:
|
||||||
return None
|
return None
|
||||||
try:
|
try:
|
||||||
result = subprocess.run(
|
result = subprocess.run(
|
||||||
[str(cuda_path), "--version"],
|
[str(cuda_path), "--version"],
|
||||||
capture_output=True, text=True, timeout=30,
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
timeout=30,
|
||||||
|
cwd=str(cuda_path.parent), # Run from the onedir directory
|
||||||
)
|
)
|
||||||
# Output format: "voicebox-server 0.2.0"
|
# Output format: "voicebox-server 0.3.0"
|
||||||
for line in result.stdout.strip().splitlines():
|
for line in result.stdout.strip().splitlines():
|
||||||
if "voicebox-server" in line:
|
if "voicebox-server" in line:
|
||||||
return line.split()[-1]
|
return line.split()[-1]
|
||||||
@@ -222,26 +364,29 @@ def get_cuda_binary_version() -> Optional[str]:
|
|||||||
async def check_and_update_cuda_binary():
|
async def check_and_update_cuda_binary():
|
||||||
"""Check if the CUDA binary is outdated and auto-download if so.
|
"""Check if the CUDA binary is outdated and auto-download if so.
|
||||||
|
|
||||||
Called on server startup. If a CUDA binary exists but its version
|
Called on server startup. Checks both server version and CUDA libs
|
||||||
doesn't match the current app version, triggers a background download
|
version. Downloads only what's needed.
|
||||||
of the updated CUDA binary. The download progress is visible to the
|
|
||||||
frontend via the existing SSE progress endpoint.
|
|
||||||
"""
|
"""
|
||||||
cuda_path = get_cuda_binary_path()
|
cuda_path = get_cuda_binary_path()
|
||||||
if not cuda_path:
|
if not cuda_path:
|
||||||
return # No CUDA binary installed, nothing to update
|
return # No CUDA binary installed, nothing to update
|
||||||
|
|
||||||
cuda_version = get_cuda_binary_version()
|
need_server = _needs_server_download()
|
||||||
current_version = __version__
|
need_libs = _needs_cuda_libs_download()
|
||||||
|
|
||||||
if cuda_version == current_version:
|
if not need_server and not need_libs:
|
||||||
logger.info(f"CUDA binary is up to date (v{current_version})")
|
logger.info(f"CUDA binary is up to date (server=v{__version__}, libs={get_installed_cuda_libs_version()})")
|
||||||
return
|
return
|
||||||
|
|
||||||
logger.info(
|
reasons = []
|
||||||
f"CUDA binary version mismatch: binary=v{cuda_version}, app=v{current_version}. "
|
if need_server:
|
||||||
f"Auto-downloading updated CUDA backend..."
|
cuda_version = get_cuda_binary_version()
|
||||||
)
|
reasons.append(f"server v{cuda_version} != v{__version__}")
|
||||||
|
if need_libs:
|
||||||
|
installed_libs = get_installed_cuda_libs_version()
|
||||||
|
reasons.append(f"libs {installed_libs} != {CUDA_LIBS_VERSION}")
|
||||||
|
|
||||||
|
logger.info(f"CUDA backend needs update ({', '.join(reasons)}). Auto-downloading...")
|
||||||
|
|
||||||
try:
|
try:
|
||||||
await download_cuda_binary()
|
await download_cuda_binary()
|
||||||
@@ -250,10 +395,12 @@ async def check_and_update_cuda_binary():
|
|||||||
|
|
||||||
|
|
||||||
async def delete_cuda_binary() -> bool:
|
async def delete_cuda_binary() -> bool:
|
||||||
"""Delete the downloaded CUDA binary. Returns True if deleted."""
|
"""Delete the downloaded CUDA backend directory. Returns True if deleted."""
|
||||||
path = get_cuda_binary_path()
|
import shutil
|
||||||
if path and path.exists():
|
|
||||||
path.unlink()
|
cuda_dir = get_cuda_dir()
|
||||||
logger.info(f"Deleted CUDA binary: {path}")
|
if cuda_dir.exists() and any(cuda_dir.iterdir()):
|
||||||
|
shutil.rmtree(cuda_dir)
|
||||||
|
logger.info(f"Deleted CUDA backend directory: {cuda_dir}")
|
||||||
return True
|
return True
|
||||||
return False
|
return False
|
||||||
|
|||||||
@@ -22,7 +22,7 @@ from ..database import (
|
|||||||
Generation as DBGeneration,
|
Generation as DBGeneration,
|
||||||
)
|
)
|
||||||
from ..models import EffectConfig
|
from ..models import EffectConfig
|
||||||
from ..utils.audio import validate_reference_audio, load_audio, save_audio
|
from ..utils.audio import validate_reference_audio, validate_and_load_reference_audio, load_audio, save_audio
|
||||||
from ..utils.images import validate_image, process_avatar
|
from ..utils.images import validate_image, process_avatar
|
||||||
from ..utils.cache import _get_cache_dir, clear_profile_cache
|
from ..utils.cache import _get_cache_dir, clear_profile_cache
|
||||||
from .tts import get_tts_model
|
from .tts import get_tts_model
|
||||||
@@ -117,11 +117,16 @@ async def add_profile_sample(
|
|||||||
Returns:
|
Returns:
|
||||||
Created sample
|
Created sample
|
||||||
"""
|
"""
|
||||||
|
import asyncio
|
||||||
|
|
||||||
profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
|
profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
|
||||||
if not profile:
|
if not profile:
|
||||||
raise ValueError(f"Profile {profile_id} not found")
|
raise ValueError(f"Profile {profile_id} not found")
|
||||||
|
|
||||||
is_valid, error_msg = validate_reference_audio(audio_path)
|
# Validate and load audio in a single pass, off the event loop
|
||||||
|
is_valid, error_msg, audio, sr = await asyncio.to_thread(
|
||||||
|
validate_and_load_reference_audio, audio_path
|
||||||
|
)
|
||||||
if not is_valid:
|
if not is_valid:
|
||||||
raise ValueError(f"Invalid reference audio: {error_msg}")
|
raise ValueError(f"Invalid reference audio: {error_msg}")
|
||||||
|
|
||||||
@@ -130,8 +135,7 @@ async def add_profile_sample(
|
|||||||
profile_dir.mkdir(parents=True, exist_ok=True)
|
profile_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
dest_path = profile_dir / f"{sample_id}.wav"
|
dest_path = profile_dir / f"{sample_id}.wav"
|
||||||
audio, sr = load_audio(audio_path)
|
await asyncio.to_thread(save_audio, audio, str(dest_path), sr)
|
||||||
save_audio(audio, str(dest_path), sr)
|
|
||||||
|
|
||||||
db_sample = DBProfileSample(
|
db_sample = DBProfileSample(
|
||||||
id=sample_id,
|
id=sample_id,
|
||||||
|
|||||||
+24
-6
@@ -217,22 +217,40 @@ def validate_reference_audio(
|
|||||||
Returns:
|
Returns:
|
||||||
Tuple of (is_valid, error_message)
|
Tuple of (is_valid, error_message)
|
||||||
"""
|
"""
|
||||||
|
result = validate_and_load_reference_audio(
|
||||||
|
audio_path, min_duration, max_duration, min_rms
|
||||||
|
)
|
||||||
|
return (result[0], result[1])
|
||||||
|
|
||||||
|
|
||||||
|
def validate_and_load_reference_audio(
|
||||||
|
audio_path: str,
|
||||||
|
min_duration: float = 2.0,
|
||||||
|
max_duration: float = 30.0,
|
||||||
|
min_rms: float = 0.01,
|
||||||
|
) -> Tuple[bool, Optional[str], Optional[np.ndarray], Optional[int]]:
|
||||||
|
"""
|
||||||
|
Validate and load reference audio in a single pass.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Tuple of (is_valid, error_message, audio_array, sample_rate)
|
||||||
|
"""
|
||||||
try:
|
try:
|
||||||
audio, sr = load_audio(audio_path)
|
audio, sr = load_audio(audio_path)
|
||||||
duration = len(audio) / sr
|
duration = len(audio) / sr
|
||||||
|
|
||||||
if duration < min_duration:
|
if duration < min_duration:
|
||||||
return False, f"Audio too short (minimum {min_duration} seconds)"
|
return False, f"Audio too short (minimum {min_duration} seconds)", None, None
|
||||||
if duration > max_duration:
|
if duration > max_duration:
|
||||||
return False, f"Audio too long (maximum {max_duration} seconds)"
|
return False, f"Audio too long (maximum {max_duration} seconds)", None, None
|
||||||
|
|
||||||
rms = np.sqrt(np.mean(audio**2))
|
rms = np.sqrt(np.mean(audio**2))
|
||||||
if rms < min_rms:
|
if rms < min_rms:
|
||||||
return False, "Audio is too quiet or silent"
|
return False, "Audio is too quiet or silent", None, None
|
||||||
|
|
||||||
if np.abs(audio).max() > 0.99:
|
if np.abs(audio).max() > 0.99:
|
||||||
return False, "Audio is clipping (reduce input gain)"
|
return False, "Audio is clipping (reduce input gain)", None, None
|
||||||
|
|
||||||
return True, None
|
return True, None, audio, sr
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
return False, f"Error validating audio: {str(e)}"
|
return False, f"Error validating audio: {str(e)}", None, None
|
||||||
|
|||||||
@@ -0,0 +1,95 @@
|
|||||||
|
"""
|
||||||
|
Minimal shim for descript-audio-codec (DAC).
|
||||||
|
|
||||||
|
TADA only imports Snake1d from dac.nn.layers and dac.model.dac.
|
||||||
|
The real DAC package pulls in descript-audiotools which depends on
|
||||||
|
onnx, tensorboard, protobuf, matplotlib, pystoi, etc. — none of
|
||||||
|
which are needed for TADA's runtime use of Snake1d.
|
||||||
|
|
||||||
|
This shim provides the exact Snake1d implementation (MIT-licensed,
|
||||||
|
from https://github.com/descriptinc/descript-audio-codec) so we can
|
||||||
|
avoid the entire audiotools dependency chain.
|
||||||
|
|
||||||
|
If the real DAC package is installed, this module is never used —
|
||||||
|
Python's import system will find the site-packages version first.
|
||||||
|
Install this shim only when descript-audio-codec is NOT installed.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import sys
|
||||||
|
import types
|
||||||
|
|
||||||
|
import torch
|
||||||
|
import torch.nn as nn
|
||||||
|
|
||||||
|
|
||||||
|
# ── Snake activation (from dac/nn/layers.py) ────────────────────────
|
||||||
|
|
||||||
|
# NOTE: The original DAC code uses @torch.jit.script here for a 1.4x
|
||||||
|
# speedup. We omit it because TorchScript calls inspect.getsource()
|
||||||
|
# which fails inside a PyInstaller frozen binary (no .py source files).
|
||||||
|
def snake(x: torch.Tensor, alpha: torch.Tensor) -> torch.Tensor:
|
||||||
|
shape = x.shape
|
||||||
|
x = x.reshape(shape[0], shape[1], -1)
|
||||||
|
x = x + (alpha + 1e-9).reciprocal() * torch.sin(alpha * x).pow(2)
|
||||||
|
x = x.reshape(shape)
|
||||||
|
return x
|
||||||
|
|
||||||
|
|
||||||
|
class Snake1d(nn.Module):
|
||||||
|
def __init__(self, channels: int):
|
||||||
|
super().__init__()
|
||||||
|
self.alpha = nn.Parameter(torch.ones(1, channels, 1))
|
||||||
|
|
||||||
|
def forward(self, x: torch.Tensor) -> torch.Tensor:
|
||||||
|
return snake(x, self.alpha)
|
||||||
|
|
||||||
|
|
||||||
|
# ── Register as dac.nn.layers and dac.model.dac ─────────────────────
|
||||||
|
|
||||||
|
def install_dac_shim() -> None:
|
||||||
|
"""Register fake dac package modules in sys.modules.
|
||||||
|
|
||||||
|
Only installs the shim if 'dac' is not already importable
|
||||||
|
(i.e. the real descript-audio-codec is not installed).
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
import dac # noqa: F401 — real package exists, do nothing
|
||||||
|
return
|
||||||
|
except ImportError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
# Create the module tree: dac -> dac.nn -> dac.nn.layers
|
||||||
|
# -> dac.model -> dac.model.dac
|
||||||
|
dac_pkg = types.ModuleType("dac")
|
||||||
|
dac_pkg.__path__ = [] # make it a package
|
||||||
|
dac_pkg.__package__ = "dac"
|
||||||
|
|
||||||
|
dac_nn = types.ModuleType("dac.nn")
|
||||||
|
dac_nn.__path__ = []
|
||||||
|
dac_nn.__package__ = "dac.nn"
|
||||||
|
|
||||||
|
dac_nn_layers = types.ModuleType("dac.nn.layers")
|
||||||
|
dac_nn_layers.__package__ = "dac.nn"
|
||||||
|
dac_nn_layers.Snake1d = Snake1d
|
||||||
|
dac_nn_layers.snake = snake
|
||||||
|
|
||||||
|
dac_model = types.ModuleType("dac.model")
|
||||||
|
dac_model.__path__ = []
|
||||||
|
dac_model.__package__ = "dac.model"
|
||||||
|
|
||||||
|
dac_model_dac = types.ModuleType("dac.model.dac")
|
||||||
|
dac_model_dac.__package__ = "dac.model"
|
||||||
|
dac_model_dac.Snake1d = Snake1d
|
||||||
|
|
||||||
|
# Wire up submodules
|
||||||
|
dac_pkg.nn = dac_nn
|
||||||
|
dac_pkg.model = dac_model
|
||||||
|
dac_nn.layers = dac_nn_layers
|
||||||
|
dac_model.dac = dac_model_dac
|
||||||
|
|
||||||
|
# Register in sys.modules
|
||||||
|
sys.modules["dac"] = dac_pkg
|
||||||
|
sys.modules["dac.nn"] = dac_nn
|
||||||
|
sys.modules["dac.nn.layers"] = dac_nn_layers
|
||||||
|
sys.modules["dac.model"] = dac_model
|
||||||
|
sys.modules["dac.model.dac"] = dac_model_dac
|
||||||
@@ -1,17 +1,64 @@
|
|||||||
"""Monkey-patch huggingface_hub to force offline mode with cached models.
|
"""Monkey-patch huggingface_hub to force offline mode with cached models.
|
||||||
|
|
||||||
Prevents mlx_audio from making network requests when models are already
|
Prevents mlx_audio / transformers from making network requests when models
|
||||||
downloaded. Must be imported BEFORE mlx_audio.
|
are already downloaded. Must be imported BEFORE mlx_audio.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
|
from contextlib import contextmanager
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Optional, Union
|
from typing import Optional, Union
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def force_offline_if_cached(is_cached: bool, model_label: str = ""):
|
||||||
|
"""Context manager that sets ``HF_HUB_OFFLINE=1`` while loading a cached model.
|
||||||
|
|
||||||
|
If *is_cached* is ``False`` the block runs normally (network allowed).
|
||||||
|
If the offline load raises an error containing "offline" we automatically
|
||||||
|
retry with network access so a partially-cached model still works.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
is_cached: Whether the model weights are already on disk.
|
||||||
|
model_label: Human-readable name used in log messages.
|
||||||
|
"""
|
||||||
|
if not is_cached:
|
||||||
|
yield
|
||||||
|
return
|
||||||
|
|
||||||
|
original_value = os.environ.get("HF_HUB_OFFLINE")
|
||||||
|
os.environ["HF_HUB_OFFLINE"] = "1"
|
||||||
|
logger.info(
|
||||||
|
"[offline-guard] %s is cached — forcing HF_HUB_OFFLINE=1",
|
||||||
|
model_label or "model",
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
except Exception as exc:
|
||||||
|
if "offline" in str(exc).lower():
|
||||||
|
logger.warning(
|
||||||
|
"[offline-guard] Offline load failed for %s, retrying with network: %s",
|
||||||
|
model_label or "model",
|
||||||
|
exc,
|
||||||
|
)
|
||||||
|
# Restore original env and retry — caller must wrap the load
|
||||||
|
# inside force_offline_if_cached so retrying here isn't possible.
|
||||||
|
# Instead, propagate a flag via the exception so the caller can
|
||||||
|
# decide. For simplicity we just let it fall through to the
|
||||||
|
# finally block and re-raise.
|
||||||
|
raise
|
||||||
|
raise
|
||||||
|
finally:
|
||||||
|
if original_value is not None:
|
||||||
|
os.environ["HF_HUB_OFFLINE"] = original_value
|
||||||
|
else:
|
||||||
|
os.environ.pop("HF_HUB_OFFLINE", None)
|
||||||
|
|
||||||
|
|
||||||
def patch_huggingface_hub_offline():
|
def patch_huggingface_hub_offline():
|
||||||
"""Monkey-patch huggingface_hub to force offline mode."""
|
"""Monkey-patch huggingface_hub to force offline mode."""
|
||||||
try:
|
try:
|
||||||
|
|||||||
@@ -246,6 +246,8 @@ class ProgressManager:
|
|||||||
# Send heartbeat
|
# Send heartbeat
|
||||||
yield ": heartbeat\n\n"
|
yield ": heartbeat\n\n"
|
||||||
continue
|
continue
|
||||||
|
except (BrokenPipeError, ConnectionResetError, asyncio.CancelledError):
|
||||||
|
logger.debug(f"SSE client disconnected from {model_name}")
|
||||||
finally:
|
finally:
|
||||||
# Remove from listeners
|
# Remove from listeners
|
||||||
if model_name in self._listeners:
|
if model_name in self._listeners:
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ from PyInstaller.utils.hooks import copy_metadata
|
|||||||
|
|
||||||
datas = []
|
datas = []
|
||||||
binaries = []
|
binaries = []
|
||||||
hiddenimports = ['backend', 'backend.main', 'backend.config', 'backend.database', 'backend.models', 'backend.profiles', 'backend.history', 'backend.tts', 'backend.transcribe', 'backend.platform_detect', 'backend.backends', 'backend.backends.pytorch_backend', 'backend.utils.audio', 'backend.utils.cache', 'backend.utils.progress', 'backend.utils.hf_progress', 'backend.utils.validation', 'backend.cuda_download', 'backend.effects', 'backend.utils.effects', 'backend.versions', 'pedalboard', 'chatterbox', 'chatterbox.tts_turbo', 'chatterbox.mtl_tts', 'backend.backends.chatterbox_backend', 'backend.backends.chatterbox_turbo_backend', 'backend.backends.luxtts_backend', 'zipvoice', 'zipvoice.luxvoice', 'torch', 'transformers', 'fastapi', 'uvicorn', 'sqlalchemy', 'librosa', 'soundfile', 'qwen_tts', 'qwen_tts.inference', 'qwen_tts.inference.qwen3_tts_model', 'qwen_tts.inference.qwen3_tts_tokenizer', 'qwen_tts.core', 'qwen_tts.cli', 'requests', 'pkg_resources.extern', 'backend.backends.mlx_backend', 'mlx', 'mlx.core', 'mlx.nn', 'mlx_audio', 'mlx_audio.tts', 'mlx_audio.stt']
|
hiddenimports = ['backend', 'backend.main', 'backend.config', 'backend.database', 'backend.models', 'backend.services.profiles', 'backend.services.history', 'backend.services.tts', 'backend.services.transcribe', 'backend.utils.platform_detect', 'backend.backends', 'backend.backends.pytorch_backend', 'backend.utils.audio', 'backend.utils.cache', 'backend.utils.progress', 'backend.utils.hf_progress', 'backend.services.cuda', 'backend.services.effects', 'backend.utils.effects', 'backend.services.versions', 'pedalboard', 'chatterbox', 'chatterbox.tts_turbo', 'chatterbox.mtl_tts', 'backend.backends.chatterbox_backend', 'backend.backends.chatterbox_turbo_backend', 'backend.backends.luxtts_backend', 'zipvoice', 'zipvoice.luxvoice', 'torch', 'transformers', 'fastapi', 'uvicorn', 'sqlalchemy', 'soundfile', 'qwen_tts', 'qwen_tts.inference', 'qwen_tts.inference.qwen3_tts_model', 'qwen_tts.inference.qwen3_tts_tokenizer', 'qwen_tts.core', 'qwen_tts.cli', 'requests', 'pkg_resources.extern', 'backend.backends.mlx_backend', 'mlx', 'mlx.core', 'mlx.nn', 'mlx_audio', 'mlx_audio.tts', 'mlx_audio.stt']
|
||||||
datas += collect_data_files('qwen_tts')
|
datas += collect_data_files('qwen_tts')
|
||||||
datas += copy_metadata('qwen-tts')
|
datas += copy_metadata('qwen-tts')
|
||||||
datas += copy_metadata('requests')
|
datas += copy_metadata('requests')
|
||||||
@@ -23,6 +23,16 @@ tmp_ret = collect_all('zipvoice')
|
|||||||
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
|
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
|
||||||
tmp_ret = collect_all('linacodec')
|
tmp_ret = collect_all('linacodec')
|
||||||
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
|
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
|
||||||
|
tmp_ret = collect_all('lazy_loader')
|
||||||
|
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
|
||||||
|
tmp_ret = collect_all('librosa')
|
||||||
|
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
|
||||||
|
tmp_ret = collect_all('inflect')
|
||||||
|
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
|
||||||
|
tmp_ret = collect_all('perth')
|
||||||
|
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
|
||||||
|
tmp_ret = collect_all('piper_phonemize')
|
||||||
|
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
|
||||||
tmp_ret = collect_all('mlx')
|
tmp_ret = collect_all('mlx')
|
||||||
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
|
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
|
||||||
tmp_ret = collect_all('mlx_audio')
|
tmp_ret = collect_all('mlx_audio')
|
||||||
|
|||||||
+1
-5
@@ -6,11 +6,7 @@ This is a Next.js application generated with
|
|||||||
Run development server:
|
Run development server:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
npm run dev
|
bun run dev
|
||||||
# or
|
|
||||||
pnpm dev
|
|
||||||
# or
|
|
||||||
yarn dev
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Open http://localhost:3000 with your browser to see the result.
|
Open http://localhost:3000 with your browser to see the result.
|
||||||
|
|||||||
@@ -1,6 +0,0 @@
|
|||||||
import { HomeLayout } from 'fumadocs-ui/layouts/home';
|
|
||||||
import { baseOptions } from '@/lib/layout.shared';
|
|
||||||
|
|
||||||
export default function Layout({ children }: LayoutProps<'/'>) {
|
|
||||||
return <HomeLayout {...baseOptions()}>{children}</HomeLayout>;
|
|
||||||
}
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
import { redirect } from 'next/navigation';
|
|
||||||
|
|
||||||
export default function HomePage() {
|
|
||||||
redirect('/docs');
|
|
||||||
}
|
|
||||||
@@ -1,8 +1,8 @@
|
|||||||
import { source } from '@/lib/source';
|
|
||||||
import { DocsLayout } from 'fumadocs-ui/layouts/docs';
|
import { DocsLayout } from 'fumadocs-ui/layouts/docs';
|
||||||
import { baseOptions } from '@/lib/layout.shared';
|
import { baseOptions } from '@/lib/layout.shared';
|
||||||
|
import { source } from '@/lib/source';
|
||||||
|
|
||||||
export default function Layout({ children }: LayoutProps<'/docs'>) {
|
export default function Layout({ children }: LayoutProps<'/[[...slug]]'>) {
|
||||||
return (
|
return (
|
||||||
<DocsLayout tree={source.pageTree} {...baseOptions()}>
|
<DocsLayout tree={source.pageTree} {...baseOptions()}>
|
||||||
{children}
|
{children}
|
||||||
@@ -7,7 +7,7 @@ import { APIPage } from '@/components/api-page';
|
|||||||
import { getPageImage, source } from '@/lib/source';
|
import { getPageImage, source } from '@/lib/source';
|
||||||
import { getMDXComponents } from '@/mdx-components';
|
import { getMDXComponents } from '@/mdx-components';
|
||||||
|
|
||||||
export default async function Page(props: PageProps<'/docs/[[...slug]]'>) {
|
export default async function Page(props: PageProps<'/[[...slug]]'>) {
|
||||||
const params = await props.params;
|
const params = await props.params;
|
||||||
const page = source.getPage(params.slug);
|
const page = source.getPage(params.slug);
|
||||||
if (!page) notFound();
|
if (!page) notFound();
|
||||||
@@ -59,7 +59,7 @@ export async function generateStaticParams() {
|
|||||||
return source.generateParams();
|
return source.generateParams();
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function generateMetadata(props: PageProps<'/docs/[[...slug]]'>): Promise<Metadata> {
|
export async function generateMetadata(props: PageProps<'/[[...slug]]'>): Promise<Metadata> {
|
||||||
const params = await props.params;
|
const params = await props.params;
|
||||||
const page = source.getPage(params.slug);
|
const page = source.getPage(params.slug);
|
||||||
if (!page) notFound();
|
if (!page) notFound();
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
{
|
||||||
|
"title": "General",
|
||||||
|
"pages": ["root__get", "health_health_get"]
|
||||||
|
}
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
{
|
||||||
|
"title": "Generation",
|
||||||
|
"pages": [
|
||||||
|
"generate_speech_generate_post",
|
||||||
|
"transcribe_audio_transcribe_post",
|
||||||
|
"get_audio_audio__generation_id__get"
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
{
|
||||||
|
"title": "History",
|
||||||
|
"pages": [
|
||||||
|
"list_history_history_get",
|
||||||
|
"get_generation_history__generation_id__get",
|
||||||
|
"delete_generation_history__generation_id__delete",
|
||||||
|
"get_stats_history_stats_get"
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
{
|
{
|
||||||
"title": "API Reference",
|
"title": "API Reference",
|
||||||
"defaultOpen": true,
|
"defaultOpen": true,
|
||||||
"pages": ["unknown"]
|
"pages": ["general", "profiles", "generation", "history", "models"]
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,10 @@
|
|||||||
|
{
|
||||||
|
"title": "Models",
|
||||||
|
"pages": [
|
||||||
|
"get_model_status_models_status_get",
|
||||||
|
"load_model_models_load_post",
|
||||||
|
"unload_model_models_unload_post",
|
||||||
|
"trigger_model_download_models_download_post",
|
||||||
|
"get_model_progress_models_progress__model_name__get"
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"title": "Profiles",
|
||||||
|
"pages": [
|
||||||
|
"list_profiles_profiles_get",
|
||||||
|
"create_profile_profiles_post",
|
||||||
|
"get_profile_profiles__profile_id__get",
|
||||||
|
"update_profile_profiles__profile_id__put",
|
||||||
|
"delete_profile_profiles__profile_id__delete",
|
||||||
|
"get_profile_samples_profiles__profile_id__samples_get",
|
||||||
|
"add_profile_sample_profiles__profile_id__samples_post",
|
||||||
|
"delete_profile_sample_profiles_samples__sample_id__delete"
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -1,27 +0,0 @@
|
|||||||
{
|
|
||||||
"title": "Endpoints",
|
|
||||||
"pages": [
|
|
||||||
"root__get",
|
|
||||||
"health_health_get",
|
|
||||||
"list_profiles_profiles_get",
|
|
||||||
"create_profile_profiles_post",
|
|
||||||
"get_profile_profiles__profile_id__get",
|
|
||||||
"update_profile_profiles__profile_id__put",
|
|
||||||
"delete_profile_profiles__profile_id__delete",
|
|
||||||
"add_profile_sample_profiles__profile_id__samples_post",
|
|
||||||
"get_profile_samples_profiles__profile_id__samples_get",
|
|
||||||
"delete_profile_sample_profiles_samples__sample_id__delete",
|
|
||||||
"generate_speech_generate_post",
|
|
||||||
"list_history_history_get",
|
|
||||||
"get_generation_history__generation_id__get",
|
|
||||||
"delete_generation_history__generation_id__delete",
|
|
||||||
"get_stats_history_stats_get",
|
|
||||||
"transcribe_audio_transcribe_post",
|
|
||||||
"get_audio_audio__generation_id__get",
|
|
||||||
"load_model_models_load_post",
|
|
||||||
"unload_model_models_unload_post",
|
|
||||||
"get_model_progress_models_progress__model_name__get",
|
|
||||||
"get_model_status_models_status_get",
|
|
||||||
"trigger_model_download_models_download_post"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
@@ -159,12 +159,14 @@ Tauri looks for `voicebox-server-${PLATFORM}` in `src-tauri/binaries/` and bundl
|
|||||||
|
|
||||||
The `build-cuda-windows` job runs separately:
|
The `build-cuda-windows` job runs separately:
|
||||||
|
|
||||||
1. Install PyTorch with CUDA 12.1
|
1. Install PyTorch with CUDA 12.8
|
||||||
2. Build with `build_binary.py --cuda`
|
2. Build with `build_binary.py --cuda` (produces `--onedir` output)
|
||||||
3. Split binary with `scripts/split_binary.py`
|
3. Package with `scripts/package_cuda.py` into two archives:
|
||||||
4. Upload parts as release artifacts
|
- `voicebox-server-cuda.tar.gz` — server core (~945 MB)
|
||||||
|
- `cuda-libs-cu128-v1.tar.gz` — NVIDIA runtime libraries (~1.7 GB, cached independently)
|
||||||
|
4. Upload archives as release artifacts
|
||||||
|
|
||||||
This binary is downloaded on-demand by users who enable CUDA in settings.
|
This binary is downloaded on-demand by users who enable CUDA in settings. The CUDA libs archive is only re-downloaded when the CUDA toolkit version changes, not on every app update.
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
|
|||||||
@@ -3,8 +3,12 @@ title: "TTS Engines"
|
|||||||
description: "How to add new text-to-speech engines to Voicebox"
|
description: "How to add new text-to-speech engines to Voicebox"
|
||||||
---
|
---
|
||||||
|
|
||||||
|
> **For humans:** This doc is optimized for AI agents to implement new TTS engines autonomously. It's structured as a phased workflow with explicit gates and a checklist so an agent can do the full integration — dependency research, backend, frontend, bundling — and hand you a draft release or prod build to test locally. It's also a useful reference if you're doing it yourself.
|
||||||
|
|
||||||
Adding an engine touches ~10 files across 4 layers. The backend protocol work is straightforward — the real time sink is dependency hell, upstream library bugs, and PyInstaller bundling.
|
Adding an engine touches ~10 files across 4 layers. The backend protocol work is straightforward — the real time sink is dependency hell, upstream library bugs, and PyInstaller bundling.
|
||||||
|
|
||||||
|
**Do not start writing code until you complete Phase 0.** The v0.2.3 release was three patch releases of PyInstaller fixes because dependency research was skipped. Every issue — `inspect.getsource()` failures, missing native data files, metadata lookups, dtype mismatches — was discoverable by reading the model library's source code before integration began.
|
||||||
|
|
||||||
## Architecture Overview
|
## Architecture Overview
|
||||||
|
|
||||||
The backend is split into layers:
|
The backend is split into layers:
|
||||||
@@ -18,6 +22,133 @@ The backend is split into layers:
|
|||||||
|
|
||||||
New engines only need to touch `backends/` and `models.py` on the backend side — the route and service layers use a model config registry that handles dispatch automatically.
|
New engines only need to touch `backends/` and `models.py` on the backend side — the route and service layers use a model config registry that handles dispatch automatically.
|
||||||
|
|
||||||
|
## Phase 0: Dependency Research
|
||||||
|
|
||||||
|
**This phase is mandatory.** Clone the model library and its key dependencies into a temporary directory and inspect them before writing any integration code. The goal is to produce a dependency audit that identifies every PyInstaller-incompatible pattern, every native data file, and every upstream bug you'll need to work around.
|
||||||
|
|
||||||
|
### 0.1 Clone and Inspect the Model Library
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Create a throwaway workspace
|
||||||
|
mkdir /tmp/engine-research && cd /tmp/engine-research
|
||||||
|
|
||||||
|
# Clone the model library
|
||||||
|
git clone https://github.com/org/model-library.git
|
||||||
|
cd model-library
|
||||||
|
```
|
||||||
|
|
||||||
|
**Read these files first, in order:**
|
||||||
|
|
||||||
|
1. **`setup.py` / `setup.cfg` / `pyproject.toml`** — Check pinned dependency versions. If the library pins `torch==2.6.0` or `numpy<1.26`, you'll need `--no-deps` installation and manual sub-dependency listing (this is what happened with `chatterbox-tts`).
|
||||||
|
|
||||||
|
2. **`__init__.py` and the main model class** — Trace the import chain. Look for:
|
||||||
|
- `from_pretrained()` — does it call `huggingface_hub` internally? Does it pass `token=True` (which crashes without a stored HF token)?
|
||||||
|
- `from_local()` — does it exist? You may need manual `snapshot_download()` + `from_local()` to bypass download bugs.
|
||||||
|
- Device handling — does it default to CUDA? Does it support MPS? Many libraries crash on MPS with unsupported operators.
|
||||||
|
|
||||||
|
3. **All `import` statements** — Recursively trace what the library imports. You're looking for:
|
||||||
|
- `inspect.getsource()` anywhere in the chain (search all `.py` files)
|
||||||
|
- `typeguard` / `@typechecked` decorators (these call `inspect.getsource()` at import time)
|
||||||
|
- `importlib.metadata.version()` or `pkg_resources.get_distribution()` (need `--copy-metadata`)
|
||||||
|
- `lazy_loader` (needs `--collect-all` to bundle `.pyi` stubs)
|
||||||
|
|
||||||
|
### 0.2 Scan for PyInstaller-Incompatible Patterns
|
||||||
|
|
||||||
|
Run these searches against the cloned library **and** its transitive dependencies:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# inspect.getsource — will crash in frozen binary without --collect-all
|
||||||
|
grep -r "inspect.getsource\|getsource(" .
|
||||||
|
|
||||||
|
# typeguard / @typechecked — calls inspect.getsource at import time
|
||||||
|
grep -r "@typechecked\|from typeguard" .
|
||||||
|
|
||||||
|
# importlib.metadata — needs --copy-metadata
|
||||||
|
grep -r "importlib.metadata\|pkg_resources.get_distribution\|pkg_resources.require" .
|
||||||
|
|
||||||
|
# Data files loaded at runtime — need --collect-all or --collect-data
|
||||||
|
grep -r "Path(__file__).parent\|os.path.dirname(__file__)\|resources_path\|pkg_resources.resource_filename" .
|
||||||
|
|
||||||
|
# Native library paths — may need env var override in frozen builds
|
||||||
|
grep -r "/usr/share\|/usr/lib\|/usr/local\|espeak\|phonemize" .
|
||||||
|
|
||||||
|
# torch.load without map_location — will crash on CPU-only builds
|
||||||
|
grep -r "torch.load(" . | grep -v "map_location"
|
||||||
|
|
||||||
|
# HuggingFace token bugs
|
||||||
|
grep -r 'token=True\|token=os.getenv' .
|
||||||
|
|
||||||
|
# Float64/Float32 assumptions — librosa returns float64, many models assume float32
|
||||||
|
grep -r "torch.from_numpy\|\.double()\|float64" .
|
||||||
|
|
||||||
|
# @torch.jit.script — calls inspect.getsource(), crashes in frozen builds
|
||||||
|
grep -r "@torch.jit.script\|torch.jit.script" .
|
||||||
|
|
||||||
|
# torchaudio.load — requires torchcodec in torchaudio 2.10+, use soundfile.read() instead
|
||||||
|
grep -r "torchaudio.load\|torchaudio.save" .
|
||||||
|
|
||||||
|
# Gated HuggingFace repos — models that hardcode gated repos as tokenizer/config sources
|
||||||
|
grep -r "from_pretrained\|tokenizer_name\|AutoTokenizer" . | grep -i "llama\|meta-llama\|gated"
|
||||||
|
```
|
||||||
|
|
||||||
|
### 0.3 Install and Trace in a Throwaway Venv
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Create isolated venv
|
||||||
|
python -m venv /tmp/engine-venv
|
||||||
|
source /tmp/engine-venv/bin/activate
|
||||||
|
|
||||||
|
# Install the package (try normally first)
|
||||||
|
pip install model-package
|
||||||
|
|
||||||
|
# Check if it conflicts with our stack
|
||||||
|
pip install model-package torch==2.10 transformers==4.57.3 numpy>=1.26
|
||||||
|
# If this fails, you need --no-deps:
|
||||||
|
pip install --no-deps model-package
|
||||||
|
|
||||||
|
# Get the full dependency tree
|
||||||
|
pip show model-package # Check Requires: field
|
||||||
|
pip show -f model-package # List all installed files (look for data files)
|
||||||
|
|
||||||
|
# Check for non-PyPI dependencies
|
||||||
|
pip install model-package 2>&1 | grep -i "no matching distribution"
|
||||||
|
```
|
||||||
|
|
||||||
|
### 0.4 Test Model Loading on CPU
|
||||||
|
|
||||||
|
Before writing any integration code, verify the model works on CPU in a plain Python script:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import torch
|
||||||
|
# Force CPU to catch map_location bugs early
|
||||||
|
model = ModelClass.from_pretrained("org/model", device="cpu")
|
||||||
|
|
||||||
|
# Test with a float32 audio array (not float64)
|
||||||
|
import numpy as np
|
||||||
|
audio = np.random.randn(16000).astype(np.float32)
|
||||||
|
output = model.generate("Hello world", audio)
|
||||||
|
print(f"Output shape: {output.shape}, dtype: {output.dtype}, sample rate: {model.sample_rate}")
|
||||||
|
```
|
||||||
|
|
||||||
|
If this crashes, you've found a bug you'll need to monkey-patch. Common ones:
|
||||||
|
- `RuntimeError: expected scalar type Float but found Double` → needs float32 cast
|
||||||
|
- `RuntimeError: map_location` → needs `torch.load` patch
|
||||||
|
- `RuntimeError: Unsupported operator aten::...` → needs MPS skip
|
||||||
|
|
||||||
|
### 0.5 Produce a Dependency Audit
|
||||||
|
|
||||||
|
Before proceeding to Phase 1, write down:
|
||||||
|
|
||||||
|
1. **PyPI vs non-PyPI deps** — which packages need `--find-links`, `git+https://`, or `--no-deps`?
|
||||||
|
2. **PyInstaller directives needed** — which packages need `--collect-all`, `--copy-metadata`, `--hidden-import`?
|
||||||
|
3. **Runtime data files** — which packages ship data files (YAML, pretrained weights, phoneme tables, shader libraries) that must be bundled?
|
||||||
|
4. **Native library paths** — which packages look for data at system paths that won't exist in a frozen binary?
|
||||||
|
5. **Monkey-patches needed** — `torch.load` map_location, float64→float32 casts, MPS skip, HF token bypass, etc.
|
||||||
|
6. **Sample rate** — what does the engine output? (24kHz, 44.1kHz, 48kHz)
|
||||||
|
7. **Model download method** — `from_pretrained()` with library-managed download, or manual `snapshot_download()` + `from_local()`?
|
||||||
|
|
||||||
|
This audit becomes your implementation plan for Phases 1, 4, and 5.
|
||||||
|
|
||||||
## Phase 1: Backend Implementation
|
## Phase 1: Backend Implementation
|
||||||
|
|
||||||
### 1.1 Create the Backend File
|
### 1.1 Create the Backend File
|
||||||
@@ -148,6 +279,8 @@ In `app/src/lib/hooks/useGenerationForm.ts`:
|
|||||||
- Add engine-to-model-name mapping
|
- Add engine-to-model-name mapping
|
||||||
- Update payload construction for engine-specific fields
|
- Update payload construction for engine-specific fields
|
||||||
|
|
||||||
|
**Watch out for model naming inconsistencies.** The HuggingFace repo name, the model size label, and the API model name don't always follow predictable patterns. For example, TADA's 3B model is named `tada-3b-ml` (not `tada-3b`), because it's a multilingual variant. Always check the actual repo names and build the frontend model name mapping from those, not from assumptions like `{engine}-{size}`.
|
||||||
|
|
||||||
### 3.5 Model Management
|
### 3.5 Model Management
|
||||||
|
|
||||||
In `app/src/components/ServerSettings/ModelManagement.tsx`:
|
In `app/src/components/ServerSettings/ModelManagement.tsx`:
|
||||||
@@ -155,54 +288,173 @@ In `app/src/components/ServerSettings/ModelManagement.tsx`:
|
|||||||
|
|
||||||
## Phase 4: Dependencies
|
## Phase 4: Dependencies
|
||||||
|
|
||||||
|
Use the dependency audit from Phase 0 to drive this phase. You should already know what packages are needed, which conflict, and which require special installation.
|
||||||
|
|
||||||
### 4.1 Python Dependencies
|
### 4.1 Python Dependencies
|
||||||
|
|
||||||
Add to `backend/requirements.txt`. Watch for:
|
Add to `backend/requirements.txt`. There are three installation patterns, depending on what Phase 0 revealed:
|
||||||
|
|
||||||
**Pinned dependency conflicts** — If the model package pins old versions, install with `--no-deps`:
|
**Normal PyPI packages:**
|
||||||
|
```
|
||||||
|
some-model-package>=1.0.0
|
||||||
|
```
|
||||||
|
|
||||||
|
**Pinned dependency conflicts (`--no-deps`)** — If the model package pins old versions of torch/numpy/transformers, install with `--no-deps` and list sub-dependencies manually. This is the pattern used for `chatterbox-tts`:
|
||||||
```bash
|
```bash
|
||||||
|
# In justfile / CI setup:
|
||||||
pip install --no-deps chatterbox-tts
|
pip install --no-deps chatterbox-tts
|
||||||
|
|
||||||
|
# In requirements.txt — list each actual sub-dependency:
|
||||||
|
conformer>=0.3.2
|
||||||
|
diffusers>=0.31.0
|
||||||
|
omegaconf>=2.3.0
|
||||||
|
resemble-perth>=0.0.2
|
||||||
|
s3tokenizer>=0.1.6
|
||||||
```
|
```
|
||||||
|
|
||||||
Then list sub-dependencies manually in `requirements.txt`.
|
To identify sub-deps: `pip show chatterbox-tts` → `Requires:` field, then cross-reference against existing `requirements.txt` to avoid duplicates.
|
||||||
|
|
||||||
**Non-PyPI packages:**
|
**Non-PyPI packages** — Some libraries only exist on GitHub or require custom indexes:
|
||||||
```
|
|
||||||
linacodec @ git+https://github.com/user/repo.git
|
|
||||||
```
|
```
|
||||||
|
# Git-only packages (no PyPI release)
|
||||||
|
linacodec @ git+https://github.com/ysharma3501/LinaCodec.git
|
||||||
|
Zipvoice @ git+https://github.com/ysharma3501/LuxTTS.git
|
||||||
|
|
||||||
**Custom package indexes:**
|
# Custom package indexes (C extensions with platform-specific wheels)
|
||||||
```
|
|
||||||
--find-links https://k2-fsa.github.io/icefall/piper_phonemize.html
|
--find-links https://k2-fsa.github.io/icefall/piper_phonemize.html
|
||||||
|
piper-phonemize>=1.2.0
|
||||||
```
|
```
|
||||||
|
|
||||||
### 4.2 Identifying Hidden Sub-Dependencies
|
### 4.2 Dependency Conflict Resolution
|
||||||
|
|
||||||
1. Install the package normally in a throwaway venv
|
Check for conflicts with the existing stack before adding anything:
|
||||||
2. Run `pip show <package>` to get its `Requires:` list
|
|
||||||
3. Cross-reference against existing requirements.txt
|
|
||||||
4. Test that the engine loads and generates
|
|
||||||
|
|
||||||
## Phase 5: PyInstaller Bundling
|
```bash
|
||||||
|
# Our current stack pins (approximate):
|
||||||
|
# Python 3.12+, torch>=2.10, transformers>=4.57, numpy>=1.26
|
||||||
|
|
||||||
This is where most of the pain lives. Common issues:
|
# Test compatibility
|
||||||
|
pip install model-package torch==2.10 transformers==4.57.3 numpy>=1.26
|
||||||
|
|
||||||
| Issue | Symptom | Fix |
|
# If it fails, check what the package pins:
|
||||||
|-------|---------|-----|
|
pip show model-package | grep Requires
|
||||||
| `inspect.getsource()` at import | "could not get source code" | `--collect-all <package>` |
|
# Look at setup.py/pyproject.toml for version constraints
|
||||||
| Data files (yaml, .pth.tar) | FileNotFoundError at runtime | `--collect-all <package>` |
|
```
|
||||||
| Native data paths (espeak-ng) | Library looks at `/usr/share/...` | Set env var in frozen builds |
|
|
||||||
| `importlib.metadata` lookups | "No package metadata found" | `--copy-metadata <package>` |
|
|
||||||
| Dynamic imports | ModuleNotFoundError | `--hidden-import <module>` |
|
|
||||||
|
|
||||||
### Testing Frozen Builds
|
**Known incompatible patterns in the wild:**
|
||||||
|
- `torch==2.6.0` — many older packages pin this
|
||||||
|
- `numpy<1.26` — conflicts with Python 3.12+
|
||||||
|
- `transformers==4.46.3` — many packages pin old transformers
|
||||||
|
- `onnxruntime` pinned versions — often conflict with torch
|
||||||
|
|
||||||
You can't skip this. Models that work in `python -m uvicorn` will break in the PyInstaller binary.
|
### 4.3 Update Installation Scripts
|
||||||
|
|
||||||
|
Dependencies must be added in multiple places:
|
||||||
|
|
||||||
|
| File | What to add |
|
||||||
|
|------|------------|
|
||||||
|
| `backend/requirements.txt` | Package and version constraint |
|
||||||
|
| `justfile` | `--no-deps` install line if needed (in `setup-python` and `setup-python-release` targets) |
|
||||||
|
| `.github/workflows/release.yml` | Same `--no-deps` line in CI build steps |
|
||||||
|
| `Dockerfile` | Same install commands for Docker builds |
|
||||||
|
|
||||||
|
## Phase 5: PyInstaller Bundling (`build_binary.py`)
|
||||||
|
|
||||||
|
This is where most of the pain lives. **The v0.2.3 release was entirely dedicated to fixing bundling issues** — every new engine that shipped in v0.2.1 (LuxTTS, Chatterbox, Chatterbox Turbo) worked in dev but failed in production builds. Don't skip this phase.
|
||||||
|
|
||||||
|
### 5.1 Register Your Engine in `build_binary.py`
|
||||||
|
|
||||||
|
Every new engine needs entries in `backend/build_binary.py`. This file drives PyInstaller and is the single most common source of "works in dev, breaks in prod" bugs. You need to decide which PyInstaller directives your engine's dependencies require:
|
||||||
|
|
||||||
|
| Directive | What It Does | When You Need It |
|
||||||
|
|-----------|-------------|-----------------|
|
||||||
|
| `--hidden-import <module>` | Includes a module PyInstaller can't detect via static analysis | Dynamic imports, lazy imports, plugin architectures |
|
||||||
|
| `--collect-all <package>` | Bundles source `.py` files, data files, AND native libraries | Packages that call `inspect.getsource()` at import time (e.g. `inflect` via `typeguard`'s `@typechecked`), or that ship pretrained model files (e.g. `perth` ships `.pth.tar` + `hparams.yaml`) |
|
||||||
|
| `--collect-data <package>` | Bundles only data files (not source or native libs) | Packages with YAML configs, vocab files, etc. |
|
||||||
|
| `--collect-submodules <package>` | Bundles all submodules | Packages with deep module trees that PyInstaller misses |
|
||||||
|
| `--copy-metadata <package>` | Copies `importlib.metadata` info | Packages that call `importlib.metadata.version()` or `pkg_resources.get_distribution()` at runtime. Already required for: `requests`, `transformers`, `huggingface-hub`, `tokenizers`, `safetensors`, `tqdm` |
|
||||||
|
|
||||||
|
**Example: adding hidden imports and collect-all for a new engine:**
|
||||||
|
|
||||||
|
```python
|
||||||
|
# In build_binary.py, inside the args list:
|
||||||
|
"--hidden-import",
|
||||||
|
"backend.backends.your_engine_backend",
|
||||||
|
"--hidden-import",
|
||||||
|
"your_engine_package",
|
||||||
|
"--hidden-import",
|
||||||
|
"your_engine_package.inference",
|
||||||
|
"--collect-all",
|
||||||
|
"some_dependency_that_uses_inspect_getsource",
|
||||||
|
"--copy-metadata",
|
||||||
|
"some_dependency_that_checks_its_own_version",
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.2 Lessons from v0.2.3 — Real Failures and Their Fixes
|
||||||
|
|
||||||
|
These are actual production failures from shipping new engines. Every one of these passed `python -m uvicorn` in dev:
|
||||||
|
|
||||||
|
| Engine | Failure | Root Cause | Fix |
|
||||||
|
|--------|---------|-----------|-----|
|
||||||
|
| LuxTTS | `"could not get source code"` on import | `inflect` uses `typeguard`'s `@typechecked` which calls `inspect.getsource()` — needs `.py` source files, not just bytecode | `--collect-all inflect` |
|
||||||
|
| LuxTTS | `espeak-ng-data` not found | `piper_phonemize` C library looks for data at `/usr/share/espeak-ng-data/` which doesn't exist in the bundle | `--collect-all piper_phonemize` + set `ESPEAK_DATA_PATH` env var at runtime (see 5.3) |
|
||||||
|
| LuxTTS | `inspect.getsource` error in Vocos codec | `linacodec` and `zipvoice` use source introspection | `--collect-all linacodec` + `--collect-all zipvoice` |
|
||||||
|
| Chatterbox | `FileNotFoundError` for watermark model | `perth` ships pretrained model files (`hparams.yaml`, `.pth.tar`) that PyInstaller doesn't bundle by default | `--collect-all perth` |
|
||||||
|
| All engines | `importlib.metadata` failures | Frozen binary doesn't include package metadata for `huggingface-hub`, `transformers`, etc. | `--copy-metadata` for each affected package |
|
||||||
|
| All engines | Download progress bars stuck at 0% | `huggingface_hub` silently disables tqdm progress bars based on logger level in frozen builds — our progress tracker never receives byte updates | Force-enable tqdm's internal counter in `HFProgressTracker` |
|
||||||
|
| TADA | `inspect.getsource` error in DAC's `Snake1d` | `@torch.jit.script` calls `inspect.getsource()` which fails without `.py` source files | Wrote a lightweight shim (`dac_shim.py`) reimplementing `Snake1d` without `@torch.jit.script`, registered fake `dac.*` modules in `sys.modules` |
|
||||||
|
| All engines | `NameError: name 'obj' is not defined` on macOS | Python 3.12.0 has a [CPython bug](https://github.com/pyinstaller/pyinstaller/issues/7992) that corrupts bytecode when PyInstaller rewrites code objects | Upgrade to Python 3.12.13+ |
|
||||||
|
| All engines | `resource_tracker` subprocess crash | `multiprocessing` in frozen binaries needs `freeze_support()` called before anything else | Added to `server.py` entry point |
|
||||||
|
|
||||||
|
### 5.3 Runtime Frozen-Build Handling (`server.py`)
|
||||||
|
|
||||||
|
Some fixes can't live in `build_binary.py` — they need runtime detection. The entry point `backend/server.py` handles these before any heavy imports:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# 1. freeze_support() — MUST be called before any multiprocessing use
|
||||||
|
import multiprocessing
|
||||||
|
multiprocessing.freeze_support()
|
||||||
|
|
||||||
|
# 2. Native data paths — redirect C libraries to bundled data
|
||||||
|
if getattr(sys, 'frozen', False):
|
||||||
|
_meipass = getattr(sys, '_MEIPASS', os.path.dirname(sys.executable))
|
||||||
|
_espeak_data = os.path.join(_meipass, 'piper_phonemize', 'espeak-ng-data')
|
||||||
|
if os.path.isdir(_espeak_data):
|
||||||
|
os.environ.setdefault('ESPEAK_DATA_PATH', _espeak_data)
|
||||||
|
|
||||||
|
# 3. stdout/stderr safety — PyInstaller --noconsole on Windows sets these to None
|
||||||
|
if not _is_writable(sys.stdout):
|
||||||
|
sys.stdout = open(os.devnull, 'w')
|
||||||
|
```
|
||||||
|
|
||||||
|
If your engine's dependencies include native libraries that look for data at system paths (like espeak-ng does), you'll need to add a similar `os.environ.setdefault()` block here.
|
||||||
|
|
||||||
|
### 5.4 CUDA vs CPU Build Branching
|
||||||
|
|
||||||
|
`build_binary.py` produces two different binaries:
|
||||||
|
|
||||||
|
- **`voicebox-server`** (CPU) — excludes all `nvidia.*` packages to avoid bundling ~3 GB of CUDA DLLs
|
||||||
|
- **`voicebox-server-cuda`** — includes `torch.cuda` and `torch.backends.cudnn`
|
||||||
|
|
||||||
|
On Windows, if the build environment has CUDA torch installed but you're building the CPU binary, the script temporarily swaps to CPU-only torch and restores CUDA torch afterward. This prevents PyInstaller from accidentally bundling CUDA libraries into the CPU build.
|
||||||
|
|
||||||
|
New engine imports go in the **common section** (not the CUDA or MLX conditional blocks) unless your engine has platform-specific dependencies.
|
||||||
|
|
||||||
|
### 5.5 MLX Conditional Inclusion
|
||||||
|
|
||||||
|
Apple Silicon builds conditionally include MLX hidden imports and `--collect-all mlx` / `--collect-all mlx_audio`. If your engine has an MLX-specific backend variant, add its imports inside the `if is_apple_silicon() and not cuda:` block.
|
||||||
|
|
||||||
|
### 5.6 Testing Frozen Builds
|
||||||
|
|
||||||
|
You can't skip this. Models that work in `python -m uvicorn` will break in the PyInstaller binary. The v0.2.3 release required **three patch releases** (v0.2.1 → v0.2.2 → v0.2.3) to get all engines working in production.
|
||||||
|
|
||||||
1. Build: `just build`
|
1. Build: `just build`
|
||||||
2. Run and try download + load + generate
|
2. Launch the binary directly (not via `python -m`)
|
||||||
3. Check stderr for the actual error
|
3. Test the **full chain**: download → load → generate → progress tracking
|
||||||
4. Fix, rebuild, repeat
|
4. Check stderr for the actual error (logs go to stderr for Tauri sidecar capture)
|
||||||
|
5. Fix, rebuild, repeat
|
||||||
|
|
||||||
|
**Common gotcha:** testing only generation with a pre-cached model from your dev install. Always test with a clean model cache to verify downloads work too.
|
||||||
|
|
||||||
## Phase 6: Common Upstream Workarounds
|
## Phase 6: Common Upstream Workarounds
|
||||||
|
|
||||||
@@ -240,6 +492,90 @@ def _get_device(self):
|
|||||||
return "cpu" # Skip MPS
|
return "cpu" # Skip MPS
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Gated HuggingFace repos as hardcoded config sources
|
||||||
|
|
||||||
|
Some models hardcode a gated HuggingFace repo as their tokenizer or config source (e.g., TADA hardcodes `"meta-llama/Llama-3.2-1B"` in both its `AlignerConfig` and `TadaConfig`). This silently fails without HF authentication.
|
||||||
|
|
||||||
|
**Fix:** Download from an ungated mirror and patch the config objects directly:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Download tokenizer from ungated mirror
|
||||||
|
UNGATED_TOKENIZER = "unsloth/Llama-3.2-1B"
|
||||||
|
tokenizer_path = snapshot_download(UNGATED_TOKENIZER, token=None)
|
||||||
|
|
||||||
|
# Patch the model config to use the local path instead of the gated repo
|
||||||
|
config = ModelConfig.from_pretrained(model_path)
|
||||||
|
config.tokenizer_name = tokenizer_path
|
||||||
|
model = ModelClass.from_pretrained(model_path, config=config)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Do NOT monkey-patch `AutoTokenizer.from_pretrained`** — it's a classmethod, and replacing it corrupts the descriptor, which breaks other engines that use different tokenizers (e.g., Qwen uses a Qwen tokenizer via `AutoTokenizer`). Always patch at the config level, not the class method level.
|
||||||
|
|
||||||
|
### `torchaudio.load()` requires `torchcodec` in 2.10+
|
||||||
|
|
||||||
|
As of `torchaudio>=2.10`, `torchaudio.load()` requires the `torchcodec` package for audio I/O. If your engine or backend code uses `torchaudio.load()`, replace it with `soundfile`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Before (breaks without torchcodec):
|
||||||
|
import torchaudio
|
||||||
|
waveform, sr = torchaudio.load("audio.wav")
|
||||||
|
|
||||||
|
# After:
|
||||||
|
import soundfile as sf
|
||||||
|
import torch
|
||||||
|
data, sr = sf.read("audio.wav", dtype="float32")
|
||||||
|
waveform = torch.from_numpy(data).unsqueeze(0)
|
||||||
|
```
|
||||||
|
|
||||||
|
Note: `torchaudio.functional.resample()` and other pure-PyTorch math functions work fine without `torchcodec` — only the I/O functions are affected.
|
||||||
|
|
||||||
|
### `@torch.jit.script` breaks in frozen builds
|
||||||
|
|
||||||
|
`torch.jit.script` calls `inspect.getsource()` to parse the decorated function's source code. In a PyInstaller binary, `.py` source files aren't available, so this crashes at import time.
|
||||||
|
|
||||||
|
**Fix:** Remove or avoid `@torch.jit.script` decorators. If the decorated function comes from an upstream dependency, write a shim that reimplements the function without the decorator (see "Toxic dependency chains" below).
|
||||||
|
|
||||||
|
### Toxic dependency chains — the shim pattern
|
||||||
|
|
||||||
|
Sometimes a model library depends on a package with a massive, hostile transitive dependency tree, but only uses a tiny piece of it. When the dependency chain is unbuildable or would pull in dozens of unwanted packages, the right move is to write a lightweight shim.
|
||||||
|
|
||||||
|
**Example:** TADA depends on `descript-audio-codec` (DAC), which pulls in `descript-audiotools` -> `onnx`, `tensorboard`, `protobuf`, `matplotlib`, `pystoi`, etc. The `onnx` package fails to build from source on macOS. But TADA only uses `Snake1d` from DAC — a 7-line PyTorch module.
|
||||||
|
|
||||||
|
**Solution:** Create a shim at `backend/utils/dac_shim.py` that registers fake modules in `sys.modules`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import sys
|
||||||
|
import types
|
||||||
|
import torch
|
||||||
|
from torch import nn
|
||||||
|
|
||||||
|
def snake(x, alpha):
|
||||||
|
"""Snake activation — reimplemented without @torch.jit.script."""
|
||||||
|
return x + (1.0 / (alpha + 1e-9)) * torch.sin(alpha * x).pow(2)
|
||||||
|
|
||||||
|
class Snake1d(nn.Module):
|
||||||
|
def __init__(self, channels):
|
||||||
|
super().__init__()
|
||||||
|
self.alpha = nn.Parameter(torch.ones(1, channels, 1))
|
||||||
|
def forward(self, x):
|
||||||
|
return snake(x, self.alpha)
|
||||||
|
|
||||||
|
# Register fake dac.* modules so "from dac.nn.layers import Snake1d" works
|
||||||
|
_nn = types.ModuleType("dac.nn")
|
||||||
|
_layers = types.ModuleType("dac.nn.layers")
|
||||||
|
_layers.Snake1d = Snake1d
|
||||||
|
_nn.layers = _layers
|
||||||
|
|
||||||
|
for name, mod in [("dac", types.ModuleType("dac")),
|
||||||
|
("dac.nn", _nn), ("dac.nn.layers", _layers)]:
|
||||||
|
sys.modules[name] = mod
|
||||||
|
```
|
||||||
|
|
||||||
|
**Key rules for shims:**
|
||||||
|
- Import the shim **before** importing the model library (so it finds the fake modules first)
|
||||||
|
- Do NOT use `@torch.jit.script` in the shim (see above)
|
||||||
|
- Only reimplement what the model actually uses — check the import chain carefully
|
||||||
|
|
||||||
## Upcoming Engines
|
## Upcoming Engines
|
||||||
|
|
||||||
Based on the current model landscape, these are candidates for future integration:
|
Based on the current model landscape, these are candidates for future integration:
|
||||||
@@ -250,8 +586,83 @@ Based on the current model landscape, these are candidates for future integratio
|
|||||||
| **Fish Speech** | 50+ | Medium | Word-level control via inline text | Ready |
|
| **Fish Speech** | 50+ | Medium | Word-level control via inline text | Ready |
|
||||||
| **Kokoro-82M** | English | 82M | CPU realtime, Apache 2.0 | Ready |
|
| **Kokoro-82M** | English | 82M | CPU realtime, Apache 2.0 | Ready |
|
||||||
| **XTTS-v2** | 17+ | Medium | Zero-shot cloning | Ready |
|
| **XTTS-v2** | 17+ | Medium | Zero-shot cloning | Ready |
|
||||||
| **HumeAI TADA** | EN (1B), Multi (3B) | Medium | 700s+ coherent audio, synced transcripts | Needs vetting |
|
|
||||||
| **MOSS-TTS** | Multilingual | Medium | Text-to-voice design, multi-speaker dialogue | Needs vetting |
|
| **MOSS-TTS** | Multilingual | Medium | Text-to-voice design, multi-speaker dialogue | Needs vetting |
|
||||||
| **Pocket TTS** | English | ~100M | CPU-first, >1× realtime | Needs vetting |
|
| **Pocket TTS** | English | ~100M | CPU-first, >1× realtime | Needs vetting |
|
||||||
|
|
||||||
The multi-engine architecture is now in place, making new model integration straightforward (~1 day for a well-documented model with a PyPI package).
|
## Implementation Checklist
|
||||||
|
|
||||||
|
Use this as a gate between phases. Do not proceed to the next phase until every item in the current phase is checked.
|
||||||
|
|
||||||
|
### Phase 0: Dependency Research
|
||||||
|
- [ ] Cloned model library source into a temp directory
|
||||||
|
- [ ] Read `setup.py` / `pyproject.toml` — noted pinned dependency versions
|
||||||
|
- [ ] Traced all imports from the model class through to leaf dependencies
|
||||||
|
- [ ] Searched for `inspect.getsource`, `@typechecked`, `typeguard` in the full dependency tree
|
||||||
|
- [ ] Searched for `importlib.metadata`, `pkg_resources.get_distribution` in the dependency tree
|
||||||
|
- [ ] Searched for `Path(__file__).parent`, `os.path.dirname(__file__)`, hardcoded system paths
|
||||||
|
- [ ] Searched for `torch.load` calls missing `map_location`
|
||||||
|
- [ ] Searched for `torch.from_numpy` without `.float()` cast
|
||||||
|
- [ ] Searched for `token=True` or `token=os.getenv("HF_TOKEN")` in HuggingFace calls
|
||||||
|
- [ ] Searched for `@torch.jit.script` / `torch.jit.script` (crashes in frozen builds)
|
||||||
|
- [ ] Searched for `torchaudio.load` / `torchaudio.save` (requires `torchcodec` in 2.10+)
|
||||||
|
- [ ] Searched for hardcoded gated HuggingFace repo names (e.g., `meta-llama/*`)
|
||||||
|
- [ ] Evaluated whether any dependency is used minimally enough to shim instead of install
|
||||||
|
- [ ] Tested model loading and generation on CPU in a throwaway venv
|
||||||
|
- [ ] Tested with a clean HuggingFace cache (no pre-downloaded models)
|
||||||
|
- [ ] Produced a written dependency audit documenting all findings
|
||||||
|
|
||||||
|
### Phase 1: Backend Implementation
|
||||||
|
- [ ] Created `backend/backends/<engine>_backend.py` implementing `TTSBackend` protocol
|
||||||
|
- [ ] Chose voice prompt pattern (pre-computed tensors vs deferred file paths)
|
||||||
|
- [ ] Implemented all monkey-patches identified in Phase 0
|
||||||
|
- [ ] Used `get_torch_device()` from `backends/base.py` for device selection
|
||||||
|
- [ ] Used `model_load_progress()` from `backends/base.py` for download/load tracking
|
||||||
|
- [ ] Tested: model downloads correctly
|
||||||
|
- [ ] Tested: model loads on CPU
|
||||||
|
- [ ] Tested: generation produces valid audio
|
||||||
|
- [ ] Tested: voice cloning from reference audio works
|
||||||
|
- [ ] Registered `ModelConfig` in `backends/__init__.py`
|
||||||
|
- [ ] Added to `TTS_ENGINES` dict
|
||||||
|
- [ ] Added factory branch in `get_tts_backend_for_engine()`
|
||||||
|
- [ ] Updated engine regex in `backend/models.py`
|
||||||
|
|
||||||
|
### Phase 2–3: Route, Service, and Frontend
|
||||||
|
- [ ] Confirmed zero changes needed in routes/services (or documented why custom behavior is needed)
|
||||||
|
- [ ] Added engine to TypeScript union type in `app/src/lib/api/types.ts`
|
||||||
|
- [ ] Added language map entry in `app/src/lib/constants/languages.ts`
|
||||||
|
- [ ] Added to `ENGINE_OPTIONS` and `ENGINE_DESCRIPTIONS` in `EngineModelSelector.tsx`
|
||||||
|
- [ ] Added to Zod schema and model-name mapping in `useGenerationForm.ts`
|
||||||
|
- [ ] Added description in `ModelManagement.tsx`
|
||||||
|
|
||||||
|
### Phase 4: Dependencies
|
||||||
|
- [ ] Added packages to `backend/requirements.txt`
|
||||||
|
- [ ] If `--no-deps` needed: listed sub-dependencies explicitly
|
||||||
|
- [ ] If git-only packages: added `@ git+https://...` entries
|
||||||
|
- [ ] If custom index needed: added `--find-links` line
|
||||||
|
- [ ] Updated `justfile` setup targets
|
||||||
|
- [ ] Updated `.github/workflows/release.yml` build steps
|
||||||
|
- [ ] Updated `Dockerfile` if applicable
|
||||||
|
- [ ] Verified `pip install` succeeds in a clean venv with existing requirements
|
||||||
|
|
||||||
|
### Phase 5: PyInstaller Bundling
|
||||||
|
- [ ] Added `--hidden-import` entries in `build_binary.py` for:
|
||||||
|
- [ ] `backend.backends.<engine>_backend`
|
||||||
|
- [ ] The model package and its key submodules
|
||||||
|
- [ ] Added `--collect-all` for any packages that:
|
||||||
|
- [ ] Use `inspect.getsource()` / `@typechecked`
|
||||||
|
- [ ] Ship pretrained model data files (`.pth.tar`, `.yaml`, etc.)
|
||||||
|
- [ ] Ship native data files (phoneme tables, shader libraries, etc.)
|
||||||
|
- [ ] Added `--copy-metadata` for any packages that use `importlib.metadata`
|
||||||
|
- [ ] If engine has native data paths: added `os.environ.setdefault()` in `server.py`
|
||||||
|
- [ ] Built frozen binary with `just build`
|
||||||
|
- [ ] Tested in frozen binary with **clean model cache** (not pre-cached from dev):
|
||||||
|
- [ ] Model download works with real-time progress
|
||||||
|
- [ ] Model loading works
|
||||||
|
- [ ] Generation produces valid audio
|
||||||
|
- [ ] No errors in stderr logs
|
||||||
|
|
||||||
|
### Phase 6: Final Verification
|
||||||
|
- [ ] Engine works in dev mode (`just dev`)
|
||||||
|
- [ ] Engine works in frozen binary (`just build` → run binary directly)
|
||||||
|
- [ ] Tested on target platform (macOS for MLX, Windows/Linux for CUDA)
|
||||||
|
- [ ] No regressions in existing engines
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user